Image processing method, apparatus and electronic device

Abstract

The application discloses an image processing method, device and electronic equipment, and belongs to the technical field of image processing. The image processing method comprises the following steps: in the case that a preview image is displayed on a shooting preview interface and a first object identification frame is displayed in the preview image, a first input is received; in response to the first input, a plurality of shooting images are acquired, and the plurality of shooting images are subjected to image fusion processing to obtain a fused image; in the preview image displayed on the shooting preview interface, a first preview image associated with the same shooting timestamp as a first shooting image is searched, the first shooting image being a reference frame in the plurality of shooting images subjected to the fusion processing; and the fused image is cropped according to first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image processing method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] With the continuous improvement of the shooting function of electronic equipment, the electronic equipment can cut and correct the image of a display target object to restore the real state of the target object. In related technologies, the electronic equipment displays a shooting preview interface, identifies a region where a target object is located in the shooting preview interface, and when a user triggers a shooting function, the electronic equipment shoots the target object displayed in the shooting preview interface, and then cuts and corrects the shot image according to the region identified in advance. Since there is a difference between the field of view angle of the image displayed in the shooting preview interface and the field of view angle of the shot image, cutting and correcting the shot image according to the region identified through the shooting preview interface results in low accuracy of cutting and correcting the target object, and the image after cutting and correcting cannot display the complete target object or display other objects in addition to the target object. SUMMARY

[0003] The embodiments of the application aim to provide an image processing method, device and electronic equipment, which can solve the problem of low accuracy of cutting and correcting an object displayed in a shot image.

[0004] In a first aspect, the embodiments of the application provide an image processing method, which comprises the following steps.

[0005] In the case that a preview image is displayed in a shooting preview interface and a first object in the preview image is identified with a frame, a first input is received.

[0006] In response to the first input, a plurality of shot images are acquired, and the plurality of shot images are subjected to image fusion processing to obtain a fused image.

[0007] In the preview image displayed in the shooting preview interface, a first preview image associated with the same shooting timestamp as a first shot image is searched, and the first shot image is a reference frame in the plurality of shot images subjected to the fusion processing.

[0008] The fused image is cut and corrected according to first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

[0009] In a second aspect, the embodiments of the application provide an image processing device, which comprises the following modules.

[0010] A first receiving module is configured to receive a first input in the case that a preview image is displayed in a shooting preview interface and a first object in the preview image is identified with a frame.

[0011] an image fusion module, configured to acquire a plurality of frames of shooting images in response to the first input, and perform image fusion processing on the plurality of frames of shooting images to obtain a fused image;

[0012] a searching module, configured to search, in a preview image displayed on the shooting preview interface, for a first preview image associated with the same shooting timestamp as the first shooting image, the first shooting image being a reference frame in the plurality of frames of shooting images that are subjected to the fusion processing;

[0013] a cutting correction module, configured to cut and correct the fused image according to the first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

[0014] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable on the processor. When the programs or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.

[0015] In a fourth aspect, a readable storage medium is provided. The readable storage medium stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method according to the first aspect are implemented.

[0016] In a fifth aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute programs or instructions to implement the method according to the first aspect.

[0017] In a sixth aspect, a computer program product is provided. The program product is stored in a storage medium. When the program product is executed by at least one processor, the method according to the first aspect is implemented.

[0018] In the embodiments of the present application, the preview image is displayed on the shooting preview interface, and the frame of the first object is identified in the preview image, which is beneficial for the user to expect the image obtained by shooting through the preview image. The plurality of frames of shooting images are acquired, and the image fusion processing is performed on the plurality of frames of shooting images to obtain the fused image, so as to improve the image quality of the fused image. The first preview image associated with the same shooting timestamp as the first shooting image is searched in the preview image displayed on the shooting preview interface, and the first coordinate information of the frame identified in the first preview image is used to cut and correct the fused image to obtain the corrected image of the first object. The consistent identification of the first object in the fused image and the first object in the preview image is implemented, the accuracy of cutting and correcting the first object displayed in the shooting image is improved, the step of identifying the first object in the shooting image after shooting is omitted, and the efficiency of generating the corrected image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application;

[0020] Figure 2 This is one of the partial flowcharts of an image processing method provided in one embodiment of this application;

[0021] Figure 3 This is one of the schematic diagrams showing the changes in the display interface provided in one embodiment of this application;

[0022] Figure 4 This is one of the partial flowcharts of an image processing method provided in one embodiment of this application;

[0023] Figure 5 This is the second schematic diagram showing the changes in the display interface provided in one embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in one embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] The image processing method, apparatus, electronic device, readable storage medium, and computer program product provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0030] Please see Figure 1The embodiment of the application provides an image processing method, which comprises the following steps:

[0031] S110, in the case that a preview image is displayed on a shooting preview interface and a first object in the preview image is identified with a frame, a first input is received;

[0032] The shooting preview interface is an interface used for viewing a shooting scene or picture in real time during a photographing or video shooting process, so as to facilitate a user to view a shooting effect, a shooting parameter and a composition in real time. When the electronic device is in a shooting preview mode, the camera module acquires a preview image at a preset frame rate and displays the preview image on the shooting preview interface for the user to browse.

[0033] When one or more objects are displayed in the preview image, a pre-stored object detection algorithm can identify a first object in the preview image, and generate a frame surrounding the first object according to the position of the identified first object in the preview image. The frame can be a regular polygon or an irregular line frame formed along the edge of the first object. When the user controls the camera module to move, the displayed preview image changes, the position of the first object in the preview image changes, and the identified frame also changes accordingly.

[0034] The first object can be paper, text in a screen, or static or dynamic articles such as clothes, utensils, household appliances, plants and animals. For example, when the first object is text, a pre-stored document detection algorithm identifies the text in the preview image, identifies a quadrilateral frame for the text, and the text surrounded by the frame can be converted into editable electronic text through an optical character recognition (OCR) technology. For example, when the first object is clothes, a pre-stored clothes detection algorithm identifies the clothes in the preview image, identifies an irregular frame for the clothes, and the clothes surrounded by the frame can be searched for similar clothes for sale through an image comparison technology. For example, when the first object is an animal, a pre-stored animal detection algorithm identifies the animal in the preview image, identifies an irregular frame for the animal, and the animal surrounded by the frame can be searched for a similar animal encyclopedia through an image comparison technology.

[0035] In the case that the electronic device displays the preview image and identifies the first object in the preview image with the frame, the first input can be a single click, double click, long press or other input made on a control, image or frame displayed on the shooting preview interface, or a specific sliding gesture or voice instruction. The first input is used to instruct the camera module to perform shooting imaging.

[0036] S120, in response to the first input, a plurality of shooting images are acquired, and the plurality of shooting images are subjected to image fusion processing to obtain a fusion image.

[0037] In response to the first input, the camera module is triggered to capture a plurality of captured images. The preview image is an image generated by the camera module with a first field of view for real-time display for the user to preview, and the captured image is an image captured by the camera module with a second field of view after the user observes and browses the image and triggers the camera module to capture and save the image. The first field of view of the preview image and the second field of view of the captured image can be different, so that the size and displayed content of the preview image and the captured image generated at the same time are different.

[0038] The image fusion processing can fuse multiple frames of captured images into a single image, so that the scene of the synthesized single image is more comprehensive and clear. By performing image fusion processing on multiple frames of captured images, a fused image is obtained to reduce or avoid blurring of the image used for subsequent cropping correction due to device shaking or movement of the first object during image capture.

[0039] Referring to Figure 2 In S120, the camera module can be triggered to capture n frames of preview images 21 and m frames of captured images 22, and then perform image fusion processing on the m frames of captured images 22 to obtain a fused image 23, which displays the first object. Based on the clarity of the image, the captured images with clarity greater than a preset value can be selected from the m frames of captured images 22 as first captured images; based on the position of the first object in the captured images 22, the first captured images in which the position of the first object in the captured images 22 is within a preset range can be selected from the m frames of captured images 22. The m frames of captured images 22 can be consecutive frames or non-consecutive frames. Alternatively, m is 4 or 8, and 4 or 8 frames of captured images are obtained for image fusion processing to avoid excessive calculation amount for image fusion processing.

[0040] In S130, in the preview image displayed on the shooting preview interface, a first preview image with the same shooting timestamp as the first captured image is searched, and the first captured image is a reference frame in the plurality of captured images for fusion processing.

[0041] When the preview image is displayed on the shooting preview interface, a plurality of frames of preview images are cached, and the shooting timestamp of the preview image is the time when the camera module generates the preview image. The shooting timestamp of the captured image is the time when the camera module generates the captured image. The first captured image and the first preview image are associated with the same shooting timestamp, indicating that they are generated by the camera module at the same time.

[0042] In S140, the first object is cropped and corrected according to the first coordinate information of the frame in the first preview image to obtain a corrected image of the first object.

[0043] The first coordinate information can be a coordinate with the first size information of the first preview image as a reference system. The first coordinate information can represent a position of the frame in the first preview image. In S110, the frame can be displayed in the shooting preview interface based on the first coordinate information. The cutting and correction of the fused image based on the first coordinate information includes, but is not limited to, cutting the fused image and correcting the perspective relationship of the first object. The cutting of the fused image can delete objects other than the first object in part of the fused image. The correction of the perspective relationship of the first object can restore the display of the first object to a front view angle.

[0044] Referring to Figure 3 , as shown in Figure 3 (a), a text 31 is displayed in the first preview image. The text 31 is identified, and a frame 32 is marked on the text 31. As shown in Figure 3 (b), the fused image is cut and corrected according to the first coordinate information of the frame 32, and a corrected image 33 displaying the text 31 is obtained.

[0045] In the image processing method provided in the present application, the preview image is displayed in the shooting preview interface, and the frame is marked on the first object in the preview image, which is beneficial for the user to expect the image obtained by shooting through the preview image. The multiple shooting images are obtained and image fusion processing is performed on the multiple shooting images to obtain the fused image, so as to improve the image quality of the fused image. The first preview image associated with the same shooting timestamp as the first shooting image is searched, the frame in the first preview image is cut and corrected according to the first coordinate information of the frame, and the corrected image of the first object is obtained. The consistent recognition of the first object in the fused image and the first object in the preview image is realized, the accuracy of cutting and correcting the first object displayed in the shooting image is improved, the step of identifying the first object in the shooting image after shooting is saved, and the efficiency of generating the corrected image is improved.

[0046] In some embodiments, after S130, the method further includes:

[0047] In S210, the first preview image is cut and corrected according to the first coordinate information, and a echo image displaying the first object is obtained.

[0048] In S220, a thumbnail of the echo image is displayed.

[0049] The echo image is an image used by the user to view and understand the general state of the corrected image before obtaining the corrected image of the first object. The echo image is cropped and corrected based on the first coordinate information of the first preview image. Correcting the image requires first performing image fusion processing on multiple captured images to obtain a fused image, and then cropping and correcting the fused image based on the first coordinate information. Therefore, compared to generating the corrected image, generating the echo image involves less computation and takes less time. Through S210, the echo image can be generated first for the user to review. Please refer to [link to relevant documentation]. Figure 4 Among the multiple preview images 21, the first preview image 21a is selected and cropped and corrected to obtain the echo image 34 displaying the first object.

[0050] In the S220, thumbnails of the returned images can be displayed in preset areas of the shooting preview interface or the shooting album interface for users to browse.

[0051] The steps of performing image fusion processing on multiple captured images in S120 to obtain a fused image and the steps of searching for a first preview image with the same capture timestamp associated with the first captured image in S130 can be performed simultaneously to improve the generation efficiency of the corrected image.

[0052] In some embodiments, after S220, the method further includes:

[0053] Receive the second input;

[0054] In response to the second input, an echo image is displayed.

[0055] The second input can be a thumbnail of the displayed image. For example, tapping, long-pressing, or two-finger movement on the thumbnail. The displayed image can be adapted to the screen size or displayed based on the original image's pixel size. Compared to the thumbnail, the displayed image has a larger size and greater clarity.

[0056] For example: Please refer to Figure 5 ,like Figure 5 As shown in (a), the shooting preview interface displays a preview image 41 and a thumbnail 43 of the playback image. The preview image 41 is marked with a border 42. The interface receives input by clicking on the thumbnail 43 of the playback image. Figure 5 As shown in (b), the echo image 44 is displayed.

[0057] In some embodiments, after S130, the method further includes:

[0058] S410, the first preview image is cropped and corrected according to the first coordinate information to obtain the echo image displaying the first object;

[0059] S420, displays echo image;

[0060] After S140, further comprising:

[0061] In the case that the echo image is displayed, switching to display the corrected image.

[0062] In some embodiments, after the echo image is generated, the echo image can be directly displayed. In other embodiments, after the echo image is generated, a thumbnail of the echo image can be directly displayed, and the echo image is displayed according to the received second input.

[0063] After S140, that is, the corrected image has been generated, in the case that the echo image is displayed, the echo image is closed, and the corrected image is displayed, so that the user can browse the corrected image which is clearer and more accurately cropped.

[0064] Optionally, after S140, further comprising: deleting the echo image.

[0065] In some embodiments, before S110, comprising:

[0066] S510, obtaining a preview image, identifying a first object displayed in the preview image to obtain first coordinate information of a frame of the first object;

[0067] S520, displaying the preview image on a shooting preview interface at a preset frame rate, and marking the frame in the preview image according to the first coordinate information.

[0068] Referring to Figure 2 In 510, the camera module obtains a preview image 21 at a preset frame rate. After obtaining each preview image 21, an object detection algorithm is used to identify a first object in the preview image, and a frame surrounding the first object is generated, that is, first coordinate information of the frame surrounding the first object is generated.

[0069] In 520, the preview image is displayed on the shooting preview interface at a preset frame rate, so that the displayed preview image is adapted to the displacement of the user operating the camera module. While the preview image is displayed, the frame is also drawn in the preview image according to the first coordinate information, so that the user can browse the preview image marked with the frame to determine whether the shooting angle of the camera module can shoot an image displaying the first object.

[0070] In some embodiments, after S510, comprising:

[0071] S610, storing at least one second preview image, a shooting time stamp of the second preview image, and first coordinate information of the frame marked in the second preview image in association, and the preview image displayed on the shooting preview interface includes the second preview image;

[0072] The at least one second preview image includes the first preview image; S130 includes:

[0073] S620, according to a shooting timestamp of the first shooting image, searching for the first preview image with the same shooting timestamp in the stored at least one second preview image.

[0074] The camera module generates a plurality of preview images, and displays the plurality of preview images in the shooting preview interface in sequence after performing first object recognition processing on the preview images. The second preview image is one of the plurality of preview images displayed in the shooting preview interface, that is, the second preview image is one of the plurality of preview images generated by the shooting module.

[0075] The j second preview images can be selected from the i preview images for caching processing based on a preset screening rule. The preset screening rule can be to set the preview images with an interval of a preset number of frames as the second preview images, or to set the preview images with a clarity greater than a preset clarity value as the second preview images.

[0076] The shooting timestamp of the second preview image is the time when the second preview image is generated. The first coordinate information of the frame in the second preview image, that is, the first coordinate information of the frame of the first object displayed in the second preview image. The second preview image, the shooting timestamp, and the first coordinate information are stored in association, so as to facilitate querying the first coordinate information consistent with the position of the first shooting image through the shooting timestamp.

[0077] The plurality of preview images and the plurality of shooting images are generated in time sequence by the camera module. Therefore, the shooting timestamps of the preview images are all different, and the shooting timestamps of the shooting images are all different. The preview images or the shooting images can be distinguished through the shooting timestamps. At the same time, the relative position of the camera module and the first object remains fixed, that is, the relative position of the camera module and the first object is consistent when the preview images and the shooting images with the same shooting timestamp are generated. Therefore, the first preview image with the same shooting timestamp is searched for in the stored at least one second preview image, and the first coordinate information generated according to the first preview image can be applied to the first shooting image, thereby saving the calculation of identifying the first object in the first shooting image and generating the frame of the first object.

[0078] In some embodiments, S120 includes:

[0079] S710, acquiring a plurality of shooting images, and selecting the first shooting image from the plurality of shooting images;

[0080] S720, taking the first shooting image as a reference frame, performing image fusion processing on the plurality of shooting images to obtain a fusion image.

[0081] The multiple frames of captured images each have a preview image with the same capture timestamp, but the preview image with the same capture timestamp as the captured image is not necessarily set as the second preview image for caching. Therefore, the captured image corresponding to the second preview image for caching can be selected as the first captured image from the multiple frames of captured images. In addition, the first captured image can be selected as the one with the highest clarity of the first object from the multiple frames of captured images.

[0082] The feature information of the images other than the first captured image in the multiple frames of captured images is acquired, and the first captured image is adjusted according to the feature information, so that the obtained fusion image is clear.

[0083] In some embodiments, S140 includes:

[0084] S810, converting the first coordinate information into second coordinate information conforming to the size information of the first captured image according to the size information of the first preview image and the size information of the first captured image;

[0085] S820, cutting the rectified fusion image according to the second coordinate information to obtain a rectified image of the first object.

[0086] The size information of the multiple preview images can be the same, and optionally, the size information of the preview images matches the size information of the display screen of the electronic device. The size information of the multiple captured images can be the same, and the fusion image obtained based on the first captured image is consistent with the size information of each captured image. Optionally, the size information of the captured image can be 1080*1080 pixels, 4032*3024 pixels, 3264*2448 pixels, etc. The size information of the captured image can be greater than the size information of the preview image, so that the captured image can display more details of the object.

[0087] Since the size information of the preview image and the size information of the first captured image are not the same, directly using the first coordinate information of the frame of the first object in the first preview image to crop the fusion image will result in a too small rectified image, so the first coordinate information of the frame is converted to the same reference system as the size information of the first captured image to obtain second coordinate information, and then the fusion image is cut and rectified according to the second coordinate information, so that the accuracy of the obtained rectified image restoring the shape of the first object is high.

[0088] The image processing method provided in the embodiments of the present application can be executed by the image processing device. In the embodiments of the present application, the image processing device is taken as an example to illustrate the image processing device provided in the embodiments of the present application.

[0089] Please refer to Figure 6 , Figure 6is a structural schematic diagram of an image processing apparatus provided by another embodiment of the present application, as shown in the figure, the image processing apparatus can comprise: Figure 6

[0090] The first receiving module 601 is configured to receive a first input in a case where a preview image is displayed on a shooting preview interface and a first object identification frame in the preview image is displayed.

[0091] The image fusion module 602 is configured to acquire a plurality of shooting images in response to the first input, and perform image fusion processing on the plurality of shooting images to obtain a fusion image.

[0092] The searching module 603 is configured to search, in the preview image displayed on the shooting preview interface, a first preview image associated with the same shooting timestamp as the first shooting image, the first shooting image being a reference frame in the plurality of shooting images subjected to the fusion processing.

[0093] The cutting correction module 604 is configured to cut and correct the fusion image according to first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

[0094] In an optional example, the image processing apparatus further comprises:

[0095] The first echo image generation module is configured to cut and correct the first preview image according to the first coordinate information to obtain an echo image displaying the first object.

[0096] The first display module is configured to display a thumbnail of the echo image.

[0097] In an optional example, the image processing apparatus further comprises:

[0098] The second receiving module is configured to receive a second input.

[0099] The second display module is configured to display the echo image in response to the second input.

[0100] In an optional example, the image processing apparatus further comprises:

[0101] The second echo image generation module is configured to cut and correct the first preview image according to the first coordinate information to obtain an echo image displaying the first object.

[0102] The third display module is configured to display the echo image.

[0103] The image processing apparatus further comprises:

[0104] The fourth display module is configured to switch to display the corrected image in a case where the echo image is displayed.

[0105] In an optional example, the image processing apparatus further comprises:​

[0106] An object recognition module is configured to acquire a preview image, recognize a first object displayed in the preview image, and obtain first coordinate information of a bounding box of the first object.

[0107] A fifth display module is configured to display the preview image on a shooting preview interface at a preset frame rate, and mark the bounding box in the preview image according to the first coordinate information.

[0108] In an optional example, the image processing apparatus further includes:

[0109] An association storage module is configured to store at least one second preview image, a shooting time stamp of the second preview image, and the first coordinate information of the bounding box marked in the second preview image in association, and the preview image displayed on the shooting preview interface includes the second preview image.

[0110] The at least one second preview image includes the first preview image, and the searching module 603 is configured to search, according to the shooting time stamp of the first shooting image, the at least one second preview image stored for the first preview image having the same shooting time stamp.

[0111] In an optional example, the image fusion module includes:

[0112] A selection unit is configured to acquire a plurality of shooting images, and select the first shooting image from the plurality of shooting images.

[0113] A fusion unit is configured to take the first shooting image as a reference frame, and perform image fusion processing on the plurality of shooting images to obtain a fusion image.

[0114] In an optional example, the cutting correction module 604 includes:

[0115] A coordinate conversion unit is configured to convert the first coordinate information into second coordinate information conforming to the size information of the first shooting image according to the size information of the first preview image and the size information of the first shooting image.

[0116] A cutting correction unit is configured to cut and correct the fusion image according to the second coordinate information to obtain a corrected image of the first object.

[0117] The image processing apparatus in the embodiments of the present applicationapplicationbe an electronic device or a component in an electronic device, for example, an integrated circuit or a chip. The electronic deviceapplicationbe a terminal or other device than a terminal. For example, the electronic deviceapplicationbe a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a kiosk, a self-service machine, or an AR / VR device, without limitation.

[0118] The extended reality device in the embodiments of the present applicationapplicationbe a device with an operating system. The operating systemapplicationbe an Android operating system, an ios operating system, or other possible operating system, without limitation.

[0119] The image processing apparatus provided in the embodiments of the present applicationapplicationimplement the method embodiments, and each process of the method embodimentsapplicationbe implemented, without repetition. Figures 1 to 5 The method embodimentsapplicationimplement each process of the method embodiments, without repetition.

[0120] Optionally, as shown in Figure 7 The embodiments of the present application further provide an electronic device 100, whichapplicationinclude a processor 101 and a memory 102. The memory 102applicationstore programs or instructions executable on the processor 101. The programs or instructionsapplicationbe executed by the processor 101 to implement each step of the above image processing method embodiments and achieve the same technical effects, without repetition.

[0121] It should be noted that the electronic device in the embodiments of the present applicationapplicationinclude the above mobile electronic device and non-mobile electronic device.

[0122] Figure 8 A hardware structure schematic diagram of an electronic device for implementing the embodiments of the present application.

[0123] The electronic device 1000 includes, but is not limited to, a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810, etc.

[0124] Those skilled in the art can understand that the electronic device 1000 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 810 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0125] The processor 810 is configured to receive a first input when a preview image is displayed on a shooting preview interface and a first object identification frame is displayed in the preview image.

[0126] The processor 810 is configured to, in response to the first input, acquire a plurality of shooting images, and perform image fusion processing on the plurality of shooting images to obtain a fusion image.

[0127] The processor 810 is configured to search for a first preview image associated with the same shooting timestamp as the first shooting image in the preview image displayed on the shooting preview interface, the first shooting image being a reference frame in the plurality of shooting images subjected to the fusion processing.

[0128] The processor 810 is configured to crop and correct the fusion image according to first coordinate information of the frame identified in the first preview image to obtain a corrected image of the first object.

[0129] Optionally, the processor 810 is configured to crop and correct the first preview image according to the first coordinate information to obtain a first object echo image.

[0130] The processor 810 is configured to display a thumbnail of the echo image.

[0131] Optionally, the processor 810 is configured to receive a second input.

[0132] The processor 810 is configured to display the echo image in response to the second input.

[0133] Optionally, the processor 810 is configured to crop and correct the first preview image according to the first coordinate information to obtain a first object echo image.

[0134] The processor 810 is configured to display the echo image.

[0135] The processor 810 is configured to switch to display the corrected image in a case of displaying the echo image.

[0136] Optionally, the processor 810 is configured to acquire a preview image, identify a first object displayed in the preview image, and obtain first coordinate information of a frame of the first object.

[0137] The processor 810 is configured to display the preview image on a shooting preview interface at a preset frame rate, and mark the frame in the preview image according to the first coordinate information.

[0138] Optionally, the processor 810 is configured to store at least one second preview image, a shooting time stamp of the second preview image, and the first coordinate information of the frame marked in the second preview image in association, and the preview image displayed on the shooting preview interface includes the second preview image.

[0139] The processor 810 is configured to search for the first preview image with the same shooting time stamp from the at least one second preview image stored according to the shooting time stamp of the first shooting image.

[0140] Optionally, the processor 810 is configured to acquire a plurality of shooting images, and select the first shooting image from the plurality of shooting images.

[0141] The processor 810 is configured to take the first shooting image as a reference frame, perform image fusion processing on the plurality of shooting images, and obtain a fused image.

[0142] Optionally, the processor 810 is configured to convert the first coordinate information into second coordinate information conforming to the size information of the first shooting image according to the size information of the first preview image and the size information of the first shooting image.

[0143] The processor 810 is configured to crop the corrected fused image according to the second coordinate information, and obtain a corrected image of the first object.

[0144] It should be understood that in the embodiments of the present application, the input unit 804 can include a graphics processing unit (GPU) 8041 and a microphone 8042. The graphics processing unit 8041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 806 can include a display panel 8061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also referred to as a touch screen. The touch panel 8071 can include two parts of a touch detection device and a touch controller. The other input devices 8072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, and the like, which will not be described here.

[0145] The memory 809 can be used to store software programs and various data. The memory 809 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 809 can include a volatile memory or a non-volatile memory, or the memory 809 can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 809 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0146] The processor 810 can include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 810.

[0147] The embodiment of the present application further provides a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize the processes of the above-mentioned image processing method embodiments, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0148] The processor is a processor in the electronic device in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0149] The embodiment of the present application further provides a chip, and the chip includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run programs or instructions to realize the processes of the above-mentioned image processing method embodiments, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0150] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip chip, etc.

[0151] The embodiment of the present application provides a computer program product, and the program product is stored in a storage medium, and the program product is executed by at least one processor to realize the processes of the above-mentioned image processing method embodiments, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0152] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the method and apparatus of the present application can be carried out by more than one process, method, article, or apparatus either simultaneously, concurrently, or with intermediate steps missing or added. Also, the features described in relation to one example can be combined with features described in relation to other examples.

[0153] From the above description of the embodiments, it is apparent that the above-described method of the embodiments can be realized by means of software and the necessary universal hardware platform, of course, also by hardware, but in many cases the former is the better embodiment. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc), including a number of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method of each embodiment of the present application.

[0154] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-described specific embodiments, which are merely illustrative rather than restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope of protection of the claims.

Claims

1. An image processing method, characterized in that, include: When the preview image is displayed on the shooting preview interface and a border is marked for the first object in the preview image, the first input is received; In response to the first input, multiple frames of captured images are acquired, and the multiple frames of captured images are subjected to image fusion processing to obtain a fused image; In the preview images displayed on the shooting preview interface, find the first preview image that has the same shooting timestamp as the first shooting image. The first shooting image is the reference frame in the multi-frame shooting images that are fused. The fused image is cropped and corrected based on the first coordinate information of the border identified in the first preview image to obtain the corrected image of the first object.

2. The image processing method according to claim 1, characterized in that, After finding the first preview image with the same shooting timestamp associated with the first captured image, the method further includes: The first preview image is cropped and corrected based on the first coordinate information to obtain a display image of the first object. A thumbnail of the echoed image is displayed.

3. The image processing method according to claim 2, characterized in that, After displaying the thumbnail of the echoed image, the method further includes: Receive the second input; In response to the second input, the echo image is displayed.

4. The image processing method according to claim 1, characterized in that, After finding the first preview image with the same shooting timestamp associated with the first captured image, the method further includes: The first preview image is cropped and corrected based on the first coordinate information to obtain a display image of the first object. Display the echo image; After cropping and correcting the fused image based on the first coordinate information of the border identified in the first preview image to obtain a corrected image displaying the first object, the method further includes: When the echo image is displayed, switch to displaying the corrected image.

5. The image processing method according to claim 1, characterized in that, Before receiving the first input, when the preview image is displayed on the shooting preview interface and a border is marked for the first object in the preview image, the process includes: Acquire a preview image, identify the first object displayed in the preview image, and obtain the first coordinate information of the border of the first object; The preview image is displayed on the shooting preview interface at a preset frame rate, and the border is marked in the preview image according to the first coordinate information.

6. The image processing method according to claim 5, characterized in that, After acquiring the preview image, identifying the first object displayed in the preview image, and obtaining the first coordinate information of the border of the first object, the process includes: At least one second preview image, the shooting timestamp of the second preview image, and the first coordinate information of the border identified in the second preview image are associated and stored together. The preview image displayed on the shooting preview interface includes the second preview image. The at least one second preview image includes the first preview image; the process of finding the first preview image with the same shooting timestamp associated with the first captured image includes: Based on the shooting timestamp of the first captured image, search for a first preview image with the same shooting timestamp among the stored at least one second preview image.

7. The image processing method according to claim 1, characterized in that, The step of acquiring multiple frames of captured images and performing image fusion processing on the multiple frames of captured images to obtain a fused image includes: Acquire multiple frames of captured images, and select the first captured image from the multiple captured images; Using the first captured image as the reference frame, the multiple captured images are subjected to image fusion processing to obtain a fused image.

8. The image processing method according to claim 1, characterized in that, The step of cropping and correcting the fused image based on the first coordinate information of the border identified in the first preview image to obtain a corrected image displaying the first object includes: Based on the size information of the first preview image and the size information of the first captured image, the first coordinate information is converted into second coordinate information that conforms to the size information of the first captured image; The fused image is cropped and corrected based on the second coordinate information to obtain a corrected image displaying the first object.

9. An image processing apparatus, characterized in that, include: The first receiving module is used to receive a first input when a preview image is displayed on the shooting preview interface and a border is marked for a first object in the preview image; An image fusion module is used to respond to the first input, acquire multiple frames of captured images, and perform image fusion processing on the multiple frames of captured images to obtain a fused image; The search module is used to search for a first preview image that has the same shooting timestamp as the first captured image. The multi-frame captured images include the first captured image, and the preview images displayed on the shooting preview interface include the first preview image. The cropping and correction module is used to crop and correct the fused image according to the first coordinate information of the border identified in the first preview image, so as to obtain a corrected image displaying the first object.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as described in any one of claims 1-8.

Images