An image processing method, apparatus, smart device, and storage medium

Abstract

This application discloses an image processing method, apparatus, smart device, and storage medium. This application can acquire multiple original images, each including a moving subject and a background. The moving subject and background in each original image are separated to obtain a separated background and a separated moving subject for each original image. The separated backgrounds in each original image are aligned to obtain an aligned background image. Shooting parameters of the moving subject in each original image are acquired. The separated moving subjects in each original image are sorted according to the shooting parameters to obtain sorted moving subjects. The sorted moving subjects are then fused with the background image to obtain a target image. This solves the problem of illogical spatial arrangement of subjects in the processed photo when subjects overlap, improving the separation effect of moving subjects in images.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to an image processing method, apparatus, smart device, and storage medium. Background Technology

[0002] With the advancement of technology, photography devices are becoming increasingly common. To capture more interesting photos, users employ various post-processing techniques to achieve satisfactory results. However, for the average user, obtaining truly interesting images through post-processing is extremely difficult. For example, some mobile phone cameras have a "magic clone" function. This allows users to activate the function, fix the phone, and then have the subject move around to take four consecutive photos, which are then combined into one. While this method has some fun aspects, when subjects overlap, their spatial arrangement in the processed photo becomes illogical, failing to achieve the desired effect. Summary of the Invention

[0003] This application provides an image processing method, apparatus, smart device, and storage medium that can improve the image splitting effect of moving subjects in images.

[0004] To address the aforementioned technical problems, this application provides the following technical solutions:

[0005] This application provides an image processing method, including:

[0006] Acquire multiple raw images, wherein the raw images include the moving subject and the background;

[0007] The moving subject in each original image is separated from the background to obtain the separated background and the separated moving subject for each original image;

[0008] Align the separated backgrounds corresponding to each original image to obtain the aligned background image;

[0009] Obtain the shooting parameters of the moving subject in each original image;

[0010] The separated moving subjects corresponding to each original image are sorted according to the shooting parameters to obtain the sorted moving subjects;

[0011] The sorted moving subject is fused with the background image to obtain the target image.

[0012] In one embodiment, the step of separating the moving subject from the background in each original image to obtain the separated background and the separated moving subject for each original image includes:

[0013] Obtain the preset background extraction algorithm;

[0014] According to the preset background extraction algorithm, the moving subject in each original image is separated from the background to obtain the separated background and the separated moving subject corresponding to each original image.

[0015] In one embodiment, aligning the separated background corresponding to each original image to obtain an aligned background image includes:

[0016] Obtain the alignment matrix for each original image;

[0017] The separated background corresponding to each original image is aligned according to the alignment matrix to obtain the aligned background image.

[0018] In one embodiment, obtaining the alignment matrix for each original image includes:

[0019] Extract feature points from each original image;

[0020] The feature points of each original image are matched to solve for the alignment matrix of each original image.

[0021] In one embodiment, the shooting parameters include depth-of-field information and shooting time information. The step of sorting the separated moving subjects corresponding to each original image according to the shooting parameters to obtain sorted moving subjects includes:

[0022] If the depth information is the same, then obtain the shooting time information corresponding to each original image with the same depth of field;

[0023] Based on the shooting time information, the moving subjects are sorted in chronological order;

[0024] If the depth information is different, the moving subjects are sorted from far to near according to the depth information.

[0025] In one embodiment, the step of fusing the sorted moving subject with the background image to obtain the target image includes:

[0026] Obtain the relative coordinates of the moving subject and the background in each photograph;

[0027] The sorted moving subject and the background image are fused together based on the relative coordinate information to obtain the target image.

[0028] In one embodiment, after fusing the sorted moving subject with the background image to obtain the target image, the method further includes:

[0029] Get the dimensions of each original image;

[0030] The maximum intersection region of all original images in the target image is determined based on the size and the alignment matrix.

[0031] The target image is cropped according to the maximum intersection region.

[0032] In one embodiment, after fusing the sorted moving subject with the background image to obtain the target image, the method further includes:

[0033] In response to a received cutting instruction, the cutting size and cutting ratio corresponding to the cutting instruction are obtained;

[0034] The target image is cropped within the maximum intersection area according to the cropping size and the cropping ratio.

[0035] According to one aspect of this application, an image processing apparatus is also provided, comprising:

[0036] The first acquisition module is used to acquire multiple original images, wherein the original images include a moving subject and a background;

[0037] The separation module is used to separate the moving subject from the background in each original image to obtain the separated background and the separated moving subject for each original image.

[0038] The alignment module is used to align the separated background corresponding to each original image to obtain an aligned background image;

[0039] The second acquisition module is used to acquire the shooting parameters of the moving subject in each original image;

[0040] The sorting module is used to sort the separated moving subjects corresponding to each original image according to the shooting parameters, so as to obtain the sorted moving subjects;

[0041] The fusion module is used to fuse the sorted moving subject with the background image to obtain the target image.

[0042] According to one aspect of this application, a smart device is also provided, including a processor and a memory, wherein the memory stores a computer program, and the processor executes any of the image processing methods provided in the embodiments of this application when it invokes the computer program in the memory.

[0043] According to one aspect of this application, a storage medium is also provided for storing a computer program, which is loaded by a processor to execute any of the image processing methods provided in the embodiments of this application.

[0044] In this embodiment, the smart device acquires multiple original images, each including a moving subject and a background. The moving subject and background in each original image are separated to obtain a separated background and a separated moving subject for each original image. The separated backgrounds in each original image are aligned to obtain an aligned background image. The shooting parameters of the moving subject in each original image are acquired. The separated moving subjects in each original image are sorted according to the shooting parameters to obtain sorted moving subjects. The sorted moving subjects are then fused with the background image to obtain a target image. This method, by sorting the separated moving objects in the original images according to the shooting parameters and fusing the sorted moving subjects with the background image to obtain the target image, avoids distant moving objects obscuring nearby moving objects, thus preventing images that violate visual perspective principles. Instead, it sorts the moving subjects according to a logical arrangement, improving the separation effect of the moving subject in the image. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0047] Figure 2 This is another schematic flowchart of the image processing method provided in the embodiments of this application;

[0048] Figure 3 This is another schematic flowchart of the image processing method provided in the embodiments of this application;

[0049] Figure 4 This is another schematic flowchart of the image processing method provided in the embodiments of this application;

[0050] Figure 5 This is a schematic diagram of image cropping provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the image processing apparatus provided in the embodiments of this application;

[0052] Figure 7 This is a schematic diagram of the structure of the smart device provided in the embodiments of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This application provides an image processing method, apparatus, smart device, and storage medium (i.e., computer-readable storage medium). The image processing method may include an image processing apparatus, which may be integrated into a smart device. This smart device can communicate with a server or terminal, etc. The server may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited thereto. The server and terminal can be directly or indirectly connected via wired or wireless communication, which is not limited herein. The terminal may be a mobile phone, computer, home appliance, or wearable device, etc.

[0055] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0056] In this embodiment, the description will be from the perspective of a smart device. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application. The image processing method may include:

[0057] S10. Acquire multiple raw images, including the moving subject and the background;

[0058] In this embodiment, the image processing method is executed by a smart device, which can be a mobile electronic device or a non-mobile electronic device, such as a mobile phone, camera, tablet, computer vehicle terminal, or wearable device. Optionally, the smart device includes an image acquisition device, which can be a camera. There can be at least two cameras, which can be fixed or rotatable. In this embodiment, to obtain the effect of a moving subject appearing multiple times in the image, the user uses an application on the smart device to open the image acquisition device and capture multiple original images of the environment in which the smart device is located. The original images include a moving subject and a background, where the moving subject can be a person or object moving within a certain distance range, and the background can be a fixed person or object.

[0059] For example, a user can point the camera at a person or object moving within a certain distance range and take multiple original images continuously based on the movement position of the moving subject. Alternatively, the user can set a preset time interval on the smart device, and the smart device can take multiple original images at each preset time interval. The preset time interval can be 1 second or can be set according to actual needs.

[0060] S20. Separate the moving subject from the background in each original image to obtain the separated background and the separated moving subject for each original image.

[0061] Optionally, step S20 may include: obtaining a preset background extraction algorithm;

[0062] Based on the preset background extraction algorithm, the moving subject in each original image is separated from the background to obtain the separated background and the separated moving subject for each original image.

[0063] Preset background extraction algorithms can include frame difference, background extraction, optical flow (sparse optical flow, dense optical flow), motion competition, motion templates, or temporal entropy algorithms. Among these, background extraction algorithms can be GMM (Gaussian Mixture Model) or ViBe (visual background extractor). For example, GMM is a background representation method based on pixel sample statistics. It constructs a color distribution model for each pixel in the video based on the temporal probability statistics (mean / variance) of each pixel, thereby achieving background modeling. After background modeling, statistical difference is used to determine target pixels to detect foreground objects, and the background model is continuously updated with target pixels to achieve modeling and fitting of dynamic backgrounds. ViBe is a pixel-level background modeling and foreground detection algorithm. Its main difference lies in the background model update strategy: it randomly selects samples of pixels to be replaced and randomly selects neighboring pixels for updating. When the model of pixel changes cannot be determined, the random update strategy can, to some extent, simulate the uncertainty of pixel changes. Optionally, according to a preset background extraction algorithm, the moving subject in each original image is separated from the background to obtain the feature points of the separated background and the feature points of the separated moving subject corresponding to each original image.

[0064] S30. Align the separated backgrounds corresponding to each original image to obtain the aligned background image;

[0065] To obtain a cloned image of a moving object, the backgrounds of multiple original images need to be aligned and merged into a single background image, and the moving object is then fitted onto the corresponding background image. Optionally, a preset alignment algorithm is obtained, which can be the RANSAC algorithm (Random Sample Consensus). The separated backgrounds are then aligned and stitched together according to the alignment algorithm to obtain the aligned background image.

[0066] S40. Obtain the shooting parameters of the moving subject in each original image;

[0067] The shooting parameters may include depth information and shooting time information of the original image. It should be noted that in this embodiment, the smart device includes at least two cameras to obtain depth information of the original image during image capture.

[0068] S50. Sort the separated moving subjects corresponding to each original image according to the shooting parameters to obtain the sorted moving subjects;

[0069] Optionally, intelligent devices can sort the separated moving subjects in the original image based on depth information, thereby avoiding distant moving objects from obscuring nearby moving objects and forming images that violate the principles of visual perspective. Instead, the moving subjects are sorted according to a reasonable ordering relationship, improving the separation effect of moving subjects in the image.

[0070] S60. The sorted moving subject and the background image are fused together to obtain the target image.

[0071] Optionally, in this embodiment, step S60 may include:

[0072] Obtain the relative coordinates of the moving subject and the background in each photo;

[0073] The sorted moving subject and the background image are fused together based on the relative coordinate information to obtain the target image.

[0074] After sorting the moving subjects, the sorted moving subjects are then fused onto the background image based on the relative coordinates between the moving subjects and the background in each original image, thus obtaining the target image, which is a clone of the moving subject.

[0075] It is understandable that during the photography process, slight shaking may occur when the photographer holds the device, or the camera's viewing angle may be moved within a small range, resulting in different perspectives for each original image. Therefore, in order to accurately synthesize the images during the image compositing stage, it is necessary to align multiple original images. Optionally, in this embodiment, after obtaining multiple original images in step S10, the first original image can be selected as the reference image according to the shooting time sequence, and the alignment matrix of each photo can be calculated using an alignment algorithm to align the original images.

[0076] In the technical solution provided in this embodiment, the intelligent device acquires multiple original images, each including a moving subject and a background. The moving subject and background in each original image are separated to obtain a separated background and a separated moving subject for each original image. The separated backgrounds in each original image are aligned to obtain an aligned background image. The shooting parameters of the moving subject in each original image are acquired. The separated moving subjects in each original image are sorted according to the shooting parameters to obtain sorted moving subjects. The sorted moving subjects are then fused with the background image to obtain the target image. In this way, the intelligent device sorts the separated moving objects in the original images according to the shooting parameters and fuses the sorted moving subjects with the background image to obtain the target image. This avoids distant moving objects obscuring nearby moving objects, thus preventing images that violate visual perspective principles. Instead, the moving subjects are sorted according to a logical arrangement, improving the separation effect of the moving subject in the image.

[0077] like Figure 2 , Figure 2 This is another schematic flowchart of the image processing method provided in this application embodiment. Step S30 may include:

[0078] S31. Obtain the alignment matrix for each original image;

[0079] Optionally, step S31 includes:

[0080] Extract feature points from each original image;

[0081] The feature points of each original image are matched to solve for the alignment matrix of each original image.

[0082] The feature points can be ORB (Oriented Fast and Rotated Brief) feature points.

[0083] S32. Align the separated background corresponding to each original image according to the alignment matrix to obtain the aligned background image.

[0084] Optionally, an alignment algorithm is obtained, which may be the RANSAC algorithm (Random Sample Consensus). Based on the alignment algorithm and the alignment matrix, the separated background corresponding to each original image is aligned and stitched together to obtain the aligned background image.

[0085] In the technical solution provided in this embodiment, the intelligent device acquires the alignment matrix of each original image; based on the alignment matrix, it performs alignment processing on the separated background corresponding to each original image to obtain an aligned background image. In this way, the intelligent device can accurately align and stitch the separated background images from each original image, facilitating the accurate fusion of moving objects into the background image later, thus improving the image processing effect.

[0086] like Figure 3 , Figure 3 This is another schematic flowchart of the image processing method provided in this application embodiment. The shooting parameters include depth-of-field information and shooting time information. Step S50 may include:

[0087] S51. If the depth of field information is the same, obtain the shooting time information corresponding to each original image with the same depth of field.

[0088] S52. Based on the shooting time information, sort the moving subjects in order from first to last;

[0089] Understandably, when smart devices acquire raw images, moving subjects may move back and forth or left and right in the background environment and may assume different poses. To avoid distant moving subjects obscuring nearby moving subjects, or later poses obscuring earlier poses, during post-processing fusion of moving subjects, it is necessary to acquire the depth-of-field information and shooting time information of each captured raw image and sort the moving subjects according to the depth-of-field information and shooting time information.

[0090] Optionally, if the depth information in the original images is the same, the moving subjects are sorted in order from first to last according to the shooting time information, so that the moving subjects captured later are on top of the layers of the moving subjects captured earlier, and the poses of the moving subjects in the image can be presented in a continuous chronological order, thus improving the effect of splitting the moving subjects in the image.

[0091] S53. If the depth information is different, then the moving subjects shall be sorted in order from far to near according to the depth information.

[0092] Optionally, the shooting parameters may also include height information, depth information, or three-dimensional position information. The shooting parameters can be set according to actual needs so that the moving subjects captured can be sorted as needed, thereby avoiding incorrect occlusion between the moving subjects.

[0093] In the technical solution provided in this embodiment, the intelligent device acquires the shooting parameters of the original image, including depth information and shooting time information. If the depth information is the same, the shooting time information corresponding to each original image with the same depth of field is acquired. Based on the shooting time information, the moving subjects are sorted in chronological order. If the depth information is different, the moving subjects are sorted in chronological order from far to near based on the depth information. In this way, the intelligent device sorts the separated moving objects corresponding to the original image according to the depth information and shooting time information, and then fuses the sorted moving subjects with the background image to obtain the target image. This avoids distant moving objects obscuring nearby moving objects, thus preventing the formation of images that violate the principles of visual perspective. Instead, the moving subjects are sorted according to a reasonable ordering relationship, improving the separation effect of moving subjects in the image.

[0094] like Figure 4 and Figure 5 , Figure 4 This is another schematic flowchart of the image processing method provided in the embodiments of this application. Figure 5 This is a schematic diagram of image cropping provided in an embodiment of this application. After step S60, it may further include:

[0095] S70. Obtain the dimensions of each original image;

[0096] S80. Determine the maximum intersection region of all original images in the target image based on the size and alignment matrix;

[0097] The region of maximum intersection is the maximum cropping region of the target image. (Refer to...) Figure 5 The dimensions and corresponding alignment matrices of the original images 201, 202, 203, and 204 are obtained. Based on the alignment matrices and alignment algorithm, the separated backgrounds from multiple original images are aligned and stitched together to obtain the background image. Each original image is sorted chronologically from left to right. Original image 202 has a closer depth of field than original image 201; therefore, the moving subjects in original image 201 are sorted first, and the moving subjects in original image 202 are sorted last, meaning the moving subjects in 202 are at the top of the layer. The sorted moving subjects are then fused with the background image to obtain the target image. The maximum intersection region of all original images is determined based on the dimensions of each original image and its corresponding alignment matrix. Figure 5 The area within the box corresponding to the dashed line.

[0098] S90. Crop the target image according to the maximum intersection area.

[0099] The target image is cropped based on the region of maximum intersection to obtain the cropped image. The cropped image is then displayed for users to manually process.

[0100] Optionally, in this embodiment, after step S80, the following may also be included:

[0101] In response to the received cutting instruction, obtain the cutting size and cutting ratio corresponding to the cutting instruction;

[0102] The target image is cropped within the area of ​​maximum intersection according to the cropping size and cropping ratio.

[0103] After the smart device displays the cropped image (cropped according to the maximum cropping area) on the display interface, the user can determine the cropping size and cropping ratio based on the cropped image on the display interface, generate a cropping instruction based on the cropping size and cropping ratio, and the smart device will crop the target image within the maximum intersection area according to the cropping size and cropping ratio corresponding to the cropping instruction to obtain the final image.

[0104] In the technical solution provided in this embodiment, the size of each original image is obtained; the maximum intersection area of ​​all original images in the target image is determined based on the size and alignment matrix; and the target image is cropped according to the maximum intersection area. In this way, the intelligent device can reasonably crop the target image to obtain an image in which the moving subject is displayed in proportion to a reasonable ratio, thus improving the effect of splitting the moving subject in the image.

[0105] To facilitate better implementation of the image processing method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described image processing method. The meanings of the terms used are the same as in the image processing method described above, and specific implementation details can be found in the descriptions in the method embodiments.

[0106] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device 300 may include a first acquisition module 301, a separation module 302, an alignment module 303, a second acquisition module 304, a sorting module 305, and a fusion module 306, etc.

[0107] The first acquisition module 301 is used to acquire multiple original images, which include a moving subject and a background.

[0108] The separation module 302 is used to separate the moving subject from the background in each original image to obtain the separated background and the separated moving subject corresponding to each original image;

[0109] Alignment module 303 is used to align the separated background corresponding to each original image to obtain an aligned background image;

[0110] The second acquisition module 304 is used to acquire the shooting parameters of the moving subject in each original image;

[0111] The sorting module 305 is used to sort the separated moving subjects corresponding to each original image according to the shooting parameters, so as to obtain the sorted moving subjects;

[0112] The fusion module 306 is used to fuse the sorted moving subject with the background image to obtain the target image.

[0113] Optionally, the image processing apparatus may further include the following modules not shown in the figures:

[0114] The third acquisition module is used to acquire the preset background extraction algorithm;

[0115] The separation module is used to separate the moving subject from the background in each original image according to a preset background extraction algorithm, so as to obtain the separated background and the separated moving subject for each original image.

[0116] The fourth acquisition module is used to acquire the alignment matrix of each original image;

[0117] The alignment module is used to align the separated background corresponding to each original image according to the alignment matrix to obtain the aligned background image.

[0118] The extraction module is used to extract feature points from each original image;

[0119] The solver module is used to match the feature points of each original image and solve for the alignment matrix of each original image.

[0120] The fifth acquisition module is used to acquire the shooting time information corresponding to each original image with the same depth of field if the depth of field information is the same.

[0121] The first sorting module is used to sort the moving subjects in chronological order according to the shooting time information;

[0122] The second sorting module is used to sort moving subjects in order from far to near based on the depth information if the depth information is different.

[0123] The sixth acquisition module is used to acquire the relative coordinate information of the moving subject and the background in each photo;

[0124] The fusion module is used to fuse the sorted moving subject with the background image based on relative coordinate information to obtain the target image.

[0125] The seventh acquisition module is used to acquire the dimensions of each original image;

[0126] The determination module is used to determine the maximum intersection region of all original images in the target image based on the size and alignment matrix.

[0127] The first cropping module is used to crop the target image according to the maximum intersection area.

[0128] The eighth acquisition module is used to obtain the cutting size and cutting ratio corresponding to the received cutting instruction in response to the cutting instruction;

[0129] The second cropping module is used to crop the target image within the maximum intersection area according to the cropping size and cropping ratio.

[0130] This application also provides a smart device, which can be a terminal such as a mobile phone, etc. Figure 7 As shown, it illustrates a structural schematic diagram of the smart device involved in the embodiments of this application, specifically:

[0131] The intelligent device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 7 The smart device structure shown does not constitute a limitation on the smart device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0132] The processor 401 is the control center of the smart device. It connects various parts of the smart device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the smart device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0133] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the smart device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0134] The smart device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0135] The smart device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0136] Although not shown, the smart device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the smart device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0137] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the image processing method above, which will not be repeated here.

[0138] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a smart device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the smart device to perform the methods provided in the various optional implementations of the above embodiments.

[0139] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by computer instructions, or by controlling related hardware through computer instructions. These computer instructions can be stored in a storage medium and loaded and executed by a processor. The storage medium is a computer-readable storage medium. Therefore, embodiments of this application provide a storage medium storing a computer program, which may include computer instructions. This computer program can be loaded by a processor to execute any of the image processing methods provided in the embodiments of this application.

[0140] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0141] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0142] Since the computer instructions stored in the storage medium can execute any of the image processing methods provided in the embodiments of this application, the beneficial effects that any of the image processing methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0143] The foregoing has provided a detailed description of an image processing method, apparatus, smart device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized in that, The image processing method includes: Acquire multiple raw images, each raw image including a moving subject and a background; wherein the moving subject is a person or object moving within a certain distance range, and the background is a fixed person or object; The moving subject in each original image is separated from the background to obtain the separated background and the separated moving subject for each original image; The separated backgrounds of each original image are aligned and merged to obtain an aligned and merged stitched background image; Obtain the shooting parameters of the moving subject in each original image; The separated moving subjects corresponding to each original image are sorted according to the shooting parameters to obtain sorted moving subjects. The shooting parameters include depth information and shooting time information. The process of sorting the separated moving subjects corresponding to each original image according to the shooting parameters to obtain sorted moving subjects includes: if the depth information is the same, obtaining the shooting time information corresponding to each original image with the same depth of field; sorting the moving subjects according to the shooting time information in chronological order; if the depth information is different, sorting the moving subjects according to the depth information in order from far to near. The sorted moving subject is fused with the stitched background image to obtain the target image.

2. The image processing method according to claim 1, characterized in that, The process of separating the moving subject from the background in each original image to obtain the separated background and the separated moving subject for each original image includes: Obtain the preset background extraction algorithm; According to the preset background extraction algorithm, the moving subject in each original image is separated from the background to obtain the separated background and the separated moving subject corresponding to each original image.

3. The image processing method according to claim 1, characterized in that, The step of aligning and fusing the separated background corresponding to each original image to obtain an aligned and fused background image includes: Obtain the alignment matrix for each original image; The separated background of each original image is aligned and fused according to the alignment matrix to obtain the stitched background image after alignment and fusion.

4. The image processing method according to claim 3, characterized in that, The step of obtaining the alignment matrix for each original image includes: Extract feature points from each original image; The feature points of each original image are matched to solve for the alignment matrix of each original image.

5. The image processing method according to claim 1, characterized in that, The step of fusing the sorted moving subject with the stitched background image to obtain the target image includes: Obtain the relative coordinates of the moving subject and the background in each photograph; The sorted moving subject and the stitched background image are fused together based on the relative coordinate information to obtain the target image.

6. The image processing method according to claim 3, characterized in that, After fusing the sorted moving subject with the stitched background image to obtain the target image, the method further includes: Get the dimensions of each original image; The maximum intersection region of all original images in the target image is determined based on the size and the alignment matrix. The target image is cropped according to the maximum intersection region.

7. The image processing method according to claim 6, characterized in that, After fusing the sorted moving subject with the stitched background image to obtain the target image, the method further includes: In response to a received cutting instruction, the cutting size and cutting ratio corresponding to the cutting instruction are obtained; The target image is cropped within the maximum intersection area according to the cropping size and the cropping ratio.

8. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire multiple original images, wherein the original images include a moving subject and a background; wherein the moving subject is a person or object moving within a certain distance range, and the background is a fixed person or object; The separation module is used to separate the moving subject from the background in each original image to obtain the separated background and the separated moving subject for each original image. The alignment module is used to align and merge the separated backgrounds of each original image to obtain an aligned and merged stitched background image. The second acquisition module is used to acquire the shooting parameters of the moving subject in each original image; A sorting module is used to sort the separated moving subjects corresponding to each original image according to the shooting parameters to obtain sorted moving subjects. The shooting parameters include depth information and shooting time information. Sorting the separated moving subjects corresponding to each original image according to the shooting parameters to obtain sorted moving subjects includes: if the depth information is the same, obtaining the shooting time information corresponding to each original image with the same depth; sorting the moving subjects in chronological order according to the shooting time information; if the depth information is different, sorting the moving subjects in chronological order according to the depth information. The fusion module is used to perform stitching and fusion processing on the sorted moving subject and the stitched background image to obtain the target image.

9. A smart device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the image processing method as described in any one of claims 1 to 7 when it invokes the computer program in the memory.

10. A storage medium, characterized in that, The storage medium is used to store a computer program, which is loaded by a processor to execute the image processing method according to any one of claims 1 to 7.

Images