Image processing method, image processing device and storage medium

Abstract

The present disclosure relates to an image processing method, an image processing device and a storage medium. The image processing method includes: obtaining a media image to be processed, wherein the media image to be processed displays a target scene object that is at least partially obscured; determining a target obstructing object that obscures the target scene object in the media image to be processed; and performing filling processing on the target scene object in a first area corresponding to the target obstructing object in the media image to be processed to generate a target media image. The image processing method disclosed in the present disclosure can ensure that the user obtains a media image that removes the target obstructing object that obscures the target scene object and is close to the real target scene object, thereby improving the user's experience and satisfaction during use.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing device and a storage medium. Background Art

[0002] With the development of terminal technology, various terminals with image shooting functions are becoming more and more popular, such as cameras, smart phones, tablet computers, smart wearable devices, etc. The popularity of terminals with image shooting functions provides convenience for people to take photos anytime and anywhere, making photography a part of people's daily life, especially when traveling or passing by certain scenic spots and historical sites, everyone wants to take a photo as a souvenir.

[0003] However, when taking photos in scenic spots, due to the large number of tourists in scenic spots, especially during tourist seasons, various other irrelevant tourists are often included in the photos. That is, when users take photos in scenic spots or have already taken photos, the photos include subject obstructing objects (such as the person to be photographed) in the field of view, and other obstructing objects (other people or objects) other than the subject obstructing objects are also included in the field of view of the photo. It is a pain point for users to want to take a photo of themselves with the scenic spots without interference from other people or objects. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, an image processing device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, which includes obtaining a media image to be processed, wherein the media image to be processed displays a target scene object that is at least partially obscured; determining a target occluded object that obscures the target scene object in the media image to be processed; and filling the target scene object in a first area corresponding to the target occluded object in the media image to be processed to generate a target media image.

[0006] In one embodiment, before filling the target scene object in the first area corresponding to the target occluded object in the media image to be processed and generating the target media image, the image processing method further includes: removing the first area corresponding to the target occluded object in the media image to be processed.

[0007] In another embodiment, the image processing method also includes: obtaining a scene image matching the target scene object from a cloud database; filling the target scene object in a first area corresponding to the target occluded object in the media image to be processed to generate a target media image, including: performing image registration on the scene image and the target scene object, and determining a second area corresponding to the first area in the scene image; and fusing the second area and the media image to be processed to obtain the target media image.

[0008] In one embodiment, fusing the second region and the media image to be processed may be fusing the image of the second region to the first region of the media image to be processed to obtain the target media image. In another embodiment, fusing the second region and the media image to be processed may be fusing the image of the second region to the corresponding region of the media image to be processed without the first region to obtain the target media image.

[0009] In another embodiment, the image processing method also includes: obtaining shooting parameters of the media image to be processed, the shooting parameters including one or more of shooting lighting information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information and shooting weather information; obtaining a scene image matching the target scene object from a cloud database, including: based on the shooting parameters, determining a subset of scene images matching the shooting parameters in a scene image set in the cloud database; and obtaining a scene image matching the target scene object in the scene image subset.

[0010] In another embodiment, in a scene image subset, a scene image that matches the target scene object is obtained, including: resetting the feature matching weights of scene objects other than the target scene object in the media image to be processed to zero, and determining, in the scene image subset, the scene image with the highest feature matching weight with the target scene object as the scene image that matches the target scene object.

[0011] In another embodiment, the target scene object is filled in the first area corresponding to the target occluded object in the media image to be processed to generate a target media image, including: based on a pre-trained neural network model, the first area corresponding to the target occluded object in the media image to be processed is repaired to generate a target media image.

[0012] In another embodiment, determining a target occluded object that occludes a target scene object in a media image to be processed includes: performing pixel segmentation on the occluded object that occludes the target scene object in the media image to be processed to obtain a two-dimensional mask image of the occluded object, wherein a region where each occluded object is located in the two-dimensional mask image has a respective class label; based on the class label, determining a region where a subject occluded object is located in the two-dimensional mask image; and determining other occluded objects in the two-dimensional mask image except for the region where the subject occluded object is located as target occluded objects.

[0013] In another embodiment, based on the class label, the area where the subject occluded object is located in the two-dimensional mask image is determined, including: based on a convolutional neural network, detecting the pixel area corresponding to the class label that meets the subject occluded object in all class labels as the area where the subject occluded object is located; or using the pixel area corresponding to the class label selected by the user as the area where the subject occluded object is located.

[0014] According to a second aspect of an embodiment of the present disclosure, an image processing device is provided, comprising: an acquisition module for acquiring a media image to be processed, wherein the media image to be processed displays a target scene object that is at least partially obscured; a target obstructing object determination module for determining a target obstructing object that obscures the target scene object in the media image to be processed; and an image processing module for performing filling processing on the target scene object in a first area corresponding to the target obstructing object in the media image to be processed to generate a target media image.

[0015] In one embodiment, the image processing device further includes a first region removal module, and the first region removal module is used to remove a first region corresponding to the target occluding object in the media image to be processed.

[0016] In another embodiment, the image processing device also includes a scene image matching module, which is used to: obtain a scene image that matches the target scene object from a cloud database; the image processing module is used to: perform image registration on the scene image and the target scene object, and determine a second area corresponding to the first area in the scene image; and fuse the second area and the media image to be processed to obtain the target media image.

[0017] In another embodiment, the image processing device also includes a shooting parameter acquisition module, which is used to: obtain shooting parameters of the media image to be processed, the shooting parameters including one or more of shooting lighting information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information and shooting weather information; the scene image matching module is used to: based on the shooting parameters, determine a scene image subset that matches the shooting parameters in the scene image set in the cloud database; and obtain a scene image that matches the target scene object in the scene image subset.

[0018] In another embodiment, the scene image matching module is used to reset the feature matching weights of other scene objects in the media image to be processed except the target scene object to zero, and determine, in the scene image subset, the scene image with the highest feature matching weight with the target scene object as the scene image that matches the target scene object.

[0019] In yet another embodiment, the image processing module is used to: based on a pre-trained neural network model, perform repair processing on a first area corresponding to the target occluded object in the media image to be processed to generate a target media image.

[0020] In another embodiment, the target occluded object determination module is used to: perform pixel segmentation on the occluded objects that occlude the target scene objects in the processed media image to obtain a two-dimensional mask image of the occluded objects, wherein the region where each occluded object is located in the two-dimensional mask image has its own class label; based on the class label, determine the region where the subject occluded object is located in the two-dimensional mask image; and determine the other occluded objects in the two-dimensional mask image except the region where the subject occluded object is located as the target occluded objects.

[0021] In another embodiment, the target occluded object determination module is used to: detect the pixel area corresponding to the class label that meets the subject occluded object in all class labels based on the convolutional neural network, as the area where the subject occluded object is located; or use the pixel area corresponding to the class label selected by the user as the area where the subject occluded object is located.

[0022] According to a third aspect of an embodiment of the present disclosure, an image processing device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to: execute the image processing method described in the first aspect of the present disclosure or any one of the embodiments of the first aspect.

[0023] According to the fourth aspect of an embodiment of the present disclosure, a temporary computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the image processing method described in the first aspect of the present disclosure or any one of the embodiments of the first aspect.

[0024] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by determining the target occluded object that occludes the target scene object in the media image to be processed, and filling the target scene object in the first area corresponding to the target occluded object to generate a target media image, it is ensured that the user can obtain a media image that removes the target occluded object that occludes the target scene object and is close to the real target scene object.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment;

[0028] Figure 2a A schematic diagram showing a photograph including an occluded target scene object;

[0029] Figure 2b A schematic diagram of a photo with a first area corresponding to a target occluding object removed is shown;

[0030] Figure 2c A schematic diagram of a scene image matching a target scene object is shown;

[0031] Figure 2d A schematic diagram showing a process of filling a target scene object after removing a first area from a media image to be processed;

[0032] Figure 3 A flowchart of another image processing method according to an exemplary embodiment;

[0033] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment;

[0034] Figure 5 is a flow chart showing a method of acquiring a scene image matching a target scene object from a cloud database according to an exemplary embodiment;

[0035] Figure 6 is a flow chart showing a method of determining a target occluding object that occludes the target scene object in a media image to be processed according to an exemplary embodiment;

[0036] Figure 7 is a schematic diagram of a process of an image processing method according to an exemplary embodiment;

[0037] Figure 8 is a block diagram of an image processing device according to an exemplary embodiment;

[0038] Fig. 9 The invention is a block diagram of a device for image processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] The image processing method provided by the embodiment of the present disclosure is applied to the scene where a target occluded object that occludes a target scene object is removed in a media image. For example, it is applied to the scene where an occluder that occludes a background image in a certain frame or several frames of a video needs to be removed, or it is applied to the scene where an occluder that occludes a scene in a photo taken in a scenic spot needs to be removed.

[0041] In the related art, the background portrait (i.e., the target occluded object) included in the background image (i.e., the target scene object) of the photo taken in the field of view is usually removed by image processing means such as (Photoshop, referred to as PS). For example, in order to obtain a photo of being alone with the scenic spot without interference from others, users can currently use post-processing methods such as PS to put themselves into the scenic spot. In the related art, the method of removing the background portrait (i.e., the target occluded object) by PS method has the following problems: 1) If you want to get a good PS photo, you need good PS technology, but most people do not have it. 2) PS needs to provide the source image of the person and the background, but the selected source image may not match, the effect after PS is poor, and it cannot be dynamically adjusted to select a satisfactory angle, etc. 3) From a psychological point of view, the photos obtained by PS are not fascinating, and they are not as memorable as the real photos taken on the spot. Therefore, removing the background portrait included in the background image of the photo taken in the field of view is still a difficult problem.

[0042] The present disclosure provides an image processing method, which determines a target occluded object that occludes a target scene object in a media image showing a target scene object that is at least partially occluded, and performs filling processing on an area corresponding to the target occluded object in the media image to be processed to generate a target media image. In one example, the occluded object area that occludes the background scenery in a real photo taken by a user is filled to obtain a photo without occluded objects, thereby ensuring that the user can obtain a photo without occluded objects and a photo close to the real background scenery, thereby improving the user experience and satisfaction.

[0043] The image processing method provided by the embodiments of the present disclosure may be applied to a terminal, which may be, for example, a mobile terminal such as a smart phone or a tablet computer having image capture and processing functions.

[0044] Figure 1 FIG. 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method includes the following steps.

[0045] In step S11, a media image to be processed is obtained, wherein the media image to be processed displays a target scene object that is at least partially blocked.

[0046] In the disclosed embodiments, the media image to be processed may include an image or a video. The image may be an image (e.g., a photo) obtained after being shot by a user, or a preview image formed by a user currently shooting. The video may be a video screen obtained after being recorded by a user, or a preview screen formed during a user's current recording process.

[0047] The media images to be processed obtained in the embodiments of the present disclosure include occluded target scene objects. For example, the target scene object may be a person, scenery, etc. that the user wants to take a photo or record in a video as a souvenir. The target scene object may be partially occluded or completely occluded. In the embodiments of the present disclosure, the obstruction that obstructs the target scene object is referred to as an occluding object, and the occluding object that is intended to be removed is referred to as a target occluding object. In one embodiment, the target scene object is partially occluded by the target occluding object.

[0048] In step S12, a target occluding object that occludes a target scene object in the media image to be processed is determined.

[0049] The target occluded object may be various other portraits, objects, environments, etc. that are not related to the target scene object.

[0050] In one example, the acquired media images to be processed may be photos taken in a scenic spot. In addition to the target scene objects that the user intends to shoot, the photos taken by the user in the scenic spot also include target obstructing objects that are irrelevant to the target scene objects and obstruct the target scene objects, such as other tourists, or other sundries that are not commensurate with the photographed scenery, such as trash cans, tower cranes, etc.

[0051] In one example, suppose a user needs to take a photo of the Forbidden City and user A. In the field of view of the photo, in addition to the target scene object, the Forbidden City, the entire view of the Forbidden City is also blocked by several other irrelevant tourists B and C and a tower crane D that is not in line with the scenery of the Forbidden City. Figure 2a shown. Figure 2aA schematic diagram of a photograph including an occluded target scene object is shown.

[0052] In step S13, the target scene object is filled in the first area corresponding to the target occluded object in the media image to be processed to generate a target media image.

[0053] In the disclosed embodiment, for the convenience of description, the area corresponding to the target occluded object in the to-be-processed media image is referred to as the first area. Taking the user intending to take a photo of the Forbidden City as an example, since the target occluded objects of the Forbidden City, namely, tourist B, tourist C, and crane D, have been determined to occlude the target scene object, the Forbidden City, the area corresponding to the above target occluded object in the Forbidden City photo can be determined as the first area.

[0054] For example, in the photo field of view, if the first area where tourist B is located corresponds to a window M in the Forbidden City. The first area where tourist C is located corresponds to a railing N in the Forbidden City. The first area where crane D is located corresponds to a step W in the Forbidden City. At this time, in the photo to be processed, the window M of the Forbidden City is filled in the first area of ​​tourist B. The railing N of the Forbidden City is filled in the first area of ​​tourist C. The step W of the Forbidden City is filled in the first area of ​​crane D. By performing filling processing on the first area in the media image to be processed, a target media image excluding the target occluded object can be obtained, that is, a photo of the target scene object Forbidden City and user A excluding the target occluded objects tourists B, tourist C and crane D are obtained.

[0055] In the embodiment of the present disclosure, the target media image is an image that does not include a target occluding object and has a real target scene object.

[0056] In one embodiment, before the target scene object is filled, the first area may be removed. In this way, the first area corresponding to the target occluded object in the to-be-processed media image may be effectively removed. Moreover, by filling the scene image matching the first area, the to-be-processed media image from which the first area has been removed can be filled with the real target scene object, thereby ensuring that the user can obtain the to-be-processed media image with the target occluded object occluding the target scene object removed and close to the real target scene object.

[0057] Figure 3 FIG. 1 is a flowchart of another image processing method according to an exemplary embodiment. Figure 3 As shown, in an exemplary embodiment of the present disclosure, the image processing method includes the following steps.

[0058] In step S21, a media image to be processed is obtained, wherein the media image to be processed displays a target scene object that is at least partially obscured.

[0059] In step S22, a target occluding object that occludes a target scene object in the media image to be processed is determined.

[0060] In step S23, a first area corresponding to the target occluding object in the media image to be processed is removed.

[0061] In the disclosed embodiment, the to-be-processed media image with the first region removed can be understood as an image with a portion of the target scene object that is not blocked by the target blocking object. In one example, the to-be-processed media image is a photo, and after removing the first region corresponding to the target blocking object, a photo with a hole and a real target scene object can be obtained.

[0062] In the embodiment of the present disclosure, the user intends to take a photo of the Forbidden City as an example. After the photo is acquired using the image processing method provided in the embodiment of the present disclosure, the target scene objects, the Forbidden City, tourist B, tourist C, and crane D are identified. After the irrelevant first regions corresponding to tourist B, tourist C, and crane D are removed from the photo to be processed, a photo of the Forbidden City with holes in the first regions corresponding to tourist B, tourist C, and crane D is obtained. Figure 2b shown. Figure 2b A schematic diagram of a photo from which a first area corresponding to a target occluding object is removed is shown.

[0063] In step S24, the target scene object is filled in the first area corresponding to the target occluded object in the media image to be processed to generate a target media image.

[0064] In one implementation of the disclosed embodiment, a scene image matching the target scene object may be stored in a cloud database, so that an image having the target scene object for filling the removed first area may be obtained from the cloud.

[0065] Figure 4 FIG. 1 is a flowchart of another image processing method according to an exemplary embodiment. Figure 4 As shown, in an exemplary embodiment of the present disclosure, the image processing method includes the following steps.

[0066] In step S31, a media image to be processed is obtained, wherein the media image to be processed displays a target scene object that is at least partially blocked.

[0067] In step S32, a target occluding object that occludes the target scene object in the media image to be processed is determined.

[0068] In step S33, a first area corresponding to the target occluding object in the media image to be processed is removed.

[0069] In step S34, a scene image matching the target scene object is obtained from the cloud database.

[0070] In the disclosed embodiment, the cloud database includes various scene images.

[0071] In one embodiment, the scene image of the cloud database may be pre-stored. The scene image may be a scene image without people or other debris, such as an unmanned background image. Furthermore, the unmanned background image stored in the cloud database may be updated in real time or at regular intervals to ensure the comprehensiveness and timeliness of the scene image stored in the cloud database.

[0072] In one embodiment, if the image processing method provided by the embodiment of the present disclosure is applied to a scene of photographing a scenic spot, the image maintenance of the cloud database can be carried out by cooperating with the management of each scenic spot to obtain scene images of each scenic spot in a timely manner. The scene images included in the cloud database may include scene images of each scenic spot at different times, different locations, and different perspectives.

[0073] In one example, the cloud database stores scene images of the Forbidden City. The cloud database may include scene images of the Forbidden City in early spring, late spring, early summer, late summer, early autumn, late autumn, early winter, and late winter, and may include scene images of the Forbidden City in different time periods of the day, such as morning, noon, and evening; scene images of the Forbidden City in different weather conditions, such as sunny days, cloudy days, rainy days, and snowy days; and scene images of the Forbidden City in directions such as east, west, south, and north.

[0074] In the disclosed embodiment, since the cloud database may include scene images of various forms or states of each scenic spot, the image matching technology in the related art can be used to find the scene image matching the target scene object in the cloud database.

[0075] In step S35, image registration is performed on the scene image and the target scene object, and a second area corresponding to the first area is determined in the scene image.

[0076] In the disclosed embodiment, based on the scene image of the selected matching target scene object, image registration is performed with the target scene object with the first area removed, so as to determine the area corresponding to the first area (hereinafter referred to as the second area) in the scene image. The second area is filled in the place where the first area is removed in the media image to be processed, thereby achieving real filling and repair of the target scene object, and obtaining a target media image with a real effect.

[0077] In the embodiment of the present disclosure, the user intends to take a photo of the Forbidden City as an example for explanation. After the scene image Q of the target scene object matching the photo taken by the user is screened out in the embodiment of the present disclosure, it is necessary to perform image registration between the scene image Q and the target scene object, and determine a second area corresponding to the first area in the scene image Q.

[0078] Since the first area of ​​tourist B corresponds to a window M in the Forbidden City; the first area of ​​tourist C corresponds to a railing N in the Forbidden City; and the first area of ​​crane D corresponds to a step W in the Forbidden City. Therefore, after image registration of scene image Q with the target scene object Forbidden City, it can be determined in scene image Q that window M of scene image Q is the second area corresponding to the first area of ​​tourist B; railing N of scene image Q is the second area corresponding to the first area of ​​tourist C; and step W of scene image Q is the second area corresponding to the first area of ​​crane D. Figure 2c shown. Figure 2c A schematic diagram of a scene image matching a target scene object is shown.

[0079] Since the scene image is a real background image without people or other debris that matches the target scene object, based on the second area in the scene image, the target scene object is filled in the area where the first area is removed in the real media image to be processed, which can ensure to a large extent that the media image to be processed has a good real effect.

[0080] In step S36 , the second region and the media image to be processed are fused to obtain a target media image.

[0081] In one embodiment, in the case where the first area is not removed from the media image to be processed, fusing the second area with the media image to be processed may be fusing the image of the second area to the corresponding first area in the media image to be processed to obtain the target media image. In another embodiment, in the case where the first area is removed from the media image to be processed, fusing the second area with the media image to be processed may be fusing the image of the second area to the corresponding area in the media image to be processed where the first area is removed to obtain the target media image.

[0082] In the disclosed embodiment, the user intends to take a photo of the scenery of the Forbidden City as an example for explanation. Figure 2d A schematic diagram is shown in which a target scene object is filled in after removing the first area from the media image to be processed. Figure 2d In the target media image obtained after the final processing, the target scene objects including the window M, the railing N, and the steps W (the unobstructed background image of the Forbidden City).

[0083] The following embodiment of the present disclosure describes the process of obtaining a scene image matching a target scene object from a cloud database in combination with actual applications.

[0084] Figure 5 The present invention is a flowchart showing a method of obtaining a scene image matching a target scene object from a cloud database according to an exemplary embodiment.

[0085] like Figure 5 As shown, in an exemplary embodiment of the present disclosure, obtaining a scene image matching a target scene object from a cloud database includes the following steps.

[0086] In step S41 , the shooting parameters of the media image to be processed are obtained.

[0087] For the same scene, different shooting parameters will result in different scene images. Therefore, the shooting parameters will affect the scene image that matches the target scene object.

[0088] Taking the Forbidden City as an example, based on the shooting position parameters, at least the scene images of the Forbidden City without people and other debris taken from shooting positions such as the Forbidden City main hall, the Forbidden City side hall, the Forbidden City square, and the Forbidden City rooftop buildings can be included. Based on the shooting orientation parameters, at least the scene images of the Forbidden City without people and other debris taken from shooting orientations such as east, south, west, and north can be included. Based on the shooting angle parameters, at least the scene images of the Forbidden City without people and other debris taken from shooting angles such as looking up, looking down, looking straight, or panorama can be included. Based on the shooting season information, at least the scene images of the Forbidden City without people and other debris taken from shooting seasons such as spring, summer, autumn, and winter can be included. Based on the shooting weather information, at least the scene images of the Forbidden City without people and other debris taken from shooting weather conditions such as sunny, cloudy, rainy, and snowy can be included.

[0089] The specific shooting parameters may be determined according to actual conditions. For example, the shooting parameters include one or more of shooting illumination information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information, and shooting weather information. In the present disclosure, the shooting parameters are not specifically limited.

[0090] In the disclosed embodiments, the process of acquiring shooting parameters may adopt different acquisition methods based on different shooting parameters. For example, the shooting time, lighting, location information, etc. may be directly obtained from the shooting parameter attributes of the shot media image. The shooting direction may be determined based on the location information. For example, the shooting direction information may be determined based on the longitude and latitude coordinates of the shooting point and the longitude and latitude coordinates of the shooting scene. The shooting season, shooting weather, etc. may be determined based on the shooting time and shooting location through a network query. Alternatively, the shooting parameters may be recorded and stored during the shooting process, and may be subsequently obtained from a storage medium. For example, the media image to be processed may be a photo that has been shot or a video that has been recorded. The shooting parameters may be stored in a storage medium, and the shooting parameters of the media image to be processed may be directly obtained from the storage medium.

[0091] In step S42, based on the shooting parameters, a subset of scene images matching the shooting parameters is determined in the scene image set in the cloud database.

[0092] With the advent of the 5G era, the process of determining a subset of scene images that match shooting parameters in a scene image set in a cloud database based on shooting parameters will take a very short time, which can lay the foundation for quickly and accurately acquiring scene images that match target scene objects.

[0093] In the disclosed embodiment, based on the shooting parameters, a preliminary screening of scene images without people and other debris can be performed in the scene image set of the cloud database to narrow the matching range of subsequent scene image matching. In the disclosed embodiment, the scene images determined in the scene image set of the cloud database based on the shooting parameters are referred to as scene image subsets.

[0094] In one embodiment, it is assumed that a user takes a photo of user A in the south direction of the Forbidden City in the morning of early spring. Therefore, it can be determined that the user's photographed object is the Forbidden City, and the shooting parameters include the shooting season of early spring, the shooting time of morning, and the shooting direction of south. Through the present disclosure, a scene image subset matching the photographed photo can be determined in the cloud database based on these shooting parameters. Among them, the scene image subset can be understood as a number of scene images that meet the shooting season of early spring, the shooting time of morning, and the shooting direction of south.

[0095] During the application process, users can find scene images that match the shooting object and all shooting parameters by entering the shooting object and all shooting parameters into the keyword search box; they can also first determine the searched related items based on the shooting object, such as the Forbidden City, and then search for all scene images that match all current shooting parameters under the related items based on the given related items.

[0096] In the above manner, a scene image matching the target scene object, ie, a scene image subset, can be quickly found.

[0097] In step S43, a scene image matching the target scene object is obtained from the scene image subset.

[0098] Based on the scene image subset, a scene image that best matches the target scene object may be determined in the scene image subset.

[0099] The present disclosure will illustrate the process of obtaining a scene image that matches a target scene object in a scene image subset through the following embodiments.

[0100] In the disclosed embodiments, in addition to the target scene object, other scene objects may exist in the media image to be processed. For example, the occluding object that occludes the target scene object in the media image to be processed may include a subject occluding object. The subject occluding object may be understood as an occluding object intended to be retained in the media image. For example, a subject portrait intended to be captured in a photograph may be understood as a subject occluding object that occludes the target scene object.

[0101] In the disclosed embodiment, when obtaining a scene image that matches a target scene object, the feature matching weights of scene objects other than the target scene object in the media image to be processed may be reset to zero, and in the scene image subset, the scene image with the highest feature matching weight with the target scene object may be determined as the scene image that matches the target scene object.

[0102] Since it is necessary to match the scene image with no people or other debris in the media image to be processed, a matching scene image is obtained. Therefore, when calculating the matching weight based on the features in the media image to be processed and the features in each scene image, the feature matching weights of other scene objects in the media image to be processed except the target scene object can be reset to zero first, that is, it can be understood that the features of the target scene object in the media image to be processed are used as the features of the media image to be processed. Furthermore, the scene image with the highest feature matching weight with the media image to be processed, which is determined based on the feature weight matching method in the scene image subset, is used as the scene image that matches the target scene object.

[0103] In the embodiments of the present disclosure, other methods may be used to implement a solution for performing real filling processing on the target scene object in the first area of ​​the media image to be processed to generate the target media image.

[0104] In one embodiment, the area of ​​the media image to be processed except the first area can be filled and repaired through big data learning to obtain the real target scene object, thereby generating the target media image; in another embodiment, the first area can be directly repaired in the first area of ​​the media image to be processed without removing the first area, so that the target occluding object that occludes the target scene object is repaired and coordinated with the target scene object, thereby generating the target media image.

[0105] In an example of the present disclosure, a first area corresponding to a target occluded object in a media image to be processed is repaired based on a pre-trained neural network model, and generation of a target media image is used as an example for explanation.

[0106] In the first area corresponding to the target occluded object in the media image to be processed, the target scene object is filled and the target media image is generated by the following method:

[0107] Based on a pre-trained neural network model, a first area corresponding to a target occluded object in a media image to be processed is repaired to generate a target media image.

[0108] In the disclosed embodiment, a deep learning-based image restoration technique may be used to perform restoration processing on a first region corresponding to a target occluded object in a media image to be processed, so as to generate a target media image.

[0109] During the application process, the first area corresponding to the target occluded object in the media image to be processed can be directly repaired based on the pre-trained neural network model to generate the target media image.

[0110] In one embodiment of the present disclosure, when there is no cloud database or when no scene image matching the target scene object is found in the cloud database, a deep learning-based image restoration technique is used to restore the first area corresponding to the target occluded object in the processed media image.

[0111] Of course, it is understandable that the implementation method of the deep learning-based image restoration technology in the embodiments of the present disclosure for repairing the first area corresponding to the target occluded object in the processed media image can be used in conjunction with the aforementioned embodiments or can be used independently. Whether used alone or in conjunction with the aforementioned embodiments, the implementation principle is similar. In the implementation of the present disclosure, some embodiments are described in terms of implementation methods used together; of course, those skilled in the art can understand that such examples are not limitations of the embodiments of the present disclosure.

[0112] The following embodiment of the present disclosure describes a process of determining a target occluding object that occludes a target scene object in a media image to be processed.

[0113] Figure 6 The present invention is a flowchart showing a method of determining a target occluding object that occludes the target scene object in a media image to be processed according to an exemplary embodiment.

[0114] like Figure 6 As shown, in an exemplary embodiment of the present disclosure, determining a target occluding object that occludes the target scene object in a media image to be processed includes the following steps.

[0115] In step S51, pixel segmentation is performed on the occluding objects that occlude the target scene objects in the processed media image to obtain a two-dimensional mask image of the occluding objects, wherein each region where the occluding objects are located in the two-dimensional mask image has its own class label.

[0116] Identify the target scene object and the occluded object in the media image to be processed, and distinguish the occluded object from the target scene object. Perform pixel segmentation on the identified occluded object to obtain the area of ​​the occluded object at the pixel level. In one embodiment, by performing pixel segmentation on the occluded object, a two-dimensional mask image of the occluded object can be obtained. In the two-dimensional mask image, each area where the occluded object is located has its own class label.

[0117] The identified occluding objects include not only the target occluding objects but also the subject occluding objects. The subject occluding objects may be the subject person in the media image to be processed. With respect to the media image to be processed, the subject occluding objects may be the occluding objects that need to be retained in the image, and the target occluding objects may be the occluding objects that need to be removed from the media image to be processed.

[0118] In one embodiment, the segmentation of the occluded object area can be achieved based on a convolutional neural network (CNN).

[0119] Based on the identified occluding objects, pixel-level segmentation is performed to obtain different occluding object regions, and different occluding object regions correspond to different class labels. The class labels can be identifiers that characterize different occluding object regions, that is, the occluding object regions correspond to the class labels. For example, the subject occluding object region corresponds to the class label of the subject occluding object region, and the target occluding object region corresponds to the class label of the target occluding object region.

[0120] In step S52, based on the class label, the area where the subject occluded the object in the two-dimensional mask image is located is determined.

[0121] Since the occluded object area corresponds to the class label, the area where the main occluded object is located can be determined based on the class label.

[0122] In step S53, other occluding objects in the two-dimensional mask image except the area where the main occluding object is located are determined as target occluding objects.

[0123] In an exemplary embodiment of the present disclosure, based on the class label, determining the area where the subject occludes the object in the two-dimensional mask image is located can be achieved in the following manner.

[0124] In one embodiment, the pixel area corresponding to the class label of the subject occluding object can be detected from all class labels based on the convolutional neural network as the area where the subject occluding object is located. That is, based on the convolutional neural network and the class label, the area where the subject occluding object is located in the media image to be processed is automatically detected and recommended.

[0125] In one embodiment, the pixel area corresponding to the class mark selected by the user can be used as the area where the subject occluded object is located. That is, the areas where the occluded objects corresponding to the class marks are located are manually selected as the areas where the subject occluded object is located, and the areas where the occluded objects corresponding to the class marks are manually selected as the areas where the target occluded object is located.

[0126] In another embodiment, the area where the first subject occluded object in the media image to be processed is automatically detected and recommended based on a convolutional neural network and a class label, and after manual correction, it is further determined which areas are the areas where the subject occluded object is located within the area where the first subject occluded object is located.

[0127] Based on the image processing method provided by the above embodiment of the present disclosure, Figure 7 The figure is a schematic diagram showing a process of an image processing method according to an exemplary embodiment.

[0128] like Figure 7 As shown, in an exemplary embodiment of the present disclosure, the image processing method includes the following steps.

[0129] In step S61, a media image to be processed is obtained.

[0130] A media image to be processed is obtained, wherein the media image to be processed displays a target scene object that is at least partially obscured, wherein the obstructing object obstructing the target scene object includes a target obstructing object and a subject obstructing object.

[0131] In step S62, a target occluding object that occludes the target scene object in the media image to be processed is determined.

[0132] In one embodiment, the occluded objects in the media image to be processed can be segmented based on a segmentation technology, such as a convolutional neural network, to obtain the area where the target occluded object is located, that is, the first area and the area where the main occluded object is located.

[0133] In step S63, a first area corresponding to the target occluding object in the media image to be processed is removed.

[0134] Among them, removing the first area corresponding to the target occluded object in the media image to be processed can be understood as retaining the main occluded object and removing the target occluded object with a part of the target scene object. That is, an image with a hole and a real part of the target scene object is obtained, wherein the hole in the image is the first area corresponding to the removed target occluded object.

[0135] In step S64, the shooting parameters of the media image to be processed are obtained.

[0136] The shooting parameters include one or more of shooting lighting information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information and shooting weather information.

[0137] In step S65, based on the shooting parameters, a scene image matching the target scene object is acquired.

[0138] In the embodiment of the present disclosure, based on the shooting parameters, a cloud database can be searched for scene images that match the shooting parameters.

[0139] In the application process, you can enter the shooting object and all shooting parameters into the keyword search box to find the scene image that matches the shooting object and all shooting parameters; you can also first determine the searched related items based on the shooting object, such as the Forbidden City, and then search for all scene images that match all current shooting parameters under the related items based on the given related items. Finally, determine the scene image that best matches the target scene object among all the scene images.

[0140] Furthermore, the feature matching weights of other scene objects except the target scene object in the media image to be processed can be reset to zero, and in the scene image subset, the scene image with the highest feature matching weight with the target scene object is determined as the scene image matching the target scene object.

[0141] In step S66, the target scene object is filled in the first area corresponding to the target occluded object in the media image to be processed to generate a target media image.

[0142] In one embodiment, when there is a scene image matching the target scene object in the cloud database, the scene image matching the target scene object is obtained based on the shooting parameters. The scene image and the target scene object are image-aligned, a second area corresponding to the first area is determined in the scene image, and the second area and the media image to be processed are fused to obtain the target media image. In this way, the connection of the fused target media image can be smooth, and the overall effect can be real, natural, and fascinating.

[0143] In another embodiment, when there is no scene image matching the target scene object in the cloud database, the target scene object can be filled in the first area corresponding to the target occluded object in the media image to be processed based on deep learning image restoration technology to generate the target media image.

[0144] It can be seen from the above description that the image processing method provided by the present disclosure realizes the effective removal of the first area corresponding to the target occluded object in the media image to be processed. Moreover, the second area that fills the matching first area can realize the filling of the real target scene object in the media image to be processed from which the first area has been removed, thereby ensuring the authenticity of the target scene object in the media image to be processed, and further ensuring that the user can obtain the media image to be processed without the target occluded object that occludes the target scene object and close to the real target scene object.

[0145] Based on the same concept, an embodiment of the present disclosure also provides an image processing device.

[0146] It is understandable that in order to realize the above functions, the image processing device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0147] Figure 8 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0148] In an exemplary embodiment of the present disclosure, referring to Figure 8 The image processing device includes an acquisition module 110, a target occluding object determination module 120 and an image processing module 130. Each module will be introduced below.

[0149] The acquisition module 110 is configured to: acquire a to-be-processed media image, wherein the to-be-processed media image displays a target scene object that is at least partially obscured.

[0150] The target occluding object determining module 120 is configured to determine a target occluding object that occludes a target scene object in the media image to be processed.

[0151] The image processing module 130 is configured to: fill in the target scene object in the first area corresponding to the target occluded object in the media image to be processed, and generate a target media image.

[0152] In an exemplary embodiment of the present disclosure, the image processing apparatus further includes a first region removing module 140. The first region removing module 140 is configured to remove a first region corresponding to a target occluding object in a to-be-processed media image.

[0153] In an exemplary embodiment of the present disclosure, the image processing device further includes a scene image matching module 150. The scene image matching module 150 is configured to: obtain a scene image matching the target scene object from a cloud database. The image processing module 130 is configured to: perform image registration on the scene image and the target scene object, determine a second area corresponding to the first area in the scene image; and fuse the second area and the media image to be processed to obtain the target media image.

[0154] In an exemplary embodiment of the present disclosure, the image processing device further includes a shooting parameter acquisition module 160. The shooting parameter acquisition module 160 is configured to: acquire shooting parameters of the media image to be processed, the shooting parameters including one or more of shooting illumination information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information, and shooting weather information; the scene image matching module 150 is configured to: determine a scene image subset matching the shooting parameters in a scene image set in a cloud database based on the shooting parameters; and acquire a scene image matching the target scene object in the scene image subset.

[0155] In an exemplary embodiment of the present disclosure, the scene image matching module 150 is configured to: reset the feature matching weights of other scene objects in the media image to be processed except the target scene object to zero, and determine, in the scene image subset, the scene image with the highest feature matching weight with the target scene object as the scene image that matches the target scene object.

[0156] In an exemplary embodiment of the present disclosure, the image processing module 130 is configured to: based on a pre-trained neural network model, perform repair processing on a first area corresponding to a target occluded object in a to-be-processed media image to generate a target media image.

[0157] In an exemplary embodiment of the present disclosure, the target occluding object determination module 120 is configured to: perform pixel segmentation on the occluding objects that occlude the target scene objects in the processed media image to obtain a two-dimensional mask image of the occluding objects, wherein the region where each occluding object is located in the two-dimensional mask image has its own class label; based on the class label, determine the region where the subject occluding object is located in the two-dimensional mask image; and determine other occluding objects in the two-dimensional mask image except the region where the subject occluding object is located as target occluding objects.

[0158] In an exemplary embodiment of the present disclosure, the target occluding object determination module 120 is configured to: detect the pixel area corresponding to the class label that meets the subject occluding object in all class labels based on a convolutional neural network as the area where the subject occluding object is located; or use the pixel area corresponding to the class label selected by the user as the area where the subject occluding object is located.

[0159] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0160] Fig. 9 2 is a block diagram of an apparatus for image processing according to an exemplary embodiment. For example, the apparatus 200 for image processing may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0161] Reference Fig. 9 The apparatus 200 for image processing may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0162] The processing component 202 generally controls the overall operation of the apparatus 200 for image processing, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0163] The memory 204 is configured to store various types of data to support operations on the apparatus for image processing 200. Examples of such data include instructions for any application or method operating on the apparatus for image processing 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0164] The power component 206 provides power to various components of the apparatus 200 for image processing. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the apparatus 200 for image processing.

[0165] The multimedia component 208 includes a screen that provides an output interface between the apparatus 200 for image processing and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the apparatus 200 for image processing is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0166] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the apparatus 200 for image processing is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0167] I / O interface 812 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0168] The sensor assembly 214 includes one or more sensors for providing various aspects of status assessment for the apparatus 200 for image processing. For example, the sensor assembly 214 can detect the open / closed state of the apparatus 200 for image processing, the relative positioning of components, such as the display and keypad of the apparatus 200 for image processing, the sensor assembly 214 can also detect the position change of the apparatus 200 for image processing or a component of the apparatus 200 for image processing, the presence or absence of user contact with the apparatus 200 for image processing, the orientation or acceleration / deceleration of the apparatus 200 for image processing, and the temperature change of the apparatus 200 for image processing. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0169] The communication component 216 is configured to facilitate wired or wireless communication between the apparatus 200 for image processing and other devices. The apparatus 200 for image processing can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0170] In an exemplary embodiment, the apparatus 200 for image processing may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the apparatus 200 for image processing to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0173] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0174] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0175] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0176] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0177] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Acquire a media image to be processed, wherein the media image to be processed displays a target scene object that is at least partially obscured; Determining a target occluding object that occludes the target scene object in the media image to be processed; Acquire shooting parameters of the media image to be processed, wherein the shooting parameters include multiple items of shooting lighting information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information, and shooting weather information; Based on the shooting parameters, acquiring a scene image matching the target scene object; In a first area corresponding to the target occluded object in the to-be-processed media image, filling the target scene object based on the scene image to generate a target media image; The acquiring, based on the shooting parameters, a scene image matching the target scene object comprises: Based on the shooting parameters, determining a subset of scene images matching the shooting parameters in a scene image set in a cloud database; Acquire a scene image matching the target scene object from the scene image subset; The acquiring, from the scene image subset, a scene image matching the target scene object comprises: The feature matching weights of other scene objects in the to-be-processed media image except the target scene object are reset to zero, and in the scene image subset, a scene image having the highest feature matching weight with the target scene object is determined as the scene image matching the target scene object.

2. The image processing method according to claim 1, characterized in that: Before the target scene object is filled in the first area corresponding to the target occluded object in the to-be-processed media image to generate the target media image, the method further comprises: A first area corresponding to the target occluding object in the media image to be processed is removed.

3. The image processing method according to claim 1 or 2, characterized in that: The first area corresponding to the target occluded object in the to-be-processed media image is filled with the target scene object based on the scene image to generate a target media image, including: Performing image registration on the scene image and the target scene object, and determining a second area corresponding to the first area in the scene image; The second region and the media image to be processed are fused to obtain a target media image.

4. The image processing method according to claim 1, characterized in that: The first area corresponding to the target occluded object in the to-be-processed media image is filled with the target scene object to generate a target media image, including: Based on a pre-trained neural network model, a first area corresponding to the target occluded object in the media image to be processed is repaired to generate a target media image.

5. The image processing method according to claim 1, characterized in that: The determining a target occluding object that occludes the target scene object in the to-be-processed media image includes: Performing pixel segmentation on the occluding object that occludes the target scene object in the to-be-processed media image to obtain a two-dimensional mask image of the occluding object, wherein each region where the occluding object is located in the two-dimensional mask image has its own class label; Based on the class label, determining the area where the subject occluded the object in the two-dimensional mask image is located; The other occluding objects in the two-dimensional mask image, except for the area where the main occluding object is located, are determined as the target occluding objects.

6. The image processing method according to claim 5, characterized in that: The step of determining the area where the subject occluded object in the two-dimensional mask image is located based on the class label includes: Detecting a pixel area corresponding to a class label that matches the subject occlusion object from all class labels based on a convolutional neural network as the area where the subject occlusion object is located; or The pixel area corresponding to the class label selected by the user is used as the area where the subject occluded object is located.

7. An image processing device, characterized in that: The image processing device comprises: An acquisition module, used for acquiring a media image to be processed, wherein the media image to be processed displays a target scene object that is at least partially obscured; A target occluding object determining module, used to determine a target occluding object that occludes the target scene object in the media image to be processed; A shooting parameter acquisition module, used to acquire shooting parameters of the media image to be processed, wherein the shooting parameters include multiple items of shooting lighting information, shooting time information, shooting location information, shooting orientation information, shooting angle information, shooting season information and shooting weather information; A scene image matching module, used to obtain a scene image matching the target scene object based on the shooting parameters; An image processing module, configured to fill a first area corresponding to the target occluded object in the media image to be processed with the target scene object based on the scene image to generate a target media image; The scene image matching module is used for: Based on the shooting parameters, determining a subset of scene images matching the shooting parameters in a scene image set in a cloud database; Acquire a scene image matching the target scene object from the scene image subset; The scene image matching module is used for: The feature matching weights of other scene objects in the to-be-processed media image except the target scene object are reset to zero, and in the scene image subset, a scene image having the highest feature matching weight with the target scene object is determined as the scene image matching the target scene object.

8. The image processing device according to claim 7, characterized in that: The image processing device further includes a first region removal module, wherein the first region removal module is configured to: A first area corresponding to the target occluding object in the media image to be processed is removed.

9. The image processing device according to claim 7 or 8, characterized in that: The image processing module is used to: perform image registration on the scene image and the target scene object, and determine a second area corresponding to the first area in the scene image; The second region and the media image to be processed are fused to obtain a target media image.

10. The image processing device according to claim 7, characterized in that: The image processing module is used for: Based on a pre-trained neural network model, a first area corresponding to the target occluded object in the media image to be processed is repaired to generate a target media image.

11. The image processing device according to claim 7, characterized in that: The target occlusion object determination module is used for: Performing pixel segmentation on the occluding object that occludes the target scene object in the to-be-processed media image to obtain a two-dimensional mask image of the occluding object, wherein each region where the occluding object is located in the two-dimensional mask image has its own class label; Based on the class label, determining the area where the subject occluded the object in the two-dimensional mask image is located; The other occluding objects in the two-dimensional mask image, except for the area where the main occluding object is located, are determined as the target occluding objects.

12. The image processing device according to claim 11, characterized in that: The target occlusion object determination module is used for: Detecting a pixel area corresponding to a class label that matches the subject occlusion object from all class labels based on a convolutional neural network as the area where the subject occlusion object is located; or The pixel area corresponding to the class label selected by the user is used as the area where the subject occluded object is located.

13. An image processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the image processing method described in any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the image processing method according to any one of claims 1 to 6.

Images