Scenic spot image processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to a scenic spot image processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed scenic spot image, determining a target region in the to-be-processed scenic spot image, removing the target region in the to-be-processed scenic spot image, and obtaining a removed image; obtaining image attribute information of the to-be-processed scenic spot image, and obtaining a similar image corresponding to the rejected image according to the image attribute information; determining a target area in the similar image, and complementing the rejected image according to the target area in the similar image to obtain a complemented image; and inputting the complemented image and the image attribute information into a preset image restoration neural network model, performing image restoration on the complemented image, and obtaining a restored image of the to-be-processed scenic region image, thereby realizing impurity elimination and restoration of the to-be-processed scenic region image, and satisfying image processing requirements of users.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, apparatus and storage medium for processing scenic area images. Background Art

[0002] With the development of smartphone photography technology and the popularity of social media, people are increasingly hoping to get perfect, distraction-free photos when taking photos of landmark buildings or attractions during travel. However, due to the large flow of people, the photos taken often contain many unwanted background figures, which not only affects the beauty of the photos, but also reduces the commemorative value of the trip. Therefore, intelligent post-processing of photos has become an urgent need to be solved.

[0003] Traditional image editing software such as Photoshop provides a certain degree of image processing functions, including tools such as clone stamp and repair brush, which can help users remove unwanted elements in photos. However, the entire process requires manual participation, which is time-consuming and laborious. In addition, the editing of photo images involves the process of image segmentation and image repair. In particular, image repair technology is not very effective in repairing and supplementing the missing parts of the image after removing the unwanted elements in the photo. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a method, device, equipment and storage medium for processing scenic spot images. Based on the constructed completed image corresponding to the scenic spot image to be processed, an image restoration neural network model is adopted to realize the removal and restoration of debris in the scenic spot image to be processed, thereby meeting the image processing needs of users.

[0005] In a first aspect, an embodiment of the present application provides a method for processing a scenic spot image, comprising the following steps:

[0006] Obtaining a scenic area image to be processed, determining a target area in the scenic area image to be processed, and removing the target area in the scenic area image to be processed to obtain a removed image;

[0007] Obtaining image attribute information of the scenic spot image to be processed, and obtaining a similar image corresponding to the eliminated image according to the image attribute information;

[0008] Determine a target area in the similar image, and complete the eliminated image according to the target area in the similar image to obtain a completed image;

[0009] The completed image and image attribute information are input into a preset image restoration neural network model, and image restoration is performed on the completed image to obtain a restored image of the scenic area image to be processed.

[0010] In a second aspect, an embodiment of the present application provides a device for processing a scenic spot image, comprising:

[0011] An image rejection module is used to obtain a scenic area image to be processed, determine a target area in the scenic area image to be processed, and reject the target area in the scenic area image to be processed to obtain a rejected image;

[0012] An image retrieval module, used to obtain image attribute information of the scenic spot image to be processed, and obtain similar images corresponding to the removed images according to the image attribute information;

[0013] An image completion module, used for determining a target area in the similar image, and completing the eliminated image according to the target area in the similar image to obtain a completed image;

[0014] The image restoration module is used to input the completed image and image attribute information into a preset image restoration neural network model, perform image restoration on the completed image, and obtain a restored image of the scenic area image to be processed.

[0015] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the method for processing scenic spot images as described in the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing a scenic spot image as described in the first aspect are implemented.

[0017] In an embodiment of the present application, a method, device, equipment and storage medium for processing a scenic spot image are provided. Based on a completed image corresponding to the scenic spot image to be processed, an image restoration neural network model is adopted to realize the removal and restoration of debris in the scenic spot image to be processed, thereby meeting the image processing needs of users.

[0018] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of a method for processing a scenic spot image provided by one embodiment of the present application;

[0020] Figure 2 A schematic diagram of the process of S2 in a method for processing a scenic area image provided in one embodiment of the present application;

[0021] Figure 3A schematic diagram of the process of S4 in a method for processing a scenic area image provided in one embodiment of the present application;

[0022] Figure 4 A schematic diagram of the flow of S5 in a method for processing a scenic spot image provided by another embodiment of the present application;

[0023] Figure 5 A schematic diagram of the flow of S55 in a method for processing a scenic area image provided by another embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of a scenic area image processing device provided in one embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0027] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0029] See also Figure 1 , Figure 1 A flowchart of a method for processing a scenic spot image provided by an embodiment of the present application is provided, wherein the method comprises the following steps:

[0030] S1: obtaining a scenic area image to be processed, determining a target area in the scenic area image to be processed, and removing the target area in the scenic area image to be processed to obtain a removed image.

[0031] The execution subject of the scenic image processing method is a processing device of the scenic image processing method (hereinafter referred to as processing device). In an optional embodiment, the processing device can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0032] In this embodiment, the processing device can obtain the image of the scenic spot to be processed input by the user, or obtain the image of the scenic spot to be processed sent by the shooting terminal through a data transmission connection with the shooting terminal, wherein the shooting terminal can be a device with a shooting function such as a mobile phone, a tablet or an interactive tablet.

[0033] The processing device determines the target area in the scenic area image to be processed, removes the target area in the scenic area image to be processed, and obtains a removed image. Specifically, the processing device responds to an image removal instruction issued by a user, and the image removal instruction can be any tactile instruction, such as a voice control instruction issued by the user: "eliminate the background characters and vehicles in the image", and the image removal instruction can also be generated by the user triggering a control in a preset editing interface and inputting information associated with the target area in the control. The processing device parses the image removal instruction, and converts the information associated with the target area in the image removal instruction into a prompt word for image segmentation, such as "segment the image according to semantics, wherein the background characters and vehicles are segmented according to categories, and the characters and vehicles are marked with different segmentation lines", to determine the target area in the scenic area image to be processed. In an optional embodiment, the processing device can determine the target area in the scenic area image to be processed by the user manually selecting the segmented image to be eliminated, which can be selected by category or one by one.

[0034] The processing device removes the target area in the scenic area image to be processed according to the scenic area image to be processed, the target area and a preset image segmentation model to obtain a removed image.

[0035] S2: Obtaining image attribute information of the scenic area image to be processed, and obtaining a similar image corresponding to the eliminated image according to the image attribute information.

[0036] In this embodiment, the processing device may obtain the image attribute information of the to-be-processed scenic spot image sent by the shooting terminal, and obtain the similar image corresponding to the eliminated image according to the image attribute information.

[0037] Specifically, the image attribute information is the environmental information of the scenic spot image to be processed that is synchronously recorded by the shooting terminal through a sensor device or a public API interface when shooting the scenic spot image to be processed, as a basis for fast similarity retrieval, wherein the image attribute information includes location information, time period information, weather information and building angle information.

[0038] See also Figure 2 , Figure 2 The flowchart of S2 in the method for processing a scenic spot image provided in one embodiment of the present application includes steps S21 to S22, which are specifically as follows:

[0039] S21: obtaining a plurality of candidate scenic area images and image attribute information of the candidate scenic area images according to the main building image information in the image attribute information.

[0040] In this embodiment, the processing device searches from a preset image database according to the main building image information in the image attribute information, obtains several images that match the main building image information as candidate scenic spot images, obtains several candidate scenic spot images and image attribute information of the candidate scenic spot images, wherein the image database is pre-established and includes images obtained by traversing and photographing buildings at different locations, time periods, weather conditions, and angles at several scenic spots.

[0041] S22: Based on the location information, time period information, weather information and building angle information in the image attribute information, a similarity calculation method is used to obtain the similarity between the eliminated image and several candidate scenic spot images, and the candidate scenic spot image with the highest similarity is taken as the similar image to obtain the similar image corresponding to the eliminated image.

[0042] In this embodiment, the processing device uses a similarity calculation method to obtain the similarity between the eliminated image and several candidate scenic spot images based on the location information, time period information, weather information and building angle information in the image attribute information, and takes the candidate scenic spot image with the highest similarity as the similar image to obtain the similar image corresponding to the eliminated image.

[0043] Specifically, based on the location information, the processing device can adopt the Euclidean distance calculation method to obtain the Euclidean distance between the eliminated image and several of the candidate scenic area images, and obtain the location similarity according to the product of the Euclidean distance and the preset location weight parameter; based on the time period information, the processing device can adopt the Euclidean distance calculation method to obtain the Euclidean distance based on the time step between the eliminated image and several of the candidate scenic area images, and obtain the time similarity according to the product of the Euclidean distance based on the time step and the preset time weight parameter; based on the weather information, the processing device can adopt the Jaccard similarity calculation method to construct a elimination map The invention relates to a processing device for processing a weather element set of a discarded image and several candidate scenic spot images, wherein the weather element set includes several types of weather elements, and obtaining weather similarity by calculating the ratio of the intersection size to the union size of the weather element set of the discarded image and several candidate scenic spot images; based on the building angle information, the processing device can adopt a cosine similarity calculation method to obtain the building angle similarity between the discarded image and several candidate scenic spot images; the processing device accumulates the position similarity, time similarity, weather similarity and building angle similarity of the discarded image and the same candidate scenic spot image to obtain the similarity between the discarded image and several candidate scenic spot images.

[0044] S3: Determine the target area in the similar image, and complete the eliminated image according to the target area in the similar image to obtain a completed image.

[0045] In this embodiment, the processing device can rotate and adjust the similar image to the same proportion, and overlap the rotated image with the removed image to determine the target area in the similar image. According to the target area in the similar image, the image of the target area is replaced to the corresponding position of the removed image, and the removed image is completed to obtain a completed image.

[0046] S4: Inputting the completed image and image attribute information into a preset image restoration neural network model, performing image restoration on the completed image, and obtaining a restored image of the scenic spot image to be processed.

[0047] Since the completed image is obtained by image replacement, in this embodiment, the processing device inputs the completed image and image attribute information into a preset image restoration neural network model, performs image restoration on the completed image, and obtains a restored image of the scenic area image to be processed, so as to overcome the problem that the completed image may have unnatural effects at the edges of the corresponding areas of the image replacement, thereby improving the effects of removing debris and restoring the scenic area image to be processed.

[0048] See also Figure 3 , Figure 3 The flowchart of S4 in the method for processing a scenic spot image provided in one embodiment of the present application includes steps S41 to S42, which are specifically as follows:

[0049] S41: Determine a repair zone area of ​​the completed image according to an edge line of a target area in the completed image.

[0050] In this embodiment, the processing device determines the repair band area of ​​the completed image based on the edge line of the target area in the completed image, and forms an annular band area that extends inward and outward at different distances starting from the edge line of the target area in the completed image.

[0051] S42: performing virtual and enhanced diffusion processing on the repaired band area of ​​the complemented image according to the image attribute information to obtain a repaired image of the scenic area image to be processed.

[0052] The image restoration neural network model is a pre-trained diffusion image generation model (Diffusion Models), which can generate images based on image attribute information.

[0053] In this embodiment, the processing device obtains a generated image corresponding to the repair band area of ​​the completed image based on the image attribute information and the image repair neural network model, splices the generated image at a corresponding position of the completed image to obtain a spliced ​​image, and performs virtual and enhanced diffusion processing on the spliced ​​image to repair the edge of the target area of ​​the completed image, thereby obtaining a repaired image of the scenic area image to be processed.

[0054] In an optional embodiment, the method further comprises step S5: training the image restoration neural network model. Figure 4 , Figure 4 The flowchart of S5 in the method for processing scenic area images provided in another embodiment of the present application includes steps S51 to S55, which are specifically as follows:

[0055] S51: Obtain a training image set and a verification image set, extract a number of verification images from the verification image set, input the several verification images into the image restoration neural network model to be trained to perform image mask random coverage processing, and obtain mask images corresponding to the several verification images.

[0056] In this embodiment, the processing device obtains a training image set and a verification image set, wherein the training image set and the verification image set are two non-overlapping databases, the training image set is used for an unsupervised training process of an artificial neural network model image generation, and the verification image set is used to converge and verify the image generation effect of the artificial neural network model, and to perform supervised fine-tuning on the model. The training image set includes a plurality of training images, and the verification image set includes a plurality of verification images.

[0057] The processing device extracts a plurality of verification images from the verification image set, inputs the plurality of verification images into the image restoration neural network model to be trained to perform image mask random coverage processing, and obtains mask images corresponding to the plurality of verification images.

[0058] S52: performing scenic area image retrieval according to the plurality of mask images and a training image set, and extracting a plurality of training images corresponding to the mask image from the training image set as similar images corresponding to the mask image.

[0059] In this embodiment, the processing device adopts a similarity calculation method to perform scenic spot image retrieval based on several mask images and a training image set, and extracts several training images corresponding to the mask image from the training image set as similar images corresponding to the mask image. For a specific embodiment, please refer to step S22, which will not be repeated here.

[0060] S53: according to the mask areas in the plurality of mask images, confirm the mask areas in the similar images corresponding to the mask images, and complement the mask areas in the corresponding mask images with the mask areas in the similar images to obtain complement images corresponding to the plurality of verification images.

[0061] In this embodiment, the processing device confirms the mask area in the similar image corresponding to the mask image based on the mask area in the several mask images, and completes the mask area in the corresponding mask image with the mask area in the similar image to obtain the completed images corresponding to the several verification images. The specific implementation method can refer to step S3, which will not be repeated here.

[0062] S54: Obtaining image attribute information of several verification images, inputting the complement images and image attribute information corresponding to the several verification images into the image restoration neural network model to be trained to perform image restoration, and obtaining the restoration images corresponding to the several verification images.

[0063] In this embodiment, the processing device obtains image attribute information of several of the verification images, inputs the completed images and image attribute information corresponding to the several verification images into the image restoration neural network model to be trained to perform image restoration, and obtains the restoration images corresponding to the several verification images. The specific embodiments can refer to steps S41 to S42, which will not be repeated here.

[0064] S55: Training the image restoration neural network model to be trained according to the plurality of verification images and the corresponding restoration images and the discriminator of the image restoration neural network model to be trained.

[0065] In this embodiment, the processing device inputs a plurality of the verification images and the corresponding restoration images into the discriminator of the image restoration neural network model to be trained, and trains the image restoration neural network model to be trained.

[0066] The image restoration neural network model to be trained includes a global discriminator and a local discriminator. Figure 5 , Figure 5 The flowchart of S55 in the method for processing a scenic spot image provided by another embodiment of the present application includes steps S551 to S552, which are specifically as follows:

[0067] S551: According to the global discriminator, global features of the verification images and the corresponding repair images are extracted to obtain global features of the verification images and the corresponding repair images; according to the local discriminator, local features of the repair band areas of the verification images and the corresponding repair images are extracted to obtain local features of the verification images and the corresponding repair images.

[0068] In this embodiment, the processing device performs global feature extraction on the several verification images and the corresponding restoration images according to the global discriminator to obtain global features of the several verification images and the corresponding restoration images. Specifically, the global discriminator includes several convolutional layers and fully connected layers. The several convolutional layers are used to perform convolution processing with a convolution kernel of K*K and a step size of S*S on the entire area of ​​the several verification images and the corresponding restoration images to obtain first convolution features of the several verification images and the corresponding restoration images. The fully connected layers are used to perform fully connected processing on the first convolution features of the several verification images and the corresponding restoration images to obtain global features of the several verification images and the corresponding restoration images, so as to extract the global feature distribution of the several verification images and the corresponding restoration images, so as to judge whether the restoration images conform to the feature patterns of the real images and evaluate the consistency, fluency and authenticity of the overall image structure.

[0069] The processing device extracts local features of the repair band areas of the verification images and the corresponding repair images according to the local discriminator to obtain local features of the verification images and the corresponding repair images. Specifically, the global discriminator includes several convolutional layers and fully connected layers. The convolutional layers are used to perform convolution processing with a convolution kernel of K*K and a step size of S*S on the verification images and the repair band areas of the corresponding repair images to obtain second convolutional features of the verification images and the corresponding repair images. The fully connected layers are used to perform fully connected processing on the second convolutional features of the verification images and the corresponding repair images to obtain local features of the verification images and the corresponding repair images, so as to capture the texture changes, edge clarity and detail authenticity of the verification images and the corresponding repair images, so as to judge whether the texture of the repair band area of ​​the repair image is coordinated with the surrounding area, whether the edge is clear and natural, and whether the details are rich and realistic.

[0070] S552: Perform loss calculation based on the global features of the verification images and the corresponding repaired images to obtain a global loss value, perform loss calculation based on the local features of the verification images and the corresponding repaired images to obtain a local loss value, and train the image repair neural network model to be trained based on the global loss value and the local loss value.

[0071] In this embodiment, the processing device performs loss calculation based on the global features of several of the verification images and the corresponding repair images to obtain a global loss value, performs loss calculation based on the local features of several of the verification images and the corresponding repair images to obtain a local loss value, and trains the image repair neural network model to be trained based on the global loss value and the local loss value.

[0072] Specifically, the processing device obtains evaluation values ​​corresponding to the global features and evaluation values ​​corresponding to the local features of the verification images and the corresponding restoration images based on the global features and local features of the verification images and the corresponding restoration images, respectively. Specifically, for the evaluation values ​​corresponding to the global features, the processing device can use algorithms based on mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), etc. for calculation; for the evaluation values ​​corresponding to the local features, the processing device can use algorithms such as L1 / L2, texture feature analysis algorithm, local structural similarity index (Local SSIM), etc. for calculation.

[0073] The processing device adopts an adversarial learning method, and based on the evaluation values ​​corresponding to the global features and the evaluation values ​​corresponding to the local features of several verification images and corresponding repaired images, respectively calculates the adversarial loss values ​​corresponding to the global features and the local features as the global loss value and the local loss value, accumulates the global loss value and the local loss value, respectively multiplies the global loss value and the local loss value with the preset corresponding hyperparameters, adds the product results to obtain the total loss value, and trains the image restoration neural network model to be trained according to the total loss value, thereby realizing the removal and restoration of debris in the scenic area image to be processed, which can solve the problem of the peripheral filling technology processing repeated or unnatural textures on large areas or complex textures, and can also solve the inaccurate problem of deep learning model generating images, meet the image processing needs of users, and increase user experience.

[0074] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a scenic image processing device provided by an embodiment of the present application. The device can implement all or part of the scenic image processing device through software, hardware, or a combination of both. The device 6 includes:

[0075] An image removal module 61 is used to obtain a scenic area image to be processed, determine a target area in the scenic area image to be processed, remove the target area in the scenic area image to be processed, and obtain a removed image;

[0076] An image retrieval module 62 is used to obtain image attribute information of the removed image, and obtain a similar image corresponding to the removed image according to the image attribute information;

[0077] An image completion module 63 is used to determine a target area in the similar image, and complete the eliminated image according to the target area in the similar image to obtain a completed image;

[0078] The image restoration module 64 is used to input the completed image into a preset image restoration neural network model, perform image restoration on the completed image, and obtain a restored image of the scenic area image to be processed.

[0079] In the embodiment of the present application, an image of a scenic spot to be processed is obtained through an image removal module, a target area in the scenic spot image to be processed is determined, and the target area in the scenic spot image to be processed is removed to obtain a removed image; an image retrieval module is used to obtain image attribute information of the removed image, and a similar image corresponding to the removed image is obtained based on the image attribute information; an image completion module is used to determine the target area in the similar image, and the removed image is completed based on the target area in the similar image to obtain a completed image; an image restoration module is used to input the completed image into a preset image restoration neural network model, and the completed image is restored to obtain a restored image of the scenic spot image to be processed. Based on the constructed completed image corresponding to the scenic spot image to be processed, an image restoration neural network model is used to realize the removal and restoration of debris in the scenic spot image to be processed, meet the image processing needs of users, and increase user experience.

[0080] Please refer to Figure 7 , Figure 7 The computer device 7 is a schematic diagram of a structure of a computer device provided in an embodiment of the present application. The computer device 7 includes: a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71; the computer device may store a plurality of instructions, which are suitable for being loaded and executed by the processor 71. Figures 1 to 5 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 5 The specific description shown will not be repeated here.

[0081] The processor 71 may include one or more processing cores. The processor 71 uses various interfaces and lines to connect various parts in the server, and executes various functions and processes data of the scenic image processing device 6 by running or executing instructions, programs, code sets or instruction sets stored in the memory 72, and calling the data in the memory 72. Optionally, the processor 71 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 71 can integrate one or more combinations of a central processing unit 71 (CPU), a scenic image processor 71 (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 71, but may be implemented separately through a chip.

[0082] Among them, the memory 72 may include a random access memory 72 (Random Access Memory, RAM), and may also include a read-only memory 72 (Read-Only Memory). Optionally, the memory 72 includes a non-transitory computer-readable storage medium. The memory 72 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 72 may also be optionally at least one storage device located away from the aforementioned processor 71.

[0083] The present application also provides a storage medium that can store multiple instructions, which are suitable for the processor to load and execute the above-mentioned Figures 1 to 5 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 5 The specific description shown will not be repeated here.

[0084] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0087] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0091] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. A method for processing scenic area images, characterized in that: The following steps are involved: Obtaining a scenic area image to be processed, determining a target area in the scenic area image to be processed, and removing the target area in the scenic area image to be processed to obtain a removed image; Obtaining image attribute information of the scenic spot image to be processed, and obtaining a similar image corresponding to the eliminated image according to the image attribute information; Determine a target area in the similar image, and complete the eliminated image according to the target area in the similar image to obtain a completed image; The completed image and image attribute information are input into a preset image restoration neural network model, and image restoration is performed on the completed image to obtain a restored image of the scenic area image to be processed.

2. The method for processing scenic area images according to claim 1, characterized in that: The image attribute information includes location information, time period information, weather information and building information, wherein the building information includes main building image information and building angle information; The method of obtaining a similar image corresponding to the removed image according to the image attribute information comprises the following steps: According to the main building image information in the image attribute information, a plurality of candidate scenic area images and image attribute information of the candidate scenic area images are obtained; According to the location information, time period information, weather information and building angle information in the image attribute information, a similarity calculation method is adopted to obtain the similarity between the eliminated image and several candidate scenic spot images, and the candidate scenic spot image with the highest similarity is taken as the similar image to obtain the similar image corresponding to the eliminated image.

3. The method for processing scenic area images according to claim 2, characterized in that: The method of inputting the completed image and the image attribute information into a preset image restoration neural network model, performing image restoration on the completed image, and obtaining a restored image of the scenic spot image to be processed comprises the following steps: Determining a repair zone area of ​​the completed image according to an edge line of a target area in the completed image; According to the image attribute information, the restoration zone area of ​​the completed image is subjected to virtual and enhanced diffusion processing to obtain a restoration image of the scenic area image to be processed.

4. The method for processing scenic area images according to claim 3, characterized in that: It also includes the steps of training the image restoration neural network model; The training of the image restoration neural network model comprises the steps of: A training image set and a verification image set are obtained, a number of verification images are extracted from the verification image set, and the number of verification images are input into the image restoration neural network model to be trained to perform image mask random coverage processing to obtain mask images corresponding to the number of verification images, wherein the training image set includes a number of training images, and the verification image set includes a number of verification images; Performing scenic spot image retrieval according to the plurality of mask images and a training image set, extracting a plurality of training images corresponding to the mask images from the training image set as similar images corresponding to the mask images; According to the mask areas in the plurality of mask images, the mask areas in the similar images corresponding to the mask images are confirmed, and the mask areas in the similar images are used to complement the mask areas in the corresponding mask images, so as to obtain complement images corresponding to the plurality of verification images; Obtaining image attribute information of several verification images, inputting the complement images and image attribute information corresponding to the several verification images into the image restoration neural network model to be trained to perform image restoration, and obtaining the restoration images corresponding to the several verification images; The image restoration neural network model to be trained is trained based on the plurality of verification images and the corresponding restoration images and the discriminator of the image restoration neural network model to be trained.

5. The method for processing scenic area images according to claim 4, characterized in that: The image restoration neural network model to be trained includes a global discriminator and a local discriminator; The method of inputting a plurality of the verification images and the corresponding restoration images into the discriminator of the image restoration neural network model to be trained, and training the image restoration neural network model to be trained comprises the steps of: According to the global discriminator, global features of the verification images and the corresponding restoration images are extracted to obtain global features of the verification images and the corresponding restoration images; according to the local discriminator, local features of the restoration band areas of the verification images and the corresponding restoration images are extracted to obtain local features of the verification images and the corresponding restoration images; Loss calculation is performed based on the global features of several of the verification images and the corresponding repaired images to obtain a global loss value, loss calculation is performed based on the local features of several of the verification images and the corresponding repaired images to obtain a local loss value, and the image repair neural network model to be trained is trained based on the global loss value and the local loss value.

6. A device for processing scenic area images, characterized in that: include: An image rejection module is used to obtain a scenic area image to be processed, determine a target area in the scenic area image to be processed, and reject the target area in the scenic area image to be processed to obtain a rejected image; An image retrieval module, used to obtain image attribute information of the scenic spot image to be processed, and obtain similar images corresponding to the removed images according to the image attribute information; An image completion module, used for determining a target area in the similar image, and completing the eliminated image according to the target area in the similar image to obtain a completed image; The image restoration module is used to input the completed image and image attribute information into a preset image restoration neural network model, perform image restoration on the completed image, and obtain a restored image of the scenic area image to be processed.

7. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the method for processing a scenic image as claimed in any one of claims 1 to 5 are implemented.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing a scenic spot image according to any one of claims 1 to 5 are implemented.

Images