Image processing method and device, storage medium and electronic device

Abstract

The present disclosure provides an image processing method and device, a storage medium and an electronic device, and relates to the field of artificial intelligence technology. The method includes: determining the latent space tensor mapping domain of the image to be processed according to the image to be processed; associating the latent space tensor in the latent space tensor mapping domain with the pixel points of the image to be processed; filling the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determining the latent space filling tensor mapping domain; locally combining the latent space filling tensors in the latent space tensor filling mapping domain to determine the weighting coefficient; determining the pixel area vector with the set pixel point of the image to be processed as the center; determining the output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector, using a trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

Description

Background Art

[0002] With the continuous development of computer technology, the clarity of images and videos has gradually improved. However, many images and videos are still too blurry, such as the low image quality of some early devices due to hardware limitations and the inevitable information loss caused by compression during transmission. How to restore these low-resolution images to high-resolution images through super-resolution reconstruction technology has become a hot research topic.

[0003] In the existing technology, on the one hand, it relies on special hardware, such as a camera with higher resolution.

[0004] On the other hand, the algorithm structure is not reasonable enough. There is no optimization and improvement for the sudden change of position information in the image. It cannot meet the requirements of training with information around pixels, resulting in the reconstructed super-resolution image may have problems such as artifacts and blurred contours.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present disclosure provides an image processing method and device, a storage medium and an electronic device, which at least to a certain extent overcome the problem of poor image reconstruction effect due to the limitations of the related art.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to one aspect of the present disclosure, there is provided an image processing method, comprising: determining a latent space tensor mapping domain of the image to be processed according to the image to be processed; associating a latent space tensor in the latent space tensor mapping domain with a pixel point of the image to be processed; filling the latent space tensor in the latent space tensor mapping domain within a set range of the pixel point into the pixel point for combination to determine a latent space filling tensor mapping domain; locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine a weighting coefficient; determining a pixel area vector with the set pixel point of the image to be processed as the center; determining an output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector using a trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

[0009] In some embodiments, based on the image to be processed, determining the latent space tensor mapping domain of the image to be processed includes: performing dimensionality reduction coding processing on the image to be processed to determine the latent space tensor associated with each pixel point of the image to be processed; and determining the latent space tensor mapping domain of the image to be processed based on the latent space tensor associated with each pixel point of the image to be processed.

[0010] In some embodiments, the latent space tensor of the latent space tensor mapping domain within the set range of the pixel point is filled into the pixel point for combination, and the latent space filling tensor mapping domain is determined, including: determining the set range according to the size of the image to be processed; dividing the image to be processed into multiple expansion areas according to the set range, and the expansion area includes at least one central pixel point and multiple surrounding pixel points; with the central pixel point in the expansion area as the center, the latent space tensor of the latent space tensor mapping domain corresponding to the surrounding pixel points in the expansion area is filled into the central pixel point, so that the latent space tensors corresponding to the central pixel point and the surrounding pixel points are combined to determine the latent space filling tensor associated with the central pixel point; according to the latent space filling tensor associated with the pixel point, the latent space filling tensor mapping domain is determined.

[0011] In some embodiments, a latent space filling tensor mapping domain is determined based on a latent space filling tensor associated with the pixel point, including: moving the expanded area, replacing a central pixel point of the expanded area with surrounding pixels, and determining a new central pixel point and a latent space filling tensor associated with the new central pixel point; cyclically moving the expanded area until all pixel points in the image to be processed are associated with a latent space filling tensor; and determining a latent space filling tensor mapping domain based on the latent space filling tensors associated with all pixel points of the image to be processed.

[0012] In some embodiments, the latent space filling tensors in the latent space filling tensor mapping domain are locally combined to determine the weighting coefficient, including: locally combining the pixel point of the image to be processed with the surrounding latent space filling tensors to determine a plurality of latent space filling tensors that are closest to the pixel point; and determining the weighting coefficient by weighted summing up the distance between the pixel point and each of the plurality of nearest latent space filling tensors.

[0013] In some embodiments, a pixel area vector is determined with a set pixel point of the image to be processed as the center, including: determining the pixel area area with a set pixel point of the image to be processed as the center and a target area width and a target area height; determining a two-dimensional vector based on the target area width and the target area height; determining a pixel area vector based on the two-dimensional vector and the pixel area.

[0014] In some embodiments, a pixel area vector is determined based on the two-dimensional vector and the pixel area area, including: dividing the image to be processed into multiple image blocks based on the pixel area area; the width of the image block is the target area width, and the height of the image block is the target area height; and the center coordinates of multiple image blocks in the image to be processed are connected with the two-dimensional vector to determine the pixel area vector.

[0015] In some embodiments, the training process of the image generation model includes: acquiring a first resolution image; determining a second resolution image corresponding to the first resolution image; the resolution of the second resolution image is smaller than the resolution of the first resolution image; and training the image generation model based on the first resolution image and the second resolution image to determine a trained image generation model.

[0016] In some embodiments, determining the second resolution image corresponding to the first resolution image includes: performing random downsampling processing on the first resolution image to determine the second resolution image.

[0017] In some embodiments, the image generation model is trained according to the first resolution image and the second resolution image to determine the trained image generation model, including: determining the latent space tensor mapping domain of the second resolution image according to the second resolution image; associating the latent space tensor in the latent space tensor mapping domain with the pixel points of the second resolution image; filling the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points into the pixel points for combination to determine the latent space filling tensor mapping domain; locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine the weighting coefficient; determining the pixel area vector with the set pixel point of the second resolution image as the center; determining the first resolution pixel according to the first resolution image; inputting the first resolution pixel, the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector into the image generation model to determine the predicted image; determining the loss value according to the predicted image and the first resolution image; training the image generation model according to the loss value so that the resolution of the predicted image determined by the image generation model approaches that of the first resolution image, and determining the trained image generation model.

[0018] In some embodiments, determining a loss value based on the predicted image and the first resolution image includes: calculating the degree of convergence between the predicted image and the first resolution image through a loss function to determine the loss value.

[0019] According to another aspect of the present disclosure, there is also provided an image processing device, comprising: a latent space tensor mapping domain determination module, used to determine the latent space tensor mapping domain of the image to be processed according to the image to be processed; the latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the image to be processed; a latent space filling tensor mapping domain determination module, used to fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points into the pixel points for combination, and determine the latent space filling tensor mapping domain; a weighting coefficient determination module, used to locally combine the latent space filling tensors in the latent space filling tensor mapping domain to determine the weighting coefficient; a pixel area vector determination module, used to determine the pixel area vector with the set pixel point of the image to be processed as the center; an output image determination module, used to determine the output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector, using a trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

[0020] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned image processing methods by executing the executable instructions.

[0021] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method described in any one of the above is implemented.

[0022] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned image processing methods.

[0023] The image processing method and device, storage medium and electronic device provided in the embodiments of the present invention construct a method for continuously representing images based on the idea of ​​continuous image storage. On the one hand, the latent space tensor within the set range of the pixel point is filled into the pixel point for combination to determine the mapping domain of the latent space filling tensor. By filling the latent space tensor around the pixel point into the pixel point for combination, the information around the pixel point is enriched, so that the output image is smoother at the continuous part and the contour is clear. On the other hand, the weighting coefficient is determined by locally combining the latent space filling tensor, which can avoid the problem of inaccurate prediction results caused by the sudden change of the nearest tensor of the pixel point, so that the graphic boundary of the output image at the sudden change is clearer. Combining the above two aspects and the pixel area vector determined with the set pixel point as the center, the goal of continuously storing the image is achieved, so that the contour of the output image obtained by super-resolution reconstruction using the trained image generation model is clearer and the generation of artifacts is reduced. The present invention realizes super-resolution reconstruction of input low-resolution images to be processed, does not require special high-definition image acquisition equipment, reduces hardware costs, has low requirements for image acquisition, can perform super-resolution reconstruction of a variety of images and videos, improves the algorithm effect, and provides strong support for many image processing tasks, such as efficient restoration of compressed images, blurred image repair, etc., and has good commercial value.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0026] Figure 1 A schematic diagram showing the system structure of an image processing method in an embodiment of the present disclosure.

[0027] Figure 2 A schematic diagram of an image processing method in an embodiment of the present disclosure is shown.

[0028] Figure 3 A schematic diagram of a process for determining a latent space tensor mapping domain of an image processing method in an embodiment of the present disclosure is shown.

[0029] Figure 4 A schematic diagram of a process of determining a latent space filling tensor mapping domain in an image processing method according to an embodiment of the present disclosure is shown.

[0030] Figure 5 A schematic diagram of a process for determining weighting coefficients of an image processing method in an embodiment of the present disclosure is shown.

[0031] Figure 6 A schematic diagram of a process of determining a pixel area vector in an image processing method in an embodiment of the present disclosure is shown.

[0032] Figure 7 A schematic diagram showing the training process of an image generation model of an image processing method in an embodiment of the present disclosure.

[0033] Figure 8 A training flowchart of an image generation model of an image processing method in an embodiment of the present disclosure is shown.

[0034] Fig. 9 A schematic diagram of an image processing device in an embodiment of the present disclosure is shown.

[0035] Fig.10 A structural block diagram of a computer device for an image processing method in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0037] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0038] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0039] Figure 1 FIG. 1 shows an exemplary application system architecture diagram to which an image processing method according to an embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0040] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0041] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0042] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.

[0043] Optionally, the client of the application installed in different terminal devices 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0044] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0045] Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0046] Those skilled in the art will know that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0047] Under the above system architecture, an image processing method is provided in an embodiment of the present disclosure. The method can be executed by any electronic device with computing and processing capabilities.

[0048] In some embodiments, an image processing method provided in an embodiment of the present disclosure may be executed by a terminal device of the above-mentioned system architecture; in other embodiments, an image processing method provided in an embodiment of the present disclosure may be executed by a server in the above-mentioned system architecture; in other embodiments, an image processing method provided in an embodiment of the present disclosure may be implemented by a terminal device and a server in the above-mentioned system architecture through interaction.

[0049] Figure 2 A flow chart of an image processing method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, an image processing method provided in an embodiment of the present disclosure includes the following steps:

[0050] S202: determining a latent space tensor mapping domain of the image to be processed according to the image to be processed; and associating a latent space tensor in the latent space tensor mapping domain with a pixel point of the image to be processed;

[0051] S204: Filling the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determining a latent space filling tensor mapping domain;

[0052] S206: locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine weighting coefficients;

[0053] S208: Determine a pixel region vector with a set pixel point of the image to be processed as the center;

[0054] S2010: Determine an output image according to the latent space filling tensor mapping domain, the weighting coefficients and the pixel area vector using a trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

[0055] The disclosed embodiment achieves the enrichment of information around pixel points by filling the latent space tensor around pixel points into the pixel points for combination, so that the output image is smoother at the continuous parts and the contour is clear; the latent space filling tensor is locally combined to determine the weighting coefficient, which can avoid the problem of inaccurate prediction results caused by the sudden change of the nearest neighbor tensor of the pixel point, so that the graphic boundary of the output image at the sudden change part is clearer, and combined with the pixel area vector determined with the set pixel point as the center, the goal of continuous image storage is achieved, so that the contour of the output image obtained by super-resolution reconstruction using the trained image generation model is clearer and the generation of artifacts is reduced.

[0056] like Figure 3 As shown, in the embodiment, determining the latent space tensor mapping domain of the image to be processed according to the image to be processed in the above step S202 may include:

[0057] S302: performing dimensionality reduction coding processing on the image to be processed to determine a latent space tensor associated with each pixel point of the image to be processed;

[0058] S304: Determine a latent space tensor mapping domain of the image to be processed according to the latent space tensor associated with each pixel point of the image to be processed.

[0059] For the image to be processed, dimensionality reduction and encoding are performed to obtain the latent space tensor associated with each pixel of the image to be processed; the latent space is the space where the feature vector is located after the original image to be processed is compressed and encoded, which belongs to the high-dimensional representation space of observable data, and the latent space tensor is a high-order generalization of the high-dimensional representation space, which belongs to a high-dimensional vector. The above processing process can be obtained by the network model technology constructed by network structures such as VGG-16, GoogLeNet, and ResNet. For the latent space tensors obtained above, each latent space tensor is configured according to the arrangement of pixels, and the latent space tensor mapping domain of the image to be processed can be obtained.

[0060] like Figure 4 As shown, in the embodiment, the step S204 of filling the latent space tensor in the latent space tensor mapping domain within the pixel point setting range to the pixel points for combination to determine the latent space filling tensor mapping domain may include:

[0061] S402: Determine a setting range according to the size of the image to be processed;

[0062] S404: Dividing the image to be processed into a plurality of expansion areas according to the set range, wherein the expansion areas at least include a central pixel point and a plurality of surrounding pixel points;

[0063] S406: Taking the central pixel point in the expanded area as the center, filling the latent space tensor of the latent space tensor mapping domain corresponding to the surrounding pixels in the expanded area to the central pixel point, so that the latent space tensors corresponding to the central pixel point and the surrounding pixels are combined, and the latent space filling tensor associated with the central pixel point is determined;

[0064] S408: Determine a latent space filling tensor mapping domain according to the latent space filling tensor associated with the pixel point.

[0065] When the image is generated and trained without combining the information around the pixel point, the obtained image will have a problem of blurred contour. In order to solve the above problem, the embodiment of the present invention first determines a set range according to the size of the image to be processed, for example, the set range can be 3×3 pixels. According to the above set range, the image to be processed is divided into multiple expansion areas, each of which includes a central pixel and eight other surrounding pixels. The central pixel point is the center, and the latent space tensor corresponding to the surrounding pixels in the expansion area is filled to the central pixel point, so that the latent space tensor corresponding to the central pixel point and the surrounding pixel points are combined to obtain the latent space filling tensor. Finally, each latent space filling tensor is configured according to the arrangement of the pixels, and the mapping domain of the latent space filling tensor can be obtained. The latent space tensor is enriched with information to achieve the expansion of the mapping domain. The latent space filling tensor can be used to store continuous information, and the pixel information around each pixel point is stored through a high-dimensional vector. Through this embodiment, the latent space tensor around the pixel point is filled to the pixel point for combination, which realizes the enrichment of the information around the pixel point, making the output image smoother at the continuous place and ensuring the clarity of the contour.

[0066] In an embodiment, determining the latent space filling tensor mapping domain according to the latent space filling tensor associated with the pixel point in the above step S408 may include:

[0067] Move the expanded area, replace the central pixel of the expanded area with surrounding pixels, and determine a new central pixel and a latent space filling tensor associated with the new central pixel;

[0068] Circularly moving the expanded area until all pixels in the image to be processed are associated with latent space filling tensors;

[0069] According to the latent space filling tensor associated with all the pixels of the image to be processed, a latent space filling tensor mapping domain is determined.

[0070] For a single expanded area, the latent space filling tensor corresponding to a single central pixel is obtained by calculation. For the surrounding pixels, their corresponding latent space filling tensors also need to be calculated. First, the expanded area is moved so that the original central pixel becomes a surrounding pixel, and one of the surrounding pixels becomes the central pixel. The new central pixel and the corresponding information surrounding pixels are re-determined in the above manner, and the latent space filling tensor associated with the new central pixel is calculated; the expanded area is moved cyclically, and the latent space filling tensor associated with the new central pixel is continuously calculated until all pixels in the image to be processed are associated with a latent space filling tensor. In this way, each latent space filling tensor is configured according to the arrangement of pixels, and the latent space filling tensor mapping domain can be obtained.

[0071] like Figure 5 As shown, in the embodiment, the step S206 in the above-mentioned step of locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine the weighting coefficients includes:

[0072] S502: locally combining the pixel point of the image to be processed with surrounding latent space filling tensors to determine a plurality of latent space filling tensors that are closest to the pixel point;

[0073] S504: Determine a weighting coefficient by performing weighted summation based on the distance between the pixel point and each of the multiple most adjacent latent space filling tensors.

[0074] Since there is no optimization improvement for the sudden change of position information in the image, the obtained image will have the problem of blurred image boundary and artifacts. In order to solve the above problems, the surrounding latent space filling tensors around the pixel point of the image to be processed are first locally combined to obtain multiple latent space filling tensors that are closest to the pixel point; for example, the nearest latent space filling tensor can be the latent space filling tensor corresponding to the upper left pixel point of the pixel point, the latent space filling tensor corresponding to the lower left pixel point, the latent space filling tensor corresponding to the upper right pixel point, and the latent space filling tensor corresponding to the lower right pixel point. Through local combination, inaccurate prediction results caused by sudden changes in the nearest tensor of the pixel point can be avoided. Next, the weighted sum is performed according to the distance between the pixel point and each of the nearest multiple latent space filling tensors to determine the weight coefficient. The above distance can be the Euclidean distance, then the weight coefficient = the Euclidean distance between the target pixel point and a certain nearest latent space filling tensor / the sum of the Euclidean distances between the target pixel point and each nearest latent space filling tensor. By locally combining the latent space filling tensors to determine the weighting coefficients in this embodiment, the problem of inaccurate prediction results caused by sudden changes in the nearest neighbor tensors of individual pixel points can be avoided, making the graphic boundaries of the output image clearer at the sudden changes.

[0075] like Figure 6 As shown, in the embodiment, determining the pixel area vector with the set pixel point of the image to be processed as the center in the above step S208 may include:

[0076] S602: Taking a set pixel point of the image to be processed as the center, determining the area of ​​the pixel region by using a target region width and a target region height;

[0077] S604: Determine a two-dimensional vector according to the target area width and the target area height;

[0078] S606: Determine a pixel region vector according to the two-dimensional vector and the pixel region area.

[0079] When predicting the RGB value of an image in a generated image, the pixel information of other coordinate points in the region except for the pixel at the center coordinate may be discarded. In order to avoid the above problem, the embodiment of the present disclosure first determines the pixel area with the set pixel point of the image to be processed as the center, and the target area width and the target area height; the above pixel area includes the coordinates of the center. Next, the target area width and the target area height are determined as a two-dimensional vector, and the pixel area vector is determined using the two-dimensional vector and the pixel area area obtained above. By replacing the target pixel point with the pixel area vector, that is, decoding the image block, it is possible to avoid discarding the pixel information of other coordinate points in the region except for the pixel at the center coordinate when predicting the RGB value of the image.

[0080] In the embodiment, determining the pixel region vector according to the two-dimensional vector and the pixel region area in S606 may include:

[0081] According to the area of ​​the pixel region, the image to be processed is divided into a plurality of image blocks; the width of the image block is the width of the target region, and the height of the image block is the height of the target region;

[0082] The center coordinates of multiple image blocks in the image to be processed are connected with the two-dimensional vector to determine a pixel area vector.

[0083] When determining the pixel region vector, firstly, the image to be processed is divided into a plurality of image blocks according to the pixel region area, for example, the image to be processed is divided into 16 image blocks of 4×4, the width of these image blocks is the width of the target region, and the height of the image blocks is the height of the target region. Next, the center coordinates of the image block are connected with the two-dimensional vector to determine the pixel region vector. Specifically, the width and height of the image block can be put into an array using the center coordinates, and combined with the two-dimensional vector to obtain a high-dimensional pixel region vector.

[0084] like Figure 7 As shown, in the embodiment, the training process of the image generation model includes:

[0085] S702: Acquire a first resolution image;

[0086] S704: Determine a second resolution image corresponding to the first resolution image; the resolution of the second resolution image is smaller than the resolution of the first resolution image;

[0087] S706: Train the image generation model according to the first resolution image and the second resolution image, and determine a trained image generation model.

[0088] The specific training process of the above-mentioned trained image generation model includes: acquiring a first resolution image through a high-resolution device, and then processing the first resolution image to reduce its resolution to obtain a corresponding second resolution image. The image generation model is trained using a high-resolution first resolution image and a low-resolution second resolution image to determine the trained image generation model. In an embodiment, the above-mentioned determination of the second resolution image corresponding to the first resolution image includes: randomly downsampling the first resolution image to determine the second resolution image. In the training process of this embodiment, by introducing the idea of ​​a generative adversarial network and training the image generation model according to the first resolution image and the second resolution image, the output image of the trained prediction model can be made more credible.

[0089] In the embodiment, Figure 8 As shown, the step of training the image generation model according to the first resolution image and the second resolution image in S706 to determine the trained image generation model may include:

[0090] Determine a latent space tensor mapping domain of the second resolution image according to the second resolution image; and associate a latent space tensor in the latent space tensor mapping domain with a pixel point of the second resolution image;

[0091] Fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determine the latent space filling tensor mapping domain;

[0092] Locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine weighting coefficients;

[0093] Determining a pixel area vector with a set pixel point of the second resolution image as the center;

[0094] Determining first resolution pixels according to the first resolution image;

[0095] Inputting the first resolution pixel, the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector into the image generation model to determine a predicted image;

[0096] Determining a loss value according to the predicted image and the first resolution image;

[0097] The image generation model is trained according to the loss value so that the resolution of the predicted image determined by the image generation model approaches that of the first resolution image, and a trained image generation model is determined.

[0098] In an embodiment, determining the loss value according to the predicted image and the first resolution image includes:

[0099] The degree of approximation between the predicted image and the first resolution image is calculated using a loss function to determine a loss value.

[0100] The training process of the above-mentioned image generation model is to first determine the latent space tensor mapping domain of the second resolution image of the low-resolution second resolution image; wherein the latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the second resolution image; next, fill the latent space tensor of the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination to determine the latent space filling tensor mapping domain; next, locally combine the latent space filling tensor in the latent space filling tensor mapping domain to determine the weighting coefficient; next, determine the pixel area vector with the set pixel point of the second resolution image as the center; next, determine the first resolution pixel according to the first resolution image; the first resolution image can be processed by the pixel sampling technology to obtain the first resolution pixel. The first resolution pixel, the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector are input into the image generation model to determine the predicted image; the image generation model can be constructed by the ResNet network structure. Next, determine the loss value according to the predicted image and the first resolution image; specifically, the loss function can be used to calculate the degree of convergence between the predicted image and the first resolution image to determine the loss value. The calculation process can also use the model constructed by the ResNet network structure to calculate the loss value. Finally, the image generation model is mutually trained according to the loss value so that the resolution of the predicted image determined by the image generation model approaches the first resolution image, and the trained image generation model is determined. In this embodiment, the image generation model is trained by using the loss value so that the resolution of the predicted image approaches the first resolution image, and the idea of ​​adversarial generation network is introduced. In the process of image super-resolution reconstruction, a large amount of information differences need to be made up. At present, most technologies need to rely on special acquisition equipment to provide richer original information. By generating and introducing the idea of ​​adversarial network, the information is supplemented by using its generation ability, so that the texture details of the high-resolution image outputted in the end are clearer and artifacts are reduced. Furthermore, when the original adversarial generation network predicts the image, there may be artifacts in itself, which need to be solved through a lot of training, but the solution effect is not very good. The present disclosure can solve the problem of artifacts by constructing a continuous representation image, and does not require a lot of training, saving hardware resources. Moreover, due to the idea of ​​adversarial generation network, the credibility of the output image reconstructed by super-resolution is higher, the hardware cost is reduced, and the algorithm effect is improved, which has certain commercial value.

[0101] The present invention realizes high-resolution reconstruction of input low-resolution images without special hardware equipment. Compared with existing solutions, based on the idea of ​​continuous image storage, it innovatively proposes to construct continuous representation images, so that the contour of the reconstructed high-resolution image is clearer and the generation of artifacts is reduced. By expanding the mapping domain, the pixel point information of the adjacent area is filled into each pixel point; the information of the adjacent area is weighted through local combination; the information outside the center point of the small area of ​​the image is retained through the graphic decoding block, and finally the purpose of continuous image storage is achieved. The high-resolution image output in the end is smoother in the continuous part and the graphic boundary in the mutation part is clearer.

[0102] The present disclosure also provides a training process for an image generation model, including:

[0103] Dataset preprocessing stage: S1-1: Input training data, obtain high-resolution images through external devices as data sets. S1-2: Obtain low-resolution images through random downsampling. Process the high-resolution images obtained externally through random downsampling technology to obtain low-resolution images for training, which are used as input of the encoder module in the training stage. S1-3: Obtain high-resolution pixels through pixel sampling. Process the high-resolution images obtained externally through pixel sampling technology to obtain high-resolution pixels for training, which are used as input of the generator module in the training stage. S1-4: Keep the real high-resolution images. Keep the high-resolution images obtained externally, which are used as input of the discriminator module in the training stage.

[0104] Training phase: S2-1: Construct encoder module, mapping domain expansion module, local combination module, graphic block decoding module, generator module, and discriminator module; the encoder module is constructed by the VGG-16 network structure; the mapping domain expansion module is obtained by storing the pixel information around each pixel in a high-dimensional vector; the weight coefficient of the local combination module is calculated by the formula "weight coefficient = Euclidean distance between the target pixel and a certain pixel / the sum of the Euclidean distances between the target pixel and each pixel"; the graphic block decoding module is obtained by dividing the image by a given area size and center point and connecting the center point coordinates with the height and width information; the generator module is constructed by the ResNet network structure; the discriminator module is constructed by ResNet. S2-2: The latent space tensor is obtained through the encoder module, and the low-resolution image obtained by random downsampling is subjected to dimensionality reduction to obtain the latent space tensor. S2-3: The new mapping domain with enriched information is obtained through the mapping domain expansion module, and the latent space tensor of each pixel point is combined with the latent space tensor of the pixel points in the surrounding 3*3 area to enrich the latent space tensor information of the image, thereby obtaining the combined radiation domain of the corresponding input image and the associated information of each position for storing continuous information. S2-4: The weighted coefficient of the latent space vector of the neighboring point is obtained through the local expansion module, and the weighted sum of the four latent space tensors closest to the target point in the mapping domain is taken to avoid the inaccurate prediction result caused by the sudden change of the nearest tensor of the pixel point alone, and the weighted coefficient of the target point and each neighboring point is obtained. S2-5: The graphic block information is obtained through the graphic block decoding module, the target pixel point is replaced with the target pixel area, the height and width of the area and the center of the area are obtained, and the image block position information is stored in a high-dimensional vector. S2-6: The predicted high-resolution image is obtained through the generator module, and the high-resolution image is predicted and trained with the discriminator module. S2-7: The authenticity score of the predicted image is obtained through the discriminator module, and the authenticity of the high-resolution image generated by the generator module is identified. The loss value is obtained through the loss function, and the generator module is trained to make the generated image continuously approach the real high-resolution image.

[0105] Reasoning stage: S3-1: Through the generator module, a predicted high-resolution image is obtained. After the model training is completed, a low-resolution image taken by a mobile phone is used as input, and finally a high-resolution image is output.

[0106] It should be noted that the acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations, and various types of data such as personal identity data, operation data, behavioral data, etc. related to individuals, customers, and groups obtained in the embodiments of this disclosure have all been authorized.

[0107] Based on the same inventive concept, an image processing device is also provided in the embodiment of the present disclosure, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0108] Fig. 9 A schematic diagram of an image processing device in an embodiment of the present disclosure is shown. Fig. 9 As shown, the device comprises:

[0109] A latent space tensor mapping domain determining module 901 is used to determine the latent space tensor mapping domain of the image to be processed according to the image to be processed; the latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the image to be processed;

[0110] A latent space filling tensor mapping domain determining module 902 is used to fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel point into the pixel point for combination, and determine the latent space filling tensor mapping domain;

[0111] A weight coefficient determination module 903, configured to locally combine the latent space filling tensors in the latent space filling tensor mapping domain to determine the weight coefficients;

[0112] A pixel region vector determination module 904 is used to determine a pixel region vector with a set pixel point of the image to be processed as the center;

[0113] The output image determination module 905 is used to determine the output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector, using the trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

[0114] It should be noted that the above-mentioned latent space tensor mapping domain determination module 901, latent space filling tensor mapping domain determination module 902, weight coefficient determination module 903, pixel area vector determination module 904 and output image determination module 905 correspond to S202 to S2010 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0115] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0116] Refer to the following Fig.10 The electronic device 1000 according to this embodiment of the present disclosure is described. Fig.10 The electronic device 1000 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0117] like Fig.10 As shown, the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010, the at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0118] The storage unit stores a program code, and the program code can be executed by the processing unit 1010, so that the processing unit 1010 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 1010 can execute the following steps of the above method embodiment: according to the image to be processed, determine the latent space tensor mapping domain of the image to be processed; the latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the image to be processed; fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determine the latent space filling tensor mapping domain; locally combine the latent space filling tensors in the latent space filling tensor mapping domain to determine the weighting coefficient; determine the pixel area vector with the set pixel point of the image to be processed as the center; determine the output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector, using the trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed.

[0119] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202 , and may further include a read-only storage unit (ROM) 10203 .

[0120] The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205, such program modules 10205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0121] Bus 1030 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0122] The electronic device 1000 may also communicate with one or more external devices 1040 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1050. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1060. As shown, the network adapter 1060 communicates with other modules of the electronic device 1000 via a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0124] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above-mentioned image processing method when executed by a processor.

[0125] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0126] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0128] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0129] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0130] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0131] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0132] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0133] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. An image processing method, characterized in that: include: Determining a latent space tensor mapping domain of the image to be processed according to the image to be processed; The latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the image to be processed; Fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determine the latent space filling tensor mapping domain; Locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine weighting coefficients; Determine a pixel region vector with a set pixel point of the image to be processed as the center; Determine an output image using a trained image generation model according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel region vector; the resolution of the output image is greater than the resolution of the image to be processed; The method of filling the latent space tensor in the latent space tensor mapping domain within the pixel point setting range to the pixel points for combination and determining the latent space filling tensor mapping domain comprises: Determining a setting range according to the size of the image to be processed; According to the set range, the image to be processed is divided into a plurality of expansion areas, wherein the expansion areas at least include a central pixel point and a plurality of surrounding pixel points; Taking the central pixel point in the expanded area as the center, filling the latent space tensor of the latent space tensor mapping domain corresponding to the surrounding pixels in the expanded area to the central pixel point, so that the latent space tensors corresponding to the central pixel point and the surrounding pixels are combined, and determining the latent space filling tensor associated with the central pixel point; Move the expanded area, replace the central pixel of the expanded area with surrounding pixels, and determine a new central pixel and a latent space filling tensor associated with the new central pixel; Circularly moving the expanded area until all pixels in the image to be processed are associated with latent space filling tensors; According to the latent space filling tensor associated with all the pixels of the image to be processed, a latent space filling tensor mapping domain is determined.

2. The image processing method according to claim 1, characterized in that: Determining a latent space tensor mapping domain of the image to be processed according to the image to be processed includes: Performing dimensionality reduction coding processing on the image to be processed to determine a latent space tensor associated with each pixel point of the image to be processed; According to the latent space tensor associated with each pixel point of the image to be processed, a latent space tensor mapping domain of the image to be processed is determined.

3. The image processing method according to claim 1, characterized in that: Locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine weighting coefficients includes: Locally combining the pixel point of the image to be processed with surrounding latent space filling tensors to determine a plurality of latent space filling tensors that are closest to the pixel point; A weighted sum is performed according to the distance between the pixel point and each latent space filling tensor of the most adjacent multiple latent space filling tensors to determine a weighting coefficient.

4. The image processing method according to claim 1, characterized in that: Determining a pixel region vector with a set pixel point of the image to be processed as the center includes: Taking the set pixel point of the image to be processed as the center, determining the area of ​​the pixel region by the target region width and the target region height; Determine a two-dimensional vector according to the target area width and the target area height; A pixel region vector is determined according to the two-dimensional vector and the pixel region area.

5. The image processing method according to claim 4, characterized in that: Determining a pixel region vector according to the two-dimensional vector and the pixel region area includes: According to the area of ​​the pixel region, the image to be processed is divided into a plurality of image blocks; the width of the image block is the width of the target region, and the height of the image block is the height of the target region; The center coordinates of multiple image blocks in the image to be processed are connected with the two-dimensional vector to determine a pixel area vector.

6. The image processing method according to claim 1, characterized in that: The training process of the image generation model includes: Acquire a first resolution image; Determine a second resolution image corresponding to the first resolution image; the resolution of the second resolution image is smaller than the resolution of the first resolution image; The image generation model is trained according to the first resolution image and the second resolution image to determine a trained image generation model.

7. The image processing method according to claim 6, characterized in that: Determining a second resolution image corresponding to the first resolution image includes: The first resolution image is randomly down-sampled to determine a second resolution image.

8. The image processing method according to claim 6, characterized in that: The step of training the image generation model according to the first resolution image and the second resolution image, and determining a trained image generation model comprises: Determine a latent space tensor mapping domain of the second resolution image according to the second resolution image; and associate a latent space tensor in the latent space tensor mapping domain with a pixel point of the second resolution image; Fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points to the pixel points for combination, and determine the latent space filling tensor mapping domain; Locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine weighting coefficients; Determining a pixel area vector with a set pixel point of the second resolution image as the center; Determining first resolution pixels according to the first resolution image; Inputting the first resolution pixel, the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector into the image generation model to determine a predicted image; Determining a loss value according to the predicted image and the first resolution image; The image generation model is trained according to the loss value so that the resolution of the predicted image determined by the image generation model approaches that of the first resolution image, and a trained image generation model is determined.

9. The image processing method according to claim 8, characterized in that: Determining a loss value according to the predicted image and the first resolution image includes: The degree of approximation between the predicted image and the first resolution image is calculated using a loss function to determine a loss value.

10. An image processing device, characterized in that: include: A latent space tensor mapping domain determination module, used to determine the latent space tensor mapping domain of the image to be processed according to the image to be processed; The latent space tensor in the latent space tensor mapping domain is associated with the pixel points of the image to be processed; A latent space filling tensor mapping domain determining module, used to fill the latent space tensor in the latent space tensor mapping domain within the set range of the pixel points into the pixel points for combination, and determine the latent space filling tensor mapping domain; A weight coefficient determination module, used for locally combining the latent space filling tensors in the latent space filling tensor mapping domain to determine the weight coefficients; A pixel region vector determination module, used to determine a pixel region vector with a set pixel point of the image to be processed as the center; An output image determination module, used to determine an output image according to the latent space filling tensor mapping domain, the weighting coefficient and the pixel area vector, using a trained image generation model; the resolution of the output image is greater than the resolution of the image to be processed; Among them, the latent space filling tensor mapping domain determination module is specifically used for: Determining a setting range according to the size of the image to be processed; According to the set range, the image to be processed is divided into a plurality of expansion areas, wherein the expansion areas at least include a central pixel point and a plurality of surrounding pixel points; Taking the central pixel point in the expanded area as the center, filling the latent space tensor of the latent space tensor mapping domain corresponding to the surrounding pixels in the expanded area to the central pixel point, so that the latent space tensors corresponding to the central pixel point and the surrounding pixels are combined, and determining the latent space filling tensor associated with the central pixel point; Move the expanded area, replace the central pixel of the expanded area with surrounding pixels, and determine a new central pixel and a latent space filling tensor associated with the new central pixel; Circularly moving the expanded area until all pixels in the image to be processed are associated with latent space filling tensors; According to the latent space filling tensor associated with all the pixels of the image to be processed, a latent space filling tensor mapping domain is determined.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the image processing method according to any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 9 is implemented.

Images