Image Processing Method, Apparatus and Electronic Device

Abstract

An embodiment of the present application provides an image processing method, apparatus, and electronic device. In this method, the electronic device stores image blocks corresponding to panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint and each angle, and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each angle and each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint. The panoramic images at each viewpoint are high-definition images. Compared with the prior art method of storing high-definition images at different angles at each viewpoint, the storage overhead can be reduced. On this basis, image processing can also be realized to obtain images with higher clarity.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image processing, and in particular, to an image processing method, apparatus, and electronic device. Background Art

[0002] With the development of terminals, many current terminals can support high-magnification photographing, and the magnification can be 30 times, 50 times, etc. However, due to the limitations of the structure of some terminals themselves, such as the need for a thin body for mobile phones, when a terminal uses high-magnification photographing, the clarity of the captured image is low and the details are blurred.

[0003] In order to improve the clarity of the images taken by a terminal using high-magnification photographing, a large number of high-definition images at different shooting positions and different shooting perspectives at the same shooting position can be pre-stored. After the terminal uses high-magnification photographing, it can obtain the high-definition image with the highest similarity to the image captured by the terminal from the pre-stored high-definition images, so that the terminal can display the high-definition image. Currently, this method requires storing a large number of high-definition images, resulting in a large storage overhead. Summary of the Invention

[0004] Embodiments of the present application provide an image processing method, apparatus, and electronic device, which can reduce the storage overhead.

[0005] In a first aspect, embodiments of the present application provide an image processing method. The execution subject of this method can be an electronic device or a chip in the electronic device. The following takes a cloud electronic device as an example for illustration. In the electronic device, there are stored image blocks corresponding to panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint and each angle of view and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each angle of view and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint. The panoramic images at each viewpoint are high-definition images, such as images obtained by a device that can capture high-clarity images, such as a single-lens reflex camera.

[0006] In this method, the electronic device can obtain a first image to be processed, and can extract the features in the first image, and then obtain the similarity between the features of the images at each angle of view and the features of the first image. Among them, the similarity between the image of the angle of view with the maximum similarity and the features of the first image is the highest. The electronic device can determine the target identifier mapped by the features of the image of the angle of view corresponding to the maximum similarity according to the features of the image of the angle of view corresponding to the maximum similarity and the mapping relationship.

[0007] Since the image patches corresponding to the panoramic images at each viewpoint are obtained based on the panoramic images at each viewpoint, after obtaining the target identifier of the feature map of the image at the viewpoint corresponding to the maximum similarity, the second image can be obtained according to the image patch corresponding to the target identifier, and the clarity of the second image is higher than that of the first image. The second image is obtained based on the image patches in the panoramic image to which the image patch corresponding to the target identifier belongs.

[0008] Among them, since the clarity of the panoramic images at each viewpoint is greater than or equal to the preset clarity, that is, the clarity of the panoramic images at each viewpoint is higher than that of the first image, and the image patches corresponding to the panoramic images at each viewpoint are obtained based on the panoramic images at each viewpoint, the clarity of the second image obtained based on the image patch corresponding to the target identifier is greater than or equal to the preset clarity. Therefore, by using this method, the terminal can obtain a second image with high clarity, and since what is stored in the electronic device are the image patches corresponding to the panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint and the identifiers of the image patches corresponding to the panoramic images at each viewpoint, compared with the prior art of storing high-definition images at different viewpoints, the storage overhead can be reduced.

[0009] In a possible implementation manner, in order to reduce the calculation amount of the similarity of the electronic device, when the electronic device obtains the first image, it can also obtain the position (i.e., the viewpoint) of the device that captures the first image. In this way, the electronic device can determine the target viewpoints within a preset range from this position, and then obtain the similarity between the features of the images at each target viewpoint and the features of the first image. This can reduce the calculation amount of the terminal for calculating the similarity, and only the similarity between the features of the images at each target viewpoint within a preset range from the position of the terminal and the features of the first image needs to be calculated, without obtaining the similarity between the features of the images at each viewpoint and the features of the first image.

[0010] In a possible implementation manner, the mapping relationship includes a first indexing relationship and a second indexing relationship. The second indexing relationship is: the mapping relationship between the features of the images at each viewpoint and the center points of the images at each viewpoint. The first indexing relationship is: the mapping relationship between the center points of the images at each viewpoint and the identifiers of the image patches corresponding to the panoramic images at each viewpoint. Among them, after the electronic device obtains the features of the image at the viewpoint corresponding to the maximum similarity, it can determine the center point of the feature map of the image at the viewpoint corresponding to the maximum similarity according to the features of the image at the viewpoint corresponding to the maximum similarity and the second indexing relationship, and then determine the target identifier according to the center point of the feature map of the image at the viewpoint corresponding to the maximum similarity and the first indexing relationship.

[0011] In one embodiment, the features of the images of each view angle under each viewpoint stored in the electronic device, the first index relationship, and the second index relationship can be pre-set in the electronic device by the staff or obtained by the electronic device itself.

[0012] Among them, in one embodiment, the electronic device can obtain the features of the images of each view angle under each viewpoint, the first index relationship, and the second index relationship according to the panoramic images under each viewpoint, and then store the features of the images of each view angle under each viewpoint, the first index relationship, and the second index relationship.

[0013] The electronic device can perform back-projection transformation on the panoramic images under each viewpoint to obtain the images of multiple view angles (high-overlap images) under each viewpoint and the coordinate positions of the center points of the images of each view angle under each viewpoint in the corresponding panoramic images. The overlap rate between the images of adjacent view angles under each viewpoint is greater than a preset overlap rate, and then extract the features of the images of each view angle under each viewpoint.

[0014] Among them, the electronic device can directly perform back-projection transformation on the panoramic images under each viewpoint.

[0015] Or, in one embodiment, the electronic device can obtain the panoramic images under each viewpoint according to the low-overlap images, and then perform back-projection transformation on the panoramic images under each viewpoint. In this embodiment, the electronic device can use panoramic image stitching technology to obtain the panoramic images under each viewpoint according to the pre-collected images of multiple view angles under each viewpoint. The overlap rate between the pre-collected images of adjacent view angles under each viewpoint is less than the preset overlap rate, that is, low-overlap images.

[0016] Specifically, the electronic device can use a sliding window with a second preset size to slide in the panoramic images under each viewpoint and perform back-projection transformation to sequentially obtain the images of the view angles corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images of the view angles corresponding to the partial panoramic images in the corresponding panoramic images. The images of each view angle under each viewpoint have the second preset size.

[0017] Among them, the electronic device can construct the second index relationship according to the coordinate positions of the center points of the images of each view angle under each viewpoint in the corresponding panoramic images and the features of the images of each view angle under each viewpoint. And,

[0018] The electronic device can cut the panoramic images at each viewpoint to obtain image blocks corresponding to the panoramic images at each viewpoint, and construct the first index relationship according to the coordinate positions of the center points of the images at each angle in the corresponding panoramic images at each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint. In a possible implementation, the image blocks corresponding to the panoramic images at each viewpoint have a first preset size.

[0019] In the above example, the electronic device uses panoramic image stitching technology to obtain the panoramic images at each viewpoint according to the pre-acquired images at multiple angles at each viewpoint, that is, projects the pre-acquired images at each angle at each viewpoint in the first viewing plane to the second viewing plane to which the panoramic image belongs to obtain the panoramic images at each viewpoint. During this projection process, the electronic device can also obtain the transformation relationship between the first viewing plane and the second viewing plane.

[0020] In a possible implementation, after obtaining the target identifier, the electronic device can project the image block corresponding to the target identifier to the first viewing plane by using inverse projection transformation according to the transformation relationship to obtain the second image.

[0021] In this way, the electronic device can obtain a second image that is in the same viewing plane as the image captured by the terminal. When the user views it, there is no difference in the viewing plane. For the user, the conversion from the first image to the second image is imperceptible, which can improve the user experience.

[0022] In a possible scenario, the electronic device can be a cloud. The terminal can capture a first image, but the clarity of the first image is not high. Therefore, in this scenario, the terminal can send the first image and the position of the terminal when the first image is captured to the cloud. In this way, the cloud can obtain the first image and the position where the first image is captured, and then use the method described in the above possible implementation to obtain the second image. After the cloud obtains the second image, it can send the second image to the terminal so that the terminal can display and store the second image. In this way, the user can see the second image with high clarity on the terminal, which can improve the user experience.

[0023] In a second aspect, an embodiment of the present application provides an image processing method. The execution subject of this method can be a terminal or a chip in the terminal. The following takes the terminal as an example for description.

[0024] The terminal uses a first magnification to capture a first image, the first magnification is greater than or equal to a preset magnification, and the terminal sends the first image to the electronic device. The terminal receives a second image from the electronic device and can display the second image in response to an image display instruction. The clarity of the second image is higher than the clarity of the first image.

[0025] In a possible implementation, the terminal stores image blocks corresponding to panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint for each perspective and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each viewpoint for each perspective and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint, and the clarity of the panoramic images at each viewpoint is greater than or equal to a preset clarity.

[0026] In response to a first image captured at a first magnification, the terminal can obtain the similarity between the features of the images at each viewpoint for each perspective and the features of the first image, and based on the features of the image at the perspective corresponding to the maximum similarity, and the mapping relationship, determine the target identifier mapped by the features of the image at the perspective corresponding to the maximum similarity. The terminal obtains a second image according to the image block corresponding to the target identifier, and the clarity of the second image is higher than that of the first image. Further, in response to an image display instruction, the second image can be displayed.

[0027] In a possible implementation, the obtaining the similarity between the features of the images at each viewpoint for each perspective and the features of the first image includes: determining a target viewpoint within a preset range of the position from the terminal, and obtaining the similarity between the features of the images at each viewpoint for each perspective and the features of the first image at the target viewpoint.

[0028] In a possible implementation, the mapping relationship includes a first index relationship and a second index relationship. The second index relationship is: the mapping relationship between the features of the images at each viewpoint for each perspective and the center points of the images at each viewpoint for each perspective. The first index relationship is: the mapping relationship between the center points of the images at each viewpoint for each perspective and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint.

[0029] The determining the target identifier mapped by the features of the image at the perspective corresponding to the maximum similarity, based on the features of the image at the perspective corresponding to the maximum similarity and the mapping relationship, includes: determining the center point mapped by the features of the image at the perspective corresponding to the maximum similarity according to the features of the image at the perspective corresponding to the maximum similarity and the second index relationship; and determining the target identifier according to the center point mapped by the features of the image at the perspective corresponding to the maximum similarity and the first index relationship.

[0030] In a possible implementation, the method further includes: obtaining the features of the images at each viewpoint for each perspective, the first index relationship, and the second index relationship according to the panoramic images at each viewpoint; and storing the features of the images at each viewpoint for each perspective, the first index relationship, and the second index relationship.

[0031] In a possible implementation, obtaining the features of the images of each view angle at each viewpoint according to the panoramic images at each viewpoint includes: performing back-projection transformation on the panoramic images at each viewpoint to obtain the images of multiple view angles at each viewpoint, and the coordinate positions of the center points of the images of each view angle at each viewpoint in the corresponding panoramic images, where the overlap rate between the images of adjacent view angles at each viewpoint is greater than a preset overlap rate; and extracting the features of the images of each view angle at each viewpoint.

[0032] In a possible implementation, performing back-projection transformation on the panoramic images at each viewpoint to obtain the images of multiple view angles at each viewpoint, and the coordinate positions of the center points of the images of each view angle at each viewpoint in the corresponding panoramic images includes: in the panoramic images at each viewpoint, sliding a sliding window with a second preset size in the panoramic image, and performing back-projection transformation to sequentially obtain the images of the view angles corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images of the view angles corresponding to the partial panoramic images in the corresponding panoramic images, where the images of each view angle at each viewpoint have the second preset size.

[0033] In a possible implementation, obtaining the second index relationship includes: constructing the second index relationship according to the coordinate positions of the center points of the images of each view angle at each viewpoint in the corresponding panoramic images and the features of the images of each view angle at each viewpoint.

[0034] In a possible implementation, obtaining the first index relationship includes: cutting the panoramic images at each viewpoint to obtain the image blocks corresponding to the panoramic images at each viewpoint; and constructing the first index relationship according to the coordinate positions of the center points of the images of each view angle at each viewpoint in the corresponding panoramic images and the image blocks corresponding to the panoramic images at each viewpoint.

[0035] In a possible implementation, the image blocks corresponding to the panoramic images at each viewpoint have a first preset size.

[0036] In a possible implementation, before obtaining the features of the images of each view angle at each viewpoint, the first index relationship, and the second index relationship according to the panoramic images at each viewpoint, it further includes: using panoramic image stitching technology to obtain the panoramic images at each viewpoint according to the pre-acquired images of multiple view angles at each viewpoint, where the overlap rate between the pre-acquired images of adjacent view angles at each viewpoint is less than the preset overlap rate.

[0037] In a possible implementation, the panoramic image stitching technology is adopted to obtain the panoramic images at each viewpoint according to the pre-collected images of multiple perspectives at each viewpoint, including: projecting the pre-collected images of each perspective at each viewpoint in the first visual plane onto the second visual plane to which the panoramic image belongs, so as to obtain the panoramic images at each viewpoint, and the transformation relationship between the first visual plane and the second visual plane.

[0038] In a possible implementation, the obtaining of the second image according to the image block corresponding to the target identifier includes: according to the transformation relationship, performing back-projection transformation on the image block corresponding to the target identifier to project it onto the first visual plane, so as to obtain the second image.

[0039] In a third aspect, an embodiment of the present application provides an image processing apparatus, which may be an electronic device or a chip in an electronic device. The image processing apparatus includes:

[0040] A processing module, configured to obtain the similarity between the features of the images of each perspective at each viewpoint and the features of the first image, determine the target identifier mapped by the features of the image of the perspective corresponding to the maximum similarity according to the features of the image of the perspective corresponding to the maximum similarity and the mapping relationship, and obtain a second image according to the image block corresponding to the target identifier, where the clarity of the second image is higher than that of the first image.

[0041] In a possible implementation, the processing module is specifically configured to obtain the first image, the position where the first image is taken, determine the target viewpoints within a preset range from the position, and obtain the similarity between the features of the images of each perspective at the target viewpoints and the features of the first image.

[0042] In a possible implementation, the mapping relationship includes a first index relationship and a second index relationship. The second index relationship is: the mapping relationship between the features of the images of each perspective at each viewpoint and the center points of the images of each perspective at each viewpoint. The first index relationship is: the mapping relationship between the center points of the images of each perspective at each viewpoint and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint.

[0043] The processing module is specifically configured to determine the center point mapped by the features of the image of the perspective corresponding to the maximum similarity according to the features of the image of the perspective corresponding to the maximum similarity and the second index relationship, and determine the target identifier according to the center point mapped by the features of the image of the perspective corresponding to the maximum similarity and the first index relationship.

[0044] In a possible implementation, the processing module is further configured to obtain the features of the images of each view angle, the first index relationship, and the second index relationship according to the panoramic images at each view point.

[0045] The storage module is configured to store the features of the images of each view angle, the first index relationship, and the second index relationship at each view point.

[0046] In a possible implementation, the processing module is specifically configured to perform back-projection transformation on the panoramic images at each view point to obtain the images of multiple view angles at each view point, and the coordinate positions of the center points of the images of each view angle in the corresponding panoramic images, where the overlap rate between the images of adjacent view angles at each view point is greater than a preset overlap rate; and extract the features of the images of each view angle.

[0047] In a possible implementation, the processing module is specifically configured to slide a sliding window with a second preset size in the panoramic images at each view point, and perform back-projection transformation to sequentially obtain the images of the view angles corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images of the view angles corresponding to the partial panoramic images in the corresponding panoramic images, where the images of each view angle at each view point have the second preset size.

[0048] In a possible implementation, the processing module is specifically configured to construct the second index relationship according to the coordinate positions of the center points of the images of each view angle in the corresponding panoramic images and the features of the images of each view angle.

[0049] In a possible implementation, the processing module is specifically configured to cut the panoramic images at each view point to obtain the image blocks corresponding to the panoramic images at each view point, and construct the first index relationship according to the coordinate positions of the center points of the images of each view angle in the corresponding panoramic images and the image blocks corresponding to the panoramic images at each view point.

[0050] In a possible implementation, the image blocks corresponding to the panoramic images at each view point have a first preset size.

[0051] In a possible implementation, the processing module is further configured to use panoramic image stitching technology to obtain the panoramic images at each view point according to the pre-acquired images of multiple view angles at each view point, where the overlap rate between the pre-acquired images of adjacent view angles at each view point is less than the preset overlap rate.

[0052] In a possible implementation, the processing module is specifically configured to project the pre - acquired images of each perspective under each viewpoint in the first view plane onto the second view plane to which the panoramic image belongs, so as to obtain the panoramic images under each viewpoint and the transformation relationship between the first view plane and the second view plane.

[0053] In a possible implementation, the processing module is specifically configured to project the image block corresponding to the target identifier onto the first view plane by using inverse projection transformation according to the transformation relationship, so as to obtain the second image.

[0054] In a possible implementation, the transceiver module is configured to receive the first image from the terminal and the position of the terminal when the terminal captures the first image, and send the second image to the terminal.

[0055] Fourthly, an image processing apparatus is provided in an embodiment of the present application. The image processing apparatus may be a terminal or a chip in the terminal. The image processing apparatus includes:

[0056] In a possible implementation, the terminal stores the image blocks corresponding to the panoramic images under each viewpoint, and the mapping relationship between the features of the images of each perspective under each viewpoint and the identifiers of the image blocks corresponding to the panoramic images under each viewpoint. The images of each perspective under each viewpoint and the image blocks corresponding to the panoramic images under each viewpoint are all obtained based on the panoramic images under each viewpoint.

[0057] The shooting module is configured to shoot a first image using a first magnification, and the first magnification is less than a preset magnification.

[0058] The processing module is configured to obtain the similarity between the features of the images of each perspective under each viewpoint and the features of the first image, and determine the target identifier mapped by the features of the image of the perspective corresponding to the maximum similarity according to the features of the image of the perspective corresponding to the maximum similarity and the mapping relationship, and obtain a second image according to the image block corresponding to the target identifier. The clarity of the second image is higher than that of the first image.

[0059] The display module is configured to display the second image in response to an image display instruction.

[0060] In a possible implementation, the processing module is specifically configured to determine the target viewpoints within a preset range from the position of the terminal, and obtain the similarity between the features of the images of each perspective under the target viewpoints and the features of the first image.

[0061] In a possible implementation, the mapping relationship includes a first indexing relationship and a second indexing relationship. The second indexing relationship is the mapping relationship between the features of the images of each perspective under each viewpoint and the center points of the images of each perspective under each viewpoint. The first indexing relationship is the mapping relationship between the center points of the images of each perspective under each viewpoint and the identifiers of the image blocks corresponding to the panoramic images under each viewpoint.

[0062] The processing module is specifically configured to determine the center point mapped by the features of the image of the perspective corresponding to the maximum similarity according to the features of the image of the perspective corresponding to the maximum similarity and the second indexing relationship, and determine the target identifier according to the center point mapped by the features of the image of the perspective corresponding to the maximum similarity and the first indexing relationship.

[0063] In a possible implementation, the processing module is further configured to obtain the features of the images of each perspective under each viewpoint, the first indexing relationship, and the second indexing relationship according to the panoramic images under each viewpoint.

[0064] The storage module is used to store the features of the images of each perspective under each viewpoint, the first indexing relationship, and the second indexing relationship.

[0065] In a possible implementation, the processing module is specifically configured to perform back-projection transformation on the panoramic images under each viewpoint to obtain the images of multiple perspectives under each viewpoint and the coordinate positions of the center points of the images of each perspective under each viewpoint in the corresponding panoramic images. The overlap rate between adjacent perspective images under each viewpoint is greater than a preset overlap rate; and extract the features of the images of each perspective under each viewpoint.

[0066] In a possible implementation, the processing module is specifically configured to slide a sliding window with a second preset size in the panoramic images under each viewpoint and perform back-projection transformation to sequentially obtain the images of the perspectives corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images of the perspectives corresponding to the partial panoramic images in the corresponding panoramic images. Each perspective image under each viewpoint has the second preset size.

[0067] In a possible implementation, the processing module is specifically configured to construct the second indexing relationship according to the coordinate positions of the center points of the images of each perspective under each viewpoint in the corresponding panoramic images and the features of the images of each perspective under each viewpoint.

[0068] In a possible implementation, the processing module is specifically configured to cut the panoramic images at each viewpoint to obtain the image blocks corresponding to the panoramic images at each viewpoint, and construct the first index relationship according to the coordinate positions of the center points of the images at each angle in the corresponding panoramic images at each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint.

[0069] In a possible implementation, the image blocks corresponding to the panoramic images at each viewpoint have a first preset size.

[0070] In a possible implementation, the processing module is further configured to use panoramic image stitching technology to obtain the panoramic images at each viewpoint according to the pre-acquired images at multiple angles at each viewpoint, and the overlap rate between the pre-acquired images at adjacent angles at each viewpoint is less than the preset overlap rate.

[0071] In a possible implementation, the processing module is specifically configured to project the pre-acquired images at each angle at each viewpoint in the first viewing plane onto the second viewing plane to which the panoramic image belongs, so as to obtain the panoramic images at each viewpoint and the transformation relationship between the first viewing plane and the second viewing plane.

[0072] In a possible implementation, the processing module is specifically configured to project the image block corresponding to the target identifier onto the first viewing plane by using inverse projection transformation according to the transformation relationship to obtain the second image.

[0073] In a fifth aspect, an embodiment of the present application provides an electronic device, which may be the above-mentioned cloud or terminal. The electronic device may include: a processor and a memory. The memory is used to store computer-executable program code, and the program code includes instructions; when the processor executes the instructions, the instructions cause the electronic device to execute the methods in the first aspect and the second aspect.

[0074] In a sixth aspect, an embodiment of the present application provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the methods in the first aspect and the second aspect.

[0075] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored, and when running on a computer, cause the computer to execute the methods in the first aspect and the second aspect.

[0076] For the possible implementation manners of the second aspect to the seventh aspect above, the beneficial effects can refer to the beneficial effects brought by the first aspect above, and will not be elaborated here. Description of the Drawings

[0077] Figure 1A schematic diagram of a scenario applicable to the embodiments of the present application;

[0078] Figure 2 A schematic diagram of high-definition images at different shooting angles at a shooting position in cloud storage in the prior art;

[0079] Figure 3 A schematic diagram of an image processing method in the prior art;

[0080] Figure 4 A schematic diagram of the relationship between image blocks and indexes stored in the cloud provided by the embodiments of the present application;

[0081] Figure 5 Another schematic diagram of the relationship between image blocks and indexes stored in the cloud provided by the embodiments of the present application;

[0082] Figure 6 A schematic diagram of obtaining images at different perspectives from a viewpoint in the cloud provided by the embodiments of the present application;

[0083] Figure 7 A flowchart of an embodiment of an image processing method provided by the embodiments of the present application;

[0084] Figure 8 A schematic diagram of a change in the shooting interface provided by the embodiments of the present application;

[0085] Figure 9 A flowchart of an embodiment of an image processing method provided by the embodiments of the present application;

[0086] Figure 10 A schematic diagram of a panoramic image stored in the cloud provided by the embodiments of the present application;

[0087] Figure 11 A flowchart of another embodiment of an image processing method provided by the embodiments of the present application;

[0088] Figure 12 A schematic diagram of the structure of an image processing device provided by the embodiments of the present application;

[0089] Figure 13 A schematic diagram of the structure of an electronic device provided by the embodiments of the present application. Detailed implementation manners

[0090] Term interpretations involved in the embodiments of the present application:

[0091] Single viewpoint: The viewpoint can be understood as the position of the shooting device (such as a mobile phone) when taking a photo, and the single viewpoint is a single position.

[0092] Panoramic Image Stitching Technology: It is to stitch multiple images into a large-scale image. Exemplarily, in the embodiments of the present application, multiple high-definition images are stitched into a panoramic image. Here, the principle of panoramic image stitching technology is briefly described. Panoramic image stitching may include but is not limited to 4 steps, and the 4 steps are respectively: detecting and extracting the features and key points of the image, matching the key points of two images, estimating the homography matrix using the random sample consensus (RANSAC) algorithm, and stitching the images.

[0093] In one embodiment, the specific implementation of panoramic image stitching technology may include: using the scale-invariant feature transform (SIFT) local descriptor to detect the key points and features (feature descriptors or SIFT features) in the image, and matching the feature descriptors between two images, that is, using the features to match the key points of two images. Then, the RANSAC algorithm is used to estimate the homography matrix (homography estimation) using the key points matched on two images, that is, to match one image with another image through association.

[0094] After estimating the homography matrix, perspective transformation can be adopted. For example, the homography matrix, the image to be distorted, and the shape of the output image can be input, and then the derived shape of the output image can be determined by obtaining the sum of the widths of the two images and then using the height of the image. For specific details, reference can be made to the relevant descriptions in the prior art of perspective transformation. Perspective transformation can be understood as: projecting the image onto a new viewing plane. Perspective transformation is also called projective mapping or projective transformation.

[0095] Projective Transformation: Reference can be made to the description of perspective transformation.

[0096] Back-projection Transformation: Projective transformation refers to the process of projecting an image onto a new viewing plane, and back-projection transformation refers to projecting the image on the new viewing plane back onto the original viewing plane of the image. It should be understood that during the projective transformation process, the transformation relationship (such as the transformation matrix) between the original viewing plane and the new viewing plane can be obtained. The back-projection transformation process is to use this "transformation relationship between the original viewing plane and the new viewing plane" to project the image on the new viewing plane back onto the original viewing plane of the image.

[0097] In one embodiment, back-projection transformation can be called back-projection mapping, inverse perspective transformation, or reverse perspective transformation. For specific details of back-projection transformation, reference can be made to the relevant descriptions in the prior art of back-projection transformation.

[0098] High-definition image: An image with high definition taken by a single-lens reflex camera, and the clarity of the high-definition image is greater than the preset clarity. In one embodiment, if the resolutions of images are the same, the higher the bit rate, the higher the clarity. In this scenario, the preset bit rate can be used to represent the preset clarity. In the embodiments of the present application, the parameters for representing clarity are not limited.

[0099] Photographing magnification: Refers to the zoom magnification.

[0100] High magnification: The zoom magnification used during photographing is greater than the preset magnification. The preset magnification depends on the photographing ability of the terminal, and the preset magnifications of different terminals can be the same or different. In one embodiment, for example, the preset magnification can be 5.

[0101] Figure 1 This is a schematic diagram of a scenario applicable to the embodiments of the present application. Figure 1 Taking the terminal as a mobile phone and a single-lens reflex camera for comparison and explanation, taking the example that both the mobile phone and the single-lens reflex camera photograph the computer screen. Refer to Figure 1 In a, when the user uses a single-lens reflex camera to take a photo with a magnification of 30, a high-definition image can be obtained. For example, the user can clearly see the text "one, two, three, four" on the computer screen in the image. When the user uses a mobile phone to take a photo with a magnification of 30 (i.e., Figure 1 30x in b), the clarity of the photographed image is low, and the user cannot clearly see the text on the computer screen, but only sees a few shadow squares, as shown in Figure 1 b. It should be understood that for the convenience of explaining the images obtained by the mobile phone and the single-lens reflex camera, the images obtained by the mobile phone and the single-lens reflex camera with a magnification of 30 are respectively shown on the right side of a in Figure 1 and on the right side of b in Figure 1 respectively.

[0102] It should be understood that in the embodiments of the present application, when the user takes a photo with a high magnification, it can be understood that: the photographing magnification is greater than the preset magnification when the user takes a photo. The preset magnification can refer to the relevant descriptions in the above term explanations.

[0103] In order to improve the clarity of the image obtained by the terminal when taking a photo with a high magnification, in the prior art, a large number of high-definition images can be pre-stored in the cloud. The high-definition images include: high-definition images taken at different shooting positions, and high-definition images taken at different shooting angles at the same shooting position. Exemplarily, Figure 2 This is a schematic diagram of high-definition images taken at different shooting angles at position A stored in the cloud in the prior art. It should be understood that Figure 2The object being photographed is represented by a black rectangle, and an example of 6 high-definition images will be used for illustration. In one embodiment, the overlap rate of the frames of the high-definition images stored in the cloud in the prior art is greater than or equal to the first overlap rate, such as 80%. In one embodiment, the shooting position can be referred to as the viewpoint, and the shooting angle can be referred to as the viewing angle. In other words, in the prior art, the cloud stores high-definition images taken from different viewpoints, as well as high-definition images taken at different viewing angles from the same viewpoint.

[0104] Referring to Figure 3 , in the prior art, when the terminal takes a high-magnification photo, it can send the taken image to the cloud. The cloud obtains the similarity between each high-definition image stored in the cloud and the image from the terminal, and then feeds back the high-definition image with the maximum similarity to the terminal. After receiving the high-definition image from the cloud, the terminal can display the high-definition image, and the user can see the high-definition image obtained by the terminal taking a high-magnification photo. Exemplarily, Figure 3 in which the terminal sends to the cloud Figure 1 the low-definition image shown in b of , the cloud can feed back a high-definition image to the terminal, such as an image with the words "one, two, three, four" displayed. Although the method of the prior art can enable the terminal to obtain a high-definition image when taking a high-magnification photo, the cloud needs to store a large number of high-definition images, occupying a large amount of storage space, and the storage overhead of the cloud is large.

[0105] In one embodiment, the high-definition images stored in the cloud can be reduced, such as storing high-definition images with an overlap rate less than the second overlap rate, such as 20%. In this way, because the number of high-definition images stored in the cloud is small, such as the shooting angles corresponding to the images at the same shooting position are reduced. Thus, based on the method of comparing the similarity between each high-definition image stored in the cloud and the image from the terminal, the shooting angle of the high-definition image fed back to the terminal is different from that of the image actually taken by the terminal, resulting in the user seeing images taken at different shooting angles, and the feedback accuracy rate of the high-definition image is low, leading to a low user experience.

[0106] Based on the above problems, on the one hand, in the embodiments of the present application, panoramic images (or image blocks segmented from panoramic images) under different single viewpoints (or viewpoints) can be stored in the cloud. For a single viewpoint, the high-definition images stored in the cloud are changed from multiple high-definition images to one panoramic image (or multiple image blocks corresponding to one panoramic image), which can reduce the storage overhead of the cloud. On the other hand, in order to ensure the accurate high-definition image fed back to the terminal, it is also necessary to store high-definition images at different shooting angles under the same viewpoint. In the embodiments of the present application, on the basis of reducing the storage overhead of the cloud, the features of high-definition images at different viewpoints and different shooting angles under the same viewpoint can be stored, which can reduce the storage overhead of the cloud while ensuring the feedback accuracy.

[0107] In one embodiment, the terminal in the embodiments of the present application may be referred to as a user equipment. The terminal has a photographing function and supports high-magnification photographing. When the terminal in the embodiments of the present application performs photographing at a high magnification (such as the photographing magnification is greater than a preset magnification), the clarity of the captured image is low. Exemplarily, the terminal may be a mobile phone, a tablet computer (portable android device, PAD), a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device, or a wearable device, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a terminal in a smart home, etc. The embodiments of the present application do not make specific limitations on the form of the terminal.

[0108] In one embodiment, the cloud may be a server or a server cluster. Exemplarily, the server may be, for example, a server corresponding to a photographing application program, or a server corresponding to an application program with a photographing function. The embodiments of the present application do not make specific limitations on the form of the cloud.

[0109] Before introducing the image processing method provided by the embodiments of the present application, the content stored in the cloud will be described first:

[0110] In one embodiment, the cloud stores a plurality of image blocks corresponding to panoramic images at different viewpoints, wherein there is no overlap or the overlap rate is less than a third overlap rate between the plurality of image blocks corresponding to the panoramic image at the same viewpoint. Exemplarily, the third overlap rate may be a smaller value such as 20% or 10%. It should be understood that the panoramic image is a high-definition image, and the plurality of image blocks corresponding to the panoramic image are also high-definition image blocks.

[0111] In one embodiment, the cloud stores panoramic images at different viewpoints.

[0112] In the embodiments of the present application, for one viewpoint, since the cloud does not store high-definition images of multiple perspectives at this viewpoint, but stores one panoramic image at this viewpoint, or a plurality of image blocks corresponding to the panoramic image, the storage overhead of the cloud can be reduced. In the following embodiments, the image processing method provided by the embodiments of the present application will be introduced by taking one viewpoint as an example.

[0113] In one embodiment, the panoramic images at different viewpoints in the embodiments of the present application may be obtained by photographing with a panoramic camera, or by stitching low-overlap high-definition images at different viewpoints. The overlap rate between the low-overlap high-definition images is less than a second overlap rate. The following will take stitching low-overlap high-definition images at different viewpoints to obtain panoramic images at different viewpoints as an example for explanation.

[0114] Refer toFigure 4 , the process of storing content in the cloud can include the following steps:

[0115] S401, the cloud uses panoramic image stitching technology to stitch low-overlap high-definition images at the same viewing point to obtain panoramic images at different viewing points.

[0116] In one embodiment, the low-overlap high-definition images at the same viewing point can be collected by a photographing device such as a single-lens reflex camera that can capture high-definition images. Exemplarily, high-definition images at different perspectives can be pre-captured by a single-lens reflex camera at the same viewing point, and then high-definition images at different perspectives at different viewing points can be obtained. In one embodiment, the low-overlap high-definition images at each viewing point can be referred to as pre-collected images at multiple perspectives at each viewing point.

[0117] Among them, the overlap rate between adjacent perspective high-definition images collected by the single-lens reflex camera at the same viewing point is less than the second overlap rate, or high-definition images can be selected from the high-definition images collected by the single-lens reflex camera at the same viewing point so that the overlap rate between adjacent perspective high-definition images is less than the second overlap rate to obtain low-overlap high-definition images at the same viewing point. Among them, the purpose of the overlap rate of the low-overlap high-definition images being less than the second overlap rate is to: reduce the computational amount of panoramic image stitching in the cloud and improve the stitching efficiency. In one embodiment, it can also be said that the overlap rate between adjacent perspective high-definition images is less than a preset overlap rate, and the preset overlap rate is greater than or equal to the second overlap rate and less than the first overlap rate.

[0118] For the low-overlap high-definition images at the same viewing point, the cloud can use panoramic image stitching technology to obtain the panoramic image at this viewing point. According to the panoramic image stitching technology, the cloud can obtain panoramic images at different viewing points. The panoramic image stitching technology can specifically refer to the relevant description in the term interpretation.

[0119] Among them, S401 is like Figure 5 S1 in Figure 5 is Figure 4 a simplified process schematic diagram.

[0120] Figure 6 is a schematic diagram of the cloud obtaining images at different perspectives at the same viewing point provided by the embodiment of the present application. Referring to Figure 6 a in Figure 6 as an example where the low-overlap high-definition images at one viewing point include 2 images, when the cloud executes S401, the panoramic image at this viewing point can be obtained, as shown in b in

[0121]

[0122] S402, the cloud cuts the panoramic images at each viewing point to obtain image blocks corresponding to the panoramic images at each viewing point.

[0122] Taking one viewpoint as an example, the cloud can cut the panoramic image under this viewpoint into image blocks with a preset size, obtaining multiple image blocks corresponding to this viewpoint. In one embodiment, the size of each image block is the same, such as 800px * 900px, that is, each image block has a first preset size, and the first preset size can be understood as having a first preset width and a first preset height. Among them, 1px represents one pixel. In one embodiment, the size of each image block can be different.

[0123] In one embodiment, there is no overlap between two adjacent image blocks corresponding to the same viewpoint, that is, they do not contain the same area. In one embodiment, the overlap rate between two adjacent image blocks corresponding to the same viewpoint can be less than the third overlap rate.

[0124] In one embodiment, after the cloud cuts the panoramic image into image blocks, it can number each image block. Exemplarily, it can be numbered according to the rows and columns of the cut image blocks in the panoramic image. For example, if an image block is located in the first row and the first column of the panoramic image, this image block can be numbered as row 1, column 1. Exemplarily, the image blocks can be numbered in the order from 1 to N. For example, the image block in the first row and the first column is numbered 1, and the image block in the first row and the second column is numbered 2, where N is an integer greater than 1. In the embodiments of the present application, there is no limitation on the way of numbering the image blocks. In the following embodiments, the example of numbering the image blocks by rows and columns is used for illustration. In one embodiment, the row and column numbers or the "1 - N" numbers of the image blocks can be called the identifiers of the image blocks.

[0125] In the embodiments of the present application, cutting the panoramic images under each viewpoint into image blocks is for the purpose of: facilitating the cloud to load the image blocks, rather than directly loading the entire panoramic image by the cloud, because the loading time of the image blocks is less than the loading time of the entire panoramic image. Therefore, the loading speed of the cloud can be improved, and the speed of the cloud to feedback high - definition images to the terminal can be improved. For details, reference can be made to Figure 7 the relevant descriptions in

[0126] Among them, S402 is like Figure 5 S2 in

[0127] Referring to Figure 6 , exemplarily, when the cloud executes S402, it can cut the panoramic image into 8 image blocks, as shown in Figure 6 c in

[0128] S403, the cloud uses inverse projection transformation to obtain images of multiple viewpoints corresponding to the panoramic image under each viewpoint.

[0129] In S401, during the process that the cloud uses panoramic image stitching technology to obtain panoramic images at different viewpoints, the transformation relationship from the first view plane where the low-overlap high-definition images at each viewpoint are located to the second view plane where the panoramic image is located can be obtained. Furthermore, the cloud can use the transformation relationships at each viewpoint to project each part of the panoramic image onto the first view plane to obtain images from different perspectives.

[0130] In one embodiment, the cloud can project the part of the panoramic image within the sliding window of the second preset size onto the second view plane in sequence according to the left-to-right and top-to-bottom order of the panoramic image, to obtain images from multiple perspectives. Among them, the image of each perspective is a high-definition image, and the sizes of the images of each perspective are the same, that is, the second preset size. For example, each image of each perspective has a second preset width and a second preset height. In one embodiment, the overlap rate between adjacent perspective images corresponding to the panoramic image at the same viewpoint is greater than the first overlap rate. That is to say, when the cloud controls the sliding window to slide each time, the overlap rate with the previous position of the sliding window can be kept greater than the first overlap rate to obtain adjacent perspective images corresponding to the panoramic image at each viewpoint.

[0131] In S403, during the inverse projection transformation process, when projecting the part of the panoramic image within the sliding window of the second preset size onto the second view plane, a one-to-one mapping relationship between each pixel point on the part of the panoramic image and the pixel points on the image of the corresponding perspective can be obtained. Furthermore, during this process, the cloud can obtain the coordinate position of the center point of the image of the corresponding perspective in the panoramic image. In one embodiment, the coordinate position of the center point in the panoramic image can be longitude and latitude coordinates.

[0132] It should be understood that the center point of the image can be understood as the physical center point of the image.

[0133] It should be understood that there is no distinction in the order between S402 and S403, and the two can be executed simultaneously.

[0134] Among them, S403 is like Figure 5 S3 in

[0135] Referring to Figure 6 , for example, when the cloud executes S403, the panoramic image can be subjected to inverse projection transformation to obtain 4 perspective images corresponding to this viewpoint, as shown in Figure 6 d in

[0136] S404, the cloud establishes the first index relationship between the center point of the image of each perspective and the image block according to the coordinate position of the center point of the image of each perspective at each viewpoint in the panoramic image.

[0137] The first index relationship can be understood as: which image blocks in the panoramic image correspond to the images of each perspective, that is, establishing the identification mapping relationship between the images of each perspective and the image blocks.

[0138] In one embodiment, since the images of each perspective have a second preset size, on the premise that the coordinate positions of the center points of the images of each perspective in the panoramic image are already known, all the image blocks corresponding to the images of each perspective can be obtained. Exemplarily, the cloud can determine the coordinate positions of the four vertices of the image of each perspective in the panoramic image based on the second preset size of the image of each perspective and the coordinate positions of the center points of the images of each perspective in the panoramic image. Furthermore, for the image of one perspective, the cloud can determine the image blocks corresponding to the image of this perspective in the panoramic image according to the four vertices of the image of each perspective and the position coordinates of the center point in the panoramic image.

[0139] In one embodiment, the cloud can store the first index relationship. In the first index relationship, the images of each perspective can be characterized by the coordinate positions of the center points of the images of each perspective in the panoramic image, and the image blocks can be characterized by the numbers of the image blocks. That is to say, the first index relationship can include: the mapping relationship between the coordinate positions of the center points of the images of each perspective in the panoramic image and the numbers of the image blocks.

[0140] Exemplarily, as shown in Figure 6 If the panoramic image corresponds to the images of 4 perspectives, the coordinate positions of the center points of the images of these 4 perspectives in the panoramic image are (longitude 1, latitude 1), (longitude 2, latitude 2), (longitude 3, latitude 3), and (longitude 4, latitude 4) respectively. Among them, the image blocks corresponding to the image of (longitude 1, latitude 1) are: (row 1, column 1), (row 1, column 2), the image blocks corresponding to the image of (longitude 2, latitude 2) are: (row 1, column 3), (row 1, column 4), the image blocks corresponding to the image of (longitude 3, latitude 3) are: (row 2, column 1), (row 2, column 2), and the image blocks corresponding to the image of (longitude 4, latitude 4) are: (row 2, column 3), (row 2, column 4). Accordingly, the first index relationship stored by the cloud can be as shown in Table 1:

[0141] Table 1

[0142]

[0143]

[0144] Among them, S404 is like Figure 5 S4 in

[0145] S405, the cloud obtains the features of the images of each perspective at each viewpoint to establish a second index relationship between the features of the images of each perspective and the coordinate positions of the center points of the images of each perspective in the panoramic image.

[0146] For each image of each perspective under each viewpoint, the cloud can obtain the features of the images of each perspective. In one embodiment, the features of the images of each perspective are embodied as feature vectors. For example, the feature vector can be a 2048-dimensional feature vector. That is to say, the cloud can obtain the feature vectors of the images of each perspective under each viewpoint.

[0147] In one embodiment, the cloud can adopt a neural network model to extract the features of the images of each perspective. Exemplarily, the neural network model can include, but is not limited to: convolutional neural networks (CNN), recurrent neural network (RNN), and long short-term memory (LSTM).

[0148] In the embodiments of the present application, the cloud can obtain the features of the images of each perspective, and then can establish a second index relationship between the features of the images of each perspective and the coordinate positions of the center points of each perspective in the panoramic image. In one embodiment, the cloud can store the second index relationship. In the second index relationship, the coordinate position of the center point of the image of each perspective in the panoramic image can be used to represent the center point of the image of each perspective, and the feature vector of the image of each perspective can be used to represent the features of the image of each perspective. That is to say, the second index relationship can include: the mapping relationship between the coordinate position of the center point of each perspective in the panoramic image and the features of the image of each perspective.

[0149] In one embodiment, the cloud can also obtain a third index relationship according to the first index relationship and the second index relationship. For example, the third index relationship is: the mapping relationship between the features of the images of each perspective and the numbers of the image blocks. That is to say, the cloud can merge the first index relationship and the second index relationship based on the coordinate position of the center point of each perspective in the panoramic image, and map the features of the images of the center points with the same coordinate position and the numbers of the image blocks to obtain the third index relationship.

[0150] It should be understood that there is no distinction in the order between S404 and S405, and the two can be executed simultaneously.

[0151] Among them, S405 is as Figure 5 S5 in

[0152] In summary, in one embodiment, multiple image blocks, a first index relationship, and a second index relationship under each viewpoint can be stored in the cloud. Alternatively, in one embodiment, multiple image blocks and a third index relationship under each viewpoint can be stored in the cloud. In this way, compared with the prior art in which high-definition images with different perspectives under each viewpoint are stored in the cloud, the storage overhead can be reduced.

[0153] Based on the relevant introduction of the content stored in the cloud, the image processing method provided in the embodiments of the present application will be described below in combination with specific embodiments. The following several embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. Figure 7 It is a schematic flowchart of an embodiment of the image processing method provided in the embodiments of the present application. It should be understood that Figure 7 it is described by taking the interaction between the terminal and the cloud as an example.

[0154] Referring to Figure 7 , the image processing method provided in the embodiments of the present application may include:

[0155] S701, the terminal takes a picture in response to a photographing instruction to obtain a first image.

[0156] In one embodiment, the photographing instruction may be an instruction triggered by a user operating a photographing interface displayed on the terminal. Exemplarily, for example, the photographing interface includes a photographing control, and the user operating the photographing control can trigger an input of a photographing instruction to the terminal. In one embodiment, the photographing instruction may be triggered by the user's voice. For example, when the user says "take a picture", it can trigger an input of a photographing instruction to the terminal. In one embodiment, the user can also trigger an input of a photographing instruction to the terminal in a custom manner or by operating other shortcut keys. The embodiments of the present application do not limit the manner in which the user triggers the photographing instruction.

[0157] The terminal takes a picture in response to the photographing instruction to obtain a first image.

[0158] Figure 8 It is a schematic diagram of a variation of the photographing interface provided in the embodiments of the present application. Figure 8 As shown in a of [], the photographing interface is shown. The photographing interface includes a preview frame 81, a photographing control 82, and a magnification adjustment bar 83. The user adjusts the magnification adjustment bar 83 to change the photographing magnification of the terminal. Exemplarily, taking the user adjusting the photographing magnification to 30 as an example for description. The user clicks the photographing control 82, and correspondingly, the terminal takes a picture in response to the photographing instruction and can obtain a first image with a photographing magnification of 30.

[0159] S702, the terminal sends the first image to the cloud.

[0160] S703, the cloud obtains the features of the first image.

[0161] The method for the cloud to obtain the features of the first image can refer to the relevant description of the cloud obtaining the features of the images of each perspective under each viewpoint in S405.

[0162] S704. The cloud obtains the similarity between the features of the images of each perspective under each viewpoint and the features of the first image.

[0163] In one embodiment, the cloud can obtain the cosine angle or Euclidean distance between the features of the images of each perspective under each viewpoint and the features of the first image, etc., to obtain the similarity between the features of the images of each perspective under each viewpoint and the features of the first image. Among them, the smaller the cosine angle and the smaller the Euclidean distance, the greater the similarity.

[0164] In one embodiment, in order to reduce the amount of similarity calculation on the cloud side, when the terminal sends the first image to the cloud, it can upload the location of the terminal. That is, the above S702 can be replaced with: The terminal sends the first image and the location of the terminal to the cloud. Correspondingly, S704 can be replaced with: The cloud obtains the similarity between the features of the images of each perspective under the viewpoints within a preset range from the location of the terminal and the features of the first image. In one embodiment, the viewpoints within a preset range from the location of the terminal can be referred to as target viewpoints.

[0165] In this embodiment, since the features of the images of each perspective under different viewpoints are stored in the cloud, the cloud can first determine the viewpoints (i.e., target viewpoints) within a preset distance range from the location of the terminal based on the location of the terminal, and then obtain the similarity between the features of the images of each perspective under the (i.e., target viewpoints) and the features of the first image, which can avoid calculating the feature similarities under all viewpoints and improve the computing efficiency of the cloud.

[0166] In one embodiment, regardless of the shooting magnification used by the terminal, the terminal can execute S702 - S708. In one embodiment, when the terminal sends the first image and the location of the terminal to the cloud, it can send the first magnification. Correspondingly, when the cloud responds that the first magnification is greater than or equal to the preset magnification, it executes S703 - S708. When the cloud responds that the first magnification is less than the preset magnification, since the terminal itself can obtain a high-definition first image, the cloud can not execute S703 - S708 to save the computing resources of the cloud.

[0167] In one embodiment, since the terminal can obtain a high-definition image at a low magnification, there is no need for the terminal to interact with the cloud to obtain a high-definition image at this time. Therefore, in the scenario where the terminal takes a photo at a high magnification, the terminal can execute S702. In this embodiment, S701 can be replaced with: in response to a photo-taking instruction, take a photo at a first magnification to obtain a first image, where the first magnification is greater than a preset magnification. Correspondingly, S702 can be replaced with: the terminal sends the first image and the position of the terminal to the cloud in response to the first magnification being greater than the preset magnification.

[0168] S705. The cloud determines the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image according to the feature corresponding to the maximum similarity and the second index relationship.

[0169] The second index relationship is: the mapping relationship between the coordinate position of the center point of the image of each view in the panoramic image and the feature of the image. After the cloud obtains the similarity between the features of the images of each view at each viewpoint and the features of the first image (or the similarity between the features of the images of each view at the viewpoints within a preset distance range from the position of the terminal) and the first image, it can determine the maximum similarity, and then determine the feature corresponding to the maximum similarity.

[0170] In this way, the cloud can obtain the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image according to the stored second index relationship and the feature corresponding to the maximum similarity.

[0171] S706. The cloud determines the identifier of the image block mapped by the center point according to the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image and the first index relationship.

[0172] The first index relationship is: the mapping relationship between the coordinate position of the center point of the image of each view in the panoramic image and the image block. Therefore, after the cloud obtains the position coordinates of the center point corresponding to the feature with the maximum similarity in the panoramic image, it can obtain the identifier of the image block mapped by the position coordinates of this center point in the panoramic image according to the position coordinates of this center point in the panoramic image and the first index relationship, that is, the identifier of the image block mapped by the feature corresponding to the maximum similarity. In one embodiment, the identifier of the image block mapped by the feature corresponding to the maximum similarity can be referred to as the target identifier.

[0173] Exemplarily, if the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image are (longitude 1, latitude 1), then based on Table 1 (the first index relationship), the numbers of the image blocks mapped by this (longitude 1, latitude 1) can be obtained as (row 1, column 1), (row 1, column 2).

[0174] In one embodiment, if a third index relationship is stored in the cloud, the third index relationship is: the mapping relationship between the features of the image and the numbers of the image blocks. In this embodiment, after the cloud obtains the features corresponding to the maximum similarity, it can, according to the third index relationship, obtain the identifier of the image block mapped by the features corresponding to the maximum similarity. Correspondingly, in this embodiment, S705 and S706 can be replaced with: the cloud determines the identifier of the image block mapped by the features corresponding to the maximum similarity according to the features corresponding to the maximum similarity and the third index relationship.

[0175] S707. The cloud uses inverse projection transformation to obtain a second image according to the image block mapped by the center point.

[0176] Among them, the image block mapped by the center point is the image block corresponding to the target identifier. Multiple image blocks corresponding to the panoramic images at each view point are stored in the cloud. After the cloud determines the target identifier of the image block mapped by the center point, it can splice the image blocks corresponding to the target identifier, and then use inverse projection transformation to obtain a second image. Among them, the clarity of the second image is higher than that of the first image. In one embodiment, the clarity of the second image is greater than a preset clarity.

[0177] Among them, the cloud can map the image block corresponding to the center point after splicing to the first view plane according to the transformation relationship between the first view plane and the second view plane to obtain a second image, which is the image of the view plane photographed by the terminal.

[0178] In one embodiment, the cloud can splice the image blocks mapped by the center point according to the numbers of the image blocks mapped by the center point. Exemplarily, if the numbers of the image blocks mapped by (longitude 1, latitude 1) are (row 1, column 1) and (row 1, column 2), the cloud can splice the image blocks numbered (row 1, column 1) and (row 1, column 2) in the row-column order to obtain a second image.

[0179] In one embodiment, when there is an overlapping area between the image blocks, the cloud can cover the overlapping area in the image block of (row 1, column 1) with the overlapping area in the image block of (row 1, column 2) to splice the image blocks numbered (row 1, column 1) and (row 1, column 2). Among them, the cloud can determine the overlapping area between the image block of (row 1, column 1) and the image block of (row 1, column 2) according to the similarity of the pixels in the image block of (row 1, column 1) and the image block of (row 1, column 2), such as taking the area with a similarity of 100% as the overlapping area.

[0180] S708. The cloud sends the second image to the terminal.

[0181] Correspondingly, the terminal receives the second image from the cloud.

[0182] S709. The terminal displays the second image in response to the image display instruction.

[0183] After the terminal receives the second image from the cloud, it can display the second image based on the user's operation. Alternatively, after the terminal receives the second image from the cloud, it can display the second image.

[0184] In one embodiment, the image display instruction can be an instruction triggered by the user operating the camera interface. For example, the camera interface includes an image display control, and the user operating this image display control can trigger an input of an image display instruction to the terminal. In one embodiment, the image display instruction can also be triggered by the user's voice. The embodiments of the present application do not limit the manner in which the terminal receives the photographing instruction.

[0185] Exemplarily, referring to Figure 8 a in, after the user clicks the photographing control 82, the terminal and the cloud interact to execute S701 - S708. After the terminal receives the second image from the cloud, it can store the second image in the local image database (such as the photo album). As Figure 8 shown in b in, the camera interface includes an image display control 84. When the user clicks the image display control 84, the terminal can display the second image with high definition, as Figure 8 shown in c in. Different from b in the above Figure 1 when the terminal takes a photo using the first magnification (high magnification), the clarity of the taken image is high. For example, the user can clearly see the text on the computer screen in the second image.

[0186] In one embodiment, Figure 7 the steps S701 - S709 shown in can be simplified to Figure 9 shown in.

[0187] In the embodiments of the present application, when the terminal takes a photo using high magnification, it can send the captured first image to the cloud. The cloud determines the image corresponding to the maximum similarity based on the characteristics of the first image and the similarity of the images at multiple viewpoints stored, and then obtains the image block corresponding to the first image based on the first index relationship and the second index relationship. Furthermore, by splicing the image blocks and performing back-projection transformation, a second image with high definition can be obtained. In this way, the terminal can display the second image with high definition, achieving the purpose that the terminal can obtain an image with high definition when taking a photo using high magnification. On the other hand, because what is stored in the cloud is multiple image blocks, the first index relationship, and the second index relationship at each viewpoint, or the cloud stores multiple image blocks and the third index relationship at each viewpoint. Thus, compared with the existing method where the cloud stores high-definition images at different viewpoints, the storage overhead of the cloud can be reduced.

[0188] Figure 7In the embodiment shown, the cloud stores image blocks corresponding to panoramic images under each viewpoint, as well as a first index relationship and a second index relationship, or the cloud stores image blocks corresponding to panoramic images under each viewpoint, as well as a third index relationship. In one embodiment, the cloud may store panoramic images under each viewpoint. Exemplarily, the panoramic images stored in the cloud may be as shown in Figure 10 It should be understood that Figure 10 in Figure 10 , different shapes (such as black rectangles, black triangles, etc.) included in the panoramic image are used to represent panoramic images under different viewpoints.

[0189] In this embodiment, according to the description in S701 - S706 above, the cloud can determine the image block corresponding to the first image (i.e., the number of the image block of the feature map corresponding to the maximum similarity). If the cloud stores panoramic images under each viewpoint, the cloud can cut out the image block corresponding to the number from the panoramic image under this viewpoint according to the number of the image block corresponding to the first image, and perform back-projection transformation to obtain the second image. For example, the cloud can first load the panoramic image under this viewpoint, and then cut out the image block corresponding to the number of the image block corresponding to the first image in the panoramic image, and perform projection transformation to obtain the second image.

[0190] Exemplarily, if the numbers of the image blocks corresponding to the first image are (row 1, column 1), (row 1, column 2), the cloud can cut out the image blocks numbered (row 1, column 1) and (row 1, column 2) from the panoramic image according to the image block number and the first preset size of the image block, and perform projection transformation to obtain the second image.

[0191] Compared with the embodiment shown in Figure 7 above, Figure 7 in the embodiment shown in Figure 7 , since the cloud stores image blocks corresponding to the panoramic images, when the cloud determines the number of the image block corresponding to the first image, it can directly load the corresponding image blocks for splicing, back-projection transformation, etc. In the embodiment of the present application, since the cloud stores panoramic images, after the cloud determines the number of the image block corresponding to the first image, it needs to first load the panoramic image to which the number of the image block belongs, and then cut out the corresponding image block in the panoramic image. And the speed of the cloud loading image blocks is much faster than the speed of loading the entire panoramic image. Therefore Figure 7 in the embodiment shown in Figure 7 , the loading efficiency of the cloud is high, and it can feedback the second image to the terminal faster.

[0192] In the image processing method provided by the embodiments of the present application, panoramic images at each viewpoint can be stored in the cloud. After obtaining the number of the image block corresponding to the first image in the cloud, the corresponding image block can be cut in the panoramic image to which the number belongs, and the second image is obtained through projective transformation. The image processing method provided by the embodiments of the present application can also achieve the purpose of obtaining high-definition images when the terminal uses high magnification photography. Compared with Figure 7 the method of loading image blocks in the cloud and performing inverse projective transformation in the embodiment shown, in the embodiments of the present application, because the cloud also needs to load the entire panoramic image and then cut the image blocks in the panoramic image, the loading time is long and the loading efficiency is low, and the efficiency of feeding back the second image to the terminal is relatively low.

[0193] In one embodiment, multiple image blocks, a first index relationship, and a second index relationship at each viewpoint can be stored in the terminal, or multiple image blocks and a third index relationship at each viewpoint are stored in the terminal, or panoramic images at each viewpoint can be stored in the terminal. When the terminal uses high magnification photography to obtain the first image, the terminal can execute S703 - S707 to obtain a high-definition second image, and then the terminal can display the second image in response to an image display instruction.

[0194] In the above embodiments, taking the interaction between the cloud and the terminal as an example, the scenarios where the cloud can process images from the terminal and the scenarios where the terminal can process the captured images are described. As shown above, for an electronic device (the electronic device can be the cloud, the terminal, or other devices with processing capabilities), referring to Figure 11 , the image processing method provided by the embodiments of the present application may further include:

[0195] S1101, obtaining a first image to be processed.

[0196] When the electronic device is the cloud, the way for the cloud to obtain the first image to be processed can be: after the terminal captures the first image and sends it to the cloud, which can refer to the relevant descriptions in S701 - S702. In one embodiment, the first image to be processed can also be uploaded to the cloud by the user, or the first image is an image stored locally in the cloud.

[0197] When the electronic device is the terminal, the terminal can capture the first image, or the first image can be an image stored locally in the terminal.

[0198] When the electronic device is other devices with processing capabilities, the device can capture the first image, or the first image is uploaded to the device by the user, or the first image can be an image stored locally in the device, or the first image can be an image transmitted from other electronic devices.

[0199] In the embodiments of the present application, there is no limitation on the manner in which the electronic device acquires the first image to be processed.

[0200] S1102. Obtain the similarity between the features of the images at each viewing angle and the features of the first image.

[0201] For the electronic device to execute the step of S1102, reference may be made to the relevant descriptions in S703 - S704.

[0202] S1103. Determine the target identifier of the feature mapping of the image at the viewing angle corresponding to the maximum similarity according to the features of the image at the viewing angle corresponding to the maximum similarity and the mapping relationship.

[0203] In one embodiment, the mapping relationship may be a third index relationship. The third index relationship is: the mapping relationship between the features of the image and the numbers of the image blocks. In this embodiment, after the cloud obtains the features corresponding to the maximum similarity, it may, according to the third index relationship, obtain the identifier of the image block corresponding to the feature mapping of the maximum similarity.

[0204] In one embodiment, the mapping relationship may include a first index relationship and a second index relationship. The first index relationship is: the mapping relationship between the coordinate positions of the center points of the images at each viewing angle in the panoramic image and the image blocks, and the second index relationship is: the mapping relationship between the coordinate positions of the center points of the images at each viewing angle in the panoramic image and the features of the images. In this embodiment, after the electronic device obtains the similarity between the features of the images at each viewing angle and the features of the first image (or the similarity between the features of the images at each viewing angle and the features of the first image at the viewpoints within the preset distance range of the position of the terminal), it may determine the maximum similarity, and then determine the features corresponding to the maximum similarity. Further, the electronic device may, according to the stored second index relationship and the features corresponding to the maximum similarity, obtain the position coordinates of the center point of the feature mapping corresponding to the maximum similarity in the panoramic image, and then, according to the position coordinates of the center point in the panoramic image and the first index relationship, obtain the identifier of the image block mapped by the position coordinates of the center point in the panoramic image.

[0205] Wherein, the identifier of the image block mapped by the position coordinates of the center point in the panoramic image is the target identifier.

[0206] S1104. Obtain a second image according to the image block corresponding to the target identifier, and the clarity of the second image is greater than that of the first image.

[0207] In one embodiment, the electronic device may splice the image blocks corresponding to the target identifier to obtain the second image.

[0208] Or, in one embodiment, the electronic device may process the image blocks corresponding to the target identifier in the manner of S707 to obtain the second image.

[0209] After the electronic device obtains the second image, since the second image is obtained based on the image block corresponding to the viewpoint, the clarity of the second image is higher than that of the first image. Therefore, the electronic device can process the first image to obtain an image with higher clarity.

[0210] In one embodiment, after the electronic device obtains the second image, the electronic device may store the second image or transmit the second image to other electronic devices. The embodiments of the present application do not limit the post-processing of the second image. For the scenario of interaction between the cloud and the terminal, the cloud may send the second image to the terminal for display and storage.

[0211] In the embodiments of the present application, the electronic device stores the image blocks corresponding to the panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint and each angle, and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each angle and each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint. The panoramic images at each viewpoint are high-definition images. Compared with the prior art of storing high-definition images at different angles at each viewpoint, the storage cost can be reduced. On this basis, the electronic device can also process the first image with low clarity to obtain a second image with higher clarity.

[0212] Figure 12 It is a schematic structural diagram of an image processing apparatus provided by the embodiments of the present application. The image processing apparatus may be the cloud, the terminal, the electronic device in the above embodiments, or a chip in the cloud, or a chip in the terminal, or a chip in the electronic device, and is used to implement the image processing method provided by the embodiments of the present application. Among them, the electronic device stores the image blocks corresponding to the panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each angle and each viewpoint and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each angle and each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint.

[0213] Refer to Figure 12 , the image processing apparatus 1200 includes: a processing module 1201, a storage module 1202, and a transceiver module 1203.

[0214] The processing module 1201 is configured to obtain the first image to be processed, and obtain the similarity between the features of the images at each angle and each viewpoint and the features of the first image. According to the features of the image at the angle corresponding to the maximum similarity, and the mapping relationship, determine the target identifier mapped by the features of the image at the angle corresponding to the maximum similarity, and obtain a second image according to the image block corresponding to the target identifier. The clarity of the second image is higher than that of the first image.

[0215] In a possible implementation, the processing module 1201 is specifically configured to obtain the first image, the position where the first image is taken, determine target viewpoints within a preset range from this position, and obtain the similarity between the features of the images at each viewpoint and the features of the first image.

[0216] In a possible implementation, the mapping relationship includes a first indexing relationship and a second indexing relationship. The second indexing relationship is: the mapping relationship between the features of the images at each viewpoint and the center points of the images at each viewpoint. The first indexing relationship is: the mapping relationship between the center points of the images at each viewpoint and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint.

[0217] The processing module 1201 is specifically configured to determine the center point mapped by the features of the image at the viewpoint corresponding to the maximum similarity according to the features of the image at the viewpoint corresponding to the maximum similarity and the second indexing relationship, and determine the target identifier according to the center point mapped by the features of the image at the viewpoint corresponding to the maximum similarity and the first indexing relationship.

[0218] In a possible implementation, the processing module 1201 is further configured to obtain the features of the images at each viewpoint, the first indexing relationship, and the second indexing relationship according to the panoramic images at each viewpoint.

[0219] The storage module 1202 is configured to store the features of the images at each viewpoint, the first indexing relationship, and the second indexing relationship.

[0220] In a possible implementation, the processing module 1201 is specifically configured to perform back-projection transformation on the panoramic images at each viewpoint to obtain the images at multiple viewpoints at each viewpoint and the coordinate positions of the center points of the images at each viewpoint in the corresponding panoramic images. The overlap rate between adjacent viewpoint images at each viewpoint is greater than a preset overlap rate; extract the features of the images at each viewpoint.

[0221] In a possible implementation, the processing module 1201 is specifically configured to slide a sliding window with a second preset size in the panoramic images at each viewpoint, and perform back-projection transformation to sequentially obtain the images at the viewpoints corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images at the viewpoints corresponding to the partial panoramic images in the corresponding panoramic images. The images at each viewpoint have the second preset size.

[0222] In a possible implementation, the processing module 1201 is specifically configured to construct the second index relationship according to the coordinate positions of the center points of the images of each perspective under each viewpoint in the corresponding panoramic image, and the features of the images of each perspective under each viewpoint.

[0223] In a possible implementation, the processing module 1201 is specifically configured to cut the panoramic images under each viewpoint to obtain the image blocks corresponding to the panoramic images under each viewpoint, and construct the first index relationship according to the coordinate positions of the center points of the images of each perspective under each viewpoint in the corresponding panoramic image, and the image blocks corresponding to the panoramic images under each viewpoint.

[0224] In a possible implementation, the image blocks corresponding to the panoramic images under each viewpoint have a first preset size.

[0225] In a possible implementation, the processing module 1201 is further configured to use panoramic image stitching technology to obtain the panoramic images under each viewpoint according to the pre-acquired images of multiple perspectives under each viewpoint, and the overlap rate between the pre-acquired images of adjacent perspectives under each viewpoint is less than the preset overlap rate.

[0226] In a possible implementation, the processing module 1201 is specifically configured to project the pre-acquired images of each perspective under each viewpoint in the first visual plane onto the second visual plane to which the panoramic image belongs, so as to obtain the panoramic images under each viewpoint, and the transformation relationship between the first visual plane and the second visual plane.

[0227] In a possible implementation, the processing module 1201 is specifically configured to project the image block corresponding to the target identifier onto the first visual plane by using inverse projection transformation according to the transformation relationship, so as to obtain the second image.

[0228] In a possible implementation, the electronic device is a cloud, and the transceiver module 1203 is configured to receive the first image from the terminal and the position of the terminal when the terminal captures the first image, and send the second image to the terminal.

[0229] The image processing device provided by the embodiments of the present application is used to execute the image processing method in the above embodiments, and has the same implementation principle and technical effects as the above embodiments.

[0230] In one embodiment, the embodiments of the present application further provide an electronic device. Referring to Figure 13 , this electronic device may be the cloud, the terminal or Figure 11The electronic device described in [reference], the electronic device may include: a processor (such as a CPU) 1301 and a memory 1302. The memory 1302 may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk memory. Various instructions can be stored in the memory 1302 to complete various processing functions and implement the method steps of this application.

[0231] In one embodiment, the electronic device may include a screen 1303 for displaying the interface and images of the electronic device, etc.

[0232] Optionally, the electronic device involved in this application may further include: a power supply 1304, a communication bus 1305, and a communication port 1306. The communication port 1306 is used to enable the electronic device to connect and communicate with other peripherals. In the embodiment of this application, the memory 1302 is used to store computer-executable program code, and the program code includes instructions; when the processor executes the instructions, the instructions cause the processor of the electronic device to perform the actions in the above method embodiment, and its implementation principle and technical effects are similar and will not be elaborated here.

[0233] It should be noted that the modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more microprocessors (digital signal processors, DSPs), or, one or more field programmable gate arrays (FPGAs), etc. Again, when the above certain module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code, such as a controller. Again, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0234] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0235] The term "a plurality" in this document refers to two or more. The term "and / or" in this document is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after; in a formula, the character " / " represents a "division" relationship between the associated objects before and after. In addition, it should be understood that in the description of the present application, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0236] It can be understood that in the embodiments of the present application, the various numerical numbers involved are only for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0237] It can be understood that in the embodiments of the present application, the magnitude of the sequence numbers of the above processes does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Claims

1. An image processing method, characterized in that, Applied to an electronic device, where the electronic device stores image blocks corresponding to panoramic images at each viewpoint, as well as the mapping relationship between the features of the images at each viewpoint for each viewing angle and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each viewpoint for each viewing angle and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint. The method includes: Obtain a first image to be processed; Obtain the similarity between the features of the images at each viewpoint for each viewing angle and the features of the first image; According to the features of the image at the viewing angle corresponding to the maximum similarity and the mapping relationship, determine the target identifier mapped by the features of the image at the viewing angle corresponding to the maximum similarity; Obtain a second image according to the image block corresponding to the target identifier, and the clarity of the second image is higher than that of the first image.

2. The method according to claim 1, characterized in that The obtaining of the first image to be processed includes: Obtain the first image and the position where the first image is taken; The obtaining of the similarity between the features of the images at each viewpoint for each viewing angle and the features of the first image includes: Determine a target viewpoint within a preset range from the position; Obtain the similarity between the features of the images at each viewing angle of the target viewpoint and the features of the first image.

3. The method according to claim 1 or 2, characterized in that, The mapping relationship includes a first index relationship and a second index relationship. The second index relationship is: the mapping relationship between the features of the images at each viewpoint for each viewing angle and the center points of the images at each viewpoint for each viewing angle. The first index relationship is: the mapping relationship between the center points of the images at each viewpoint for each viewing angle and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint; The determining of the target identifier mapped by the features of the image at the viewing angle corresponding to the maximum similarity according to the features of the image at the viewing angle corresponding to the maximum similarity and the mapping relationship includes: According to the features of the image at the viewing angle corresponding to the maximum similarity and the second index relationship, determine the center point mapped by the features of the image at the viewing angle corresponding to the maximum similarity; According to the center point mapped by the features of the image at the viewing angle corresponding to the maximum similarity and the first index relationship, determine the target identifier.

4. The method according to claim 3, wherein Before obtaining the first image to be processed, it further includes: According to the panoramic images at each viewpoint, obtain the features of the images at each viewpoint for each viewing angle, the first index relationship, and the second index relationship; Store the features of the images at each viewpoint for each viewing angle, the first index relationship, and the second index relationship.

5. The method according to claim 4, wherein The obtaining of the features of the images at each viewpoint for each viewing angle according to the panoramic images at each viewpoint includes: Perform back-projection transformation on the panoramic images at each viewpoint to obtain the images at multiple viewing angles at each viewpoint and the coordinate positions of the center points of the images at each viewpoint for each viewing angle in the corresponding panoramic images. The overlap rate between adjacent images at each viewpoint is greater than a preset overlap rate; Extract the features of the images at each viewpoint for each viewing angle.

6. The method according to claim 5, wherein Performing inverse projection transformation on the panoramic images at each viewpoint to obtain images at multiple viewing angles at each viewpoint and the coordinate positions of the center points of the images at each viewing angle in the corresponding panoramic images, including: In the panoramic images at each viewpoint, a sliding window with a second preset size is slid in the panoramic image, and inverse projection transformation is used to sequentially obtain the images at the viewing angles corresponding to the partial panoramic images within the sliding window and the coordinate positions of the center points of the images at the viewing angles corresponding to the partial panoramic images in the corresponding panoramic images. The images at each viewing angle at each viewpoint have the second preset size.

7. The method according to claim 5 or 6, characterized in that Obtaining the second index relationship includes: Constructing the second index relationship according to the coordinate positions of the center points of the images at each viewing angle in the corresponding panoramic images and the features of the images at each viewing angle at each viewpoint.

8. The method according to claim 5 or 6, characterized in that, Obtaining the first index relationship includes: Cutting the panoramic images at each viewpoint to obtain the image blocks corresponding to the panoramic images at each viewpoint; Constructing the first index relationship according to the coordinate positions of the center points of the images at each viewing angle in the corresponding panoramic images and the image blocks corresponding to the panoramic images at each viewpoint.

9. The method according to claim 8, wherein The image blocks corresponding to the panoramic images at each viewpoint have a first preset size.

10. The method according to any one of claims 5, 6 and 9, characterized in that, Before obtaining the features of the images at each viewing angle, the first index relationship, and the second index relationship according to the panoramic images at each viewpoint, it further includes: Using panoramic image stitching technology to obtain the panoramic images at each viewpoint according to the pre-acquired images at multiple viewing angles at each viewpoint, where the overlap rate between the pre-acquired images at adjacent viewing angles at each viewpoint is less than the preset overlap rate.

11. The method according to claim 10, wherein The using panoramic image stitching technology to obtain the panoramic images at each viewpoint according to the pre-acquired images at multiple viewing angles at each viewpoint includes: Projecting the pre-acquired images at each viewing angle in the first viewing plane to the second viewing plane to which the panoramic image belongs to obtain the panoramic images at each viewpoint and the transformation relationship between the first viewing plane and the second viewing plane.

12. The method according to claim 11, wherein Obtaining the second image according to the image block corresponding to the target identifier includes: According to the transformation relationship, using inverse projection transformation to project the image block corresponding to the target identifier to the first viewing plane to obtain the second image.

13. The method according to claim 1, wherein When the electronic device is the cloud, obtaining the first image and the position where the first image is taken includes: Receiving the first image from the terminal and the position of the terminal when the terminal takes the first image; After obtaining the second image, it further includes: Sending the second image to the terminal.

14. An image processing apparatus, characterized in that, The electronic device stores the mapping relationship between the image blocks corresponding to the panoramic images at each viewpoint, the features of the images at each viewing angle at each viewpoint, and the identifiers of the image blocks corresponding to the panoramic images at each viewpoint. The images at each viewing angle at each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint are both obtained based on the panoramic images at each viewpoint. The device includes: A processing module for: Obtain a first image to be processed; Obtain the similarity between the features of the images of each perspective under each view point and the features of the first image; Determine the target identifier of the feature mapping of the image of the perspective corresponding to the maximum similarity according to the features of the image of the perspective corresponding to the maximum similarity and the mapping relationship; Obtain a second image according to the image block corresponding to the target identifier, and the clarity of the second image is higher than that of the first image.

15. An electronic device, characterized in that, Comprising: A processor and a memory; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, A computer program or instruction is stored in the computer-readable storage medium, and when the computer program or instruction is run, the method according to any one of claims 1-13 is implemented.

17. A computer program product, characterized in that, Comprising a computer program or instruction, and when the computer program or instruction is executed by a processor, the method according to any one of claims 1-13 is implemented.

Images