Photographing method, photographing device, electronic equipment and storage medium

Abstract

The invention relates to a photographing method, a photographing device, electronic equipment and a storage medium. The method comprises the steps that the terminal is controlled to be switched from a first camera to a second camera, a first image is acquired, and a second image is acquired; determining a first feature based on the first image, the first feature being an association feature of a first feature point in the first image and pixel points around the first feature point, and the first feature point being a pixel point used for determining a target mapping relationship in the first image; based on the first image, the second image and the first feature, a target mapping relation is determined, and the target mapping relation is used for representing a position projection relation between pixel points in the first image and pixel points in the second image; and determining a target image based on the first image and the target mapping relationship, and taking the target image as a photographing image corresponding to the second camera. The target mapping relation determined by the photographing method provided by the embodiment of the invention can be more accurate, and the influence of picture jump caused during the camera switching period can be reduced.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to photographing methods, photographing devices, electronic devices, and storage media. Background Technology

[0002] With the development of terminal technology, users' demand for mobile phone photography is gradually increasing.

[0003] In related technologies, users typically adjust the camera's focal length magnification to achieve the desired image quality. However, users often make significant adjustments to the focal length magnification. To address this need, terminals with multi-camera modules have been proposed.

[0004] However, when users take photos with a terminal equipped with multiple camera modules, the captured image may exhibit a jumpy display, thus affecting the user experience. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a photographing method, a photographing device, an electronic device, and a storage medium.

[0006] According to a first aspect of the present disclosure, a method for taking a picture is provided, comprising: controlling a terminal to switch from a first camera to a second camera; acquiring a first image captured by the first camera; and acquiring a second image captured by the second camera; determining a first feature based on the first image, wherein the first feature is a correlation feature between a first feature point in the first image and pixels surrounding the first feature point, and the first feature point is a pixel in the first image used to determine a target mapping relationship; determining a target mapping relationship based on the first image, the second image, and the first feature, wherein the target mapping relationship is used to characterize the positional projection relationship between pixels in the first image and pixels in the second image; determining a target image based on the first image and the target mapping relationship, and using the target image as the picture image corresponding to the second camera.

[0007] In one implementation, determining the target mapping relationship based on the first image, the second image, and the first feature includes: determining a second feature point and a second feature based on the first image and the second image, wherein the second feature point is the corresponding projected pixel point of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point; determining a matching point set between the first image and the second image based on the first feature point, the first feature, and the second feature point, wherein the matching point set includes at least one matching point pair, and each matching point pair includes the first feature point and the second feature point; and determining the target mapping relationship based on the matching point set.

[0008] In one implementation, determining the target mapping relationship based on the matching point set includes: determining a first mapping relationship based on a first subset of the matching point set, the first subset including at least one pair of matching points; calculating the position information of the first feature point in the second image based on the first mapping relationship based on a first feature point in the matching point set and the first mapping relationship; if the error between the position information and the position information corresponding to the second feature point meets an error threshold, then determining the first mapping relationship as the target mapping relationship.

[0009] In another embodiment, determining the target image based on the first image and the target mapping relationship includes: decomposing the target mapping relationship into a second mapping relationship and a third mapping relationship, wherein the second mapping relationship characterizes the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image, and the third mapping relationship characterizes the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image; determining the pixel translation amount and pixel rotation amount corresponding to the projection of the first image onto the second image based on the focal length magnification of the first camera; and determining the target image based on the first image, the pixel translation amount, and the pixel rotation amount.

[0010] In another embodiment, determining a first feature based on a first image includes: enlarging the first image and cropping the enlarged image to obtain a third image; performing grayscale registration on the third image based on the second image to obtain a fourth image; selecting the first feature point in the fourth image and determining the pixels surrounding the first feature point; and determining the first feature based on the first feature point and the pixels surrounding the first feature point.

[0011] According to a second aspect of the present disclosure, a photographing device is provided, comprising: a control unit for controlling a terminal to switch from a first camera to a second camera; an acquisition unit for acquiring a first image captured by the first camera and acquiring a second image captured by the second camera; a processing unit for determining a first feature based on the first image, wherein the first feature is an association feature between a first feature point in the first image and pixels surrounding the first feature point, and the first feature point is a pixel in the first image used to determine a target mapping relationship; the processing unit is further configured to determine a target mapping relationship based on the first image, the second image, and the first feature, wherein the target mapping relationship characterizes the positional projection relationship between pixels in the first image and pixels in the second image; the processing unit is further configured to determine a target image based on the first image and the target mapping relationship, and use the target image as the photographed image corresponding to the second camera.

[0012] In one embodiment, the processing unit determines the target mapping relationship based on a first image, a second image, and a first feature in the following manner: determining a second feature point and a second feature based on the first image and the second image, wherein the second feature point is the corresponding projected pixel point of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point; determining a matching point set between the first image and the second image based on the first feature point, the first feature, and the second feature point, wherein the matching point set includes at least one matching point pair, and each matching point pair includes the first feature point and the second feature point; and determining the target mapping relationship based on the matching point set.

[0013] In another embodiment, the processing unit determines the target mapping relationship based on the matching point set in the following manner: determining a first mapping relationship based on a first subset of the matching point set, the first subset including at least one pair of matching points; calculating the position information of the first feature point in the second image based on the first mapping relationship based on a first feature point in the matching point set and the first mapping relationship; and determining the first mapping relationship as the target mapping relationship if the error between the position information and the position information corresponding to the second feature point meets an error threshold.

[0014] In another embodiment, the processing unit determines the target image based on the first image and the target mapping relationship in the following manner: decomposing the target mapping relationship into a second mapping relationship and a third mapping relationship, wherein the second mapping relationship characterizes the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image, and the third mapping relationship characterizes the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image; determining the pixel translation amount and pixel rotation amount corresponding to the projection of the first image onto the second image based on the focal length magnification of the first camera; and determining the target image based on the first image, the pixel translation amount, and the pixel rotation amount.

[0015] In another embodiment, the processing unit determines the first feature based on the first image in the following manner: the first image is magnified and the magnified image is cropped at the center to obtain a third image; the third image is grayscale registered based on the second image to obtain a fourth image; the first feature point is selected in the fourth image and the pixels around the first feature point are determined; the first feature is determined based on the first feature point and the pixels around the first feature point.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to: execute the photographing method described in the first aspect or any embodiment of the first aspect.

[0017] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of a terminal, enable the terminal to perform the photographing method described in the first aspect or any embodiment of the first aspect.

[0018] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: determining a first feature based on a first image acquired by a first camera, wherein the first feature represents the association feature between a first feature point and surrounding pixels, and determining a target mapping relationship based on the first image, a second image, and the first feature. The target mapping relationship determined by the shooting method proposed in the embodiments of this disclosure can be more accurate, thereby reducing the impact of image jumps during camera switching and improving the user experience.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a schematic diagram illustrating the screen transition after a camera switch, according to an exemplary embodiment.

[0022] Figure 2 This is a flowchart illustrating a photographing method according to an exemplary embodiment.

[0023] Figure 3 This is a flowchart illustrating a method for determining a target mapping relationship according to an exemplary embodiment.

[0024] Figure 4 This is a flowchart illustrating a method for determining a target mapping relationship according to an exemplary embodiment.

[0025] Figure 5 This is a flowchart illustrating a method for determining a first feature according to an exemplary embodiment.

[0026] Figure 6 This is a flowchart illustrating a method for determining a target image according to an exemplary embodiment.

[0027] Figure 7 This is a block diagram of a photographing device according to an exemplary embodiment.

[0028] Figure 8 This is a block diagram illustrating an apparatus for taking a photograph according to an exemplary embodiment.

[0029] Figure 9 This is an apparatus for taking pictures, as shown in an exemplary embodiment. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0031] In related technologies, with the development of terminal technology, the ability of terminals to take pictures has been enhanced. Users have also gradually increased their usage needs while taking pictures. During the photo-taking process, in order to clearly capture subjects in different shooting positions, users usually adjust the focal length magnification. This includes, for example, using cameras with different focal length magnifications, or using a camera with a large focal length magnification adjustment range and adjusting the corresponding focal length magnification range. Understandably, the larger the focal length magnification adjustment range of a camera, the more space its hardware structure needs to occupy (e.g., the space required for adjusting the camera's focus).

[0032] Understandably, for some devices (including, for example, mobile phones or tablets), the limitations of their hardware structure make it difficult to achieve a wide range of focal length adjustments using a single camera, thus failing to meet users' photography needs. Therefore, devices typically employ multiple cameras of different types to provide users with a wider range of focal length adjustments, thereby satisfying their photography requirements.

[0033] Understandably, different types of cameras may have different effective focal length magnification ranges. For example, a camera used for wide-angle shooting might have an effective focal length magnification range of A to B, while a high-definition camera used for capturing high-resolution images might have an effective focal length magnification range of C to D (where A, B, C, and D represent focal length magnifications, and are all different). Users can switch between different cameras by adjusting the focal length magnification when taking photos with their device. For example, a user can first use a wide-angle camera to capture a larger shooting area, and then adjust the focal length magnification to use the high-definition camera to capture a specific target within that area.

[0034] However, during camera switching, different cameras have different focal lengths and are located in different physical positions on the terminal. This can cause a sudden change in the displayed image after the camera switch is complete.

[0035] Figure 1This is a schematic diagram illustrating the image transition after switching cameras, according to an exemplary embodiment. As shown in the figure, on the terminal display screen 101, the user uses a first camera (e.g., a wide-angle camera) to acquire an image within the dashed box 103 (i.e., the dashed image portion within the dashed box 103). To capture a clearer image within the dashed box 103, the user switches to a second camera (e.g., a high-definition camera) by adjusting the focal length to acquire an image within the dashed box 102 (i.e., the solid image portion within the dashed box 102). Since the first and second cameras are positioned in different physical locations on the terminal, the content presented on the screen will change after switching (for example, only a portion of the tree on the left side under the second camera can be captured under the first camera), and its display position on the terminal screen will also change significantly, which is the image transition described above.

[0036] It is understandable that during the process of taking photos with a mobile device, if there is a sudden (or noticeable) change in the image, it will negatively impact the user experience. Therefore, to eliminate this image change, related technologies have proposed using a mapping relationship (such as a projection matrix) during camera switching to mitigate the problem. That is, assuming the camera switching process is from a first camera to a second camera, after the first camera acquires its image, before switching to the second camera, this image is projected onto the second camera. The resulting projected image is then displayed to the user as the image acquired by the second camera, thereby reducing the impact of the image change.

[0037] In some embodiments, the mapping relationship is currently implemented using either method -1) or -2):

[0038] -1) Determine the mapping relationship based on the calibration parameters of the camera module during factory manufacturing.

[0039] -2) Based on traditional algorithms, the mapping relationship is determined based on the feature points of the captured image during the photo-taking process.

[0040] However, the drawback of -1) is that it relies too much on the hardware parameters of the camera module. If the hardware parameters of the camera module change (e.g., aging or damage), the mapping relationship set will not meet the actual shooting needs, and the problem of image jump will still occur.

[0041] While -2) overcomes the drawbacks of -1), its limitation lies in the fact that traditional algorithms, in determining the mapping relationship, require using prominent elements in the image as features (e.g., features extracted from areas with obvious textures) and basing the mapping relationship on these extracted features. If the texture features of the user's shooting scene are not obvious (e.g., in a dark room or a solid-color shooting scene), the difficulty of feature determination increases, leading to inaccurate mapping relationships based on feature points. Consequently, the generated mapping relationship in this shooting scene may fail to meet the user's shooting needs, resulting in image abrupt changes.

[0042] Based on this, embodiments of this disclosure propose a photographing method that determines a first feature based on a first image acquired by a first camera, wherein the first feature represents the association feature between a first feature point and surrounding pixels, and a target mapping relationship is determined based on the first image, a second image, and the first feature. The target mapping relationship determined by the photographing method proposed in this disclosure is more accurate, thereby mitigating the impact of image jumps during camera switching and improving the user experience.

[0043] It should be noted that the photographing method disclosed in this embodiment can be applied to a terminal. In some embodiments, the terminal includes, but is not limited to, at least one of the following: mobile phone, wearable device, Internet of Things device, car with communication function, smart car, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home.

[0044] For ease of understanding, the exemplary descriptions of "terminal" in the following embodiments of this disclosure can be understood as "smartphone".

[0045] It should be noted that in this embodiment, the terminal configured with multiple cameras can be a terminal configured with at least two cameras. Since the camera switching process for a terminal with more than two cameras is the same as that for a terminal with two cameras, the following embodiments will use a terminal with two cameras as an example for illustrative purposes.

[0046] It should be noted that, in order to facilitate understanding of the embodiments of this disclosure, some terms used in the embodiments of this disclosure will be explained below so that those skilled in the art can understand them.

[0047] Transformer network registration-based models, also known as Transformer models, are a deep learning architecture based on a self-attention mechanism. The core of the Transformer model is the self-attention mechanism and the cross-attention mechanism. The self-attention mechanism allows the model to dynamically focus on any other positional information within a sequence while processing it. The cross-attention mechanism allows the model to exchange and integrate information between different modalities or representation spaces.

[0048] Self-attention mechanism: The self-attention mechanism determines which information is most important to the current task by calculating the attention score at each position in the sequence, thereby achieving effective information capture and utilization. This mechanism greatly improves the model's ability to handle long-range dependencies and has significant advantages in parallel computing because it can process all positions in the sequence simultaneously without the computational efficiency decreasing with increasing sequence length, unlike RNNs.

[0049] Cross-attention mechanisms are used to process information from different sequences or representations. They are commonly used in encoder-decoder architectures, such as machine translation tasks, where the encoder processes the input sentence and the decoder generates the output sentence. Cross-attention allows the decoder to focus on parts of the encoder related to the current output word while generating the output word, thus more effectively capturing relevant information from the input sentence and improving translation quality.

[0050] Singular Value Decomposition (SVD) is an important mathematical tool in image processing. It's a method that decomposes any linear transformation into a product of multiple linear transformations, each corresponding to a different feature of the image. SVD can be used to analyze the features of an image matrix, transforming the image to a new coordinate system composed of the image's principal components. In image processing, SVD is commonly used for image compression, noise reduction, and feature extraction. SVD effectively reduces the dimensionality of image data while preserving its essential information, achieving image compression.

[0051] Figure 2 This is a flowchart illustrating a photographing method according to an exemplary embodiment, such as... Figure 2 As shown, the photo-taking method used in the terminal includes the following steps.

[0052] In step S11, the control terminal switches from the first camera to the second camera.

[0053] In step S12, a first image captured by the first camera is acquired, and a second image captured by the second camera is acquired.

[0054] In step S13, a first feature is determined based on the first image.

[0055] In step S14, the target mapping relationship is determined based on the first image, the second image, and the first feature.

[0056] In step S15, the target image is determined based on the first image and the target mapping relationship, and the target image is used as the image captured by the second camera.

[0057] In some embodiments, the first feature is the association feature between a first feature point in a first image and the pixels surrounding the first feature point, and the first feature point is the pixel in the first image used to determine the target mapping relationship.

[0058] In some embodiments, the target mapping relationship is used to characterize the positional projection relationship between pixels in the first image and pixels in the second image.

[0059] It should be noted that the feature points mentioned in the embodiments of this disclosure (including, for example, a first feature point, a second feature point, or an Nth feature point, etc.) can be understood as pixels in the corresponding image, which are used to determine the target mapping relationship. Compared with features in related technologies, it is not necessary to consider whether their texture features are obvious enough. That is, the first feature point mentioned in the embodiments of this disclosure can be any pixel in the first image used to determine the target mapping relationship based on algorithms, artificial intelligence (AI) models, or rules.

[0060] In this embodiment, by determining a first feature point in the first image and its association with surrounding pixels (i.e., a first feature), accurate identification of the first feature point and its surrounding pixels can be achieved even when the texture features of the first feature point are not obvious. This results in a smaller error in the target mapping relationship determined based on the first feature point and the first feature compared to a mapping relationship determined solely by the first feature point. Consequently, the impact of sudden changes in the captured image corresponding to the second camera, determined based on the target mapping relationship, is minimized, thereby improving the user experience.

[0061] In some embodiments, the first feature can be obtained through a corresponding AI model. For example, it can be implemented using a model based on transformer network registration. For instance, the first image is input into a model based on transformer network registration, and feature extraction is performed based on its convolutional neural network (CNN) layer to obtain the first feature point. Based on the self-attention mechanism in the network, the association relationship between the first feature point and surrounding pixels (i.e., the first feature) is obtained.

[0062] In some embodiments, the terminal switches from a first camera to a second camera by adjusting the focal length magnification. It is understood that the focal length magnification range corresponding to the first camera is different from that corresponding to the second camera. For example, the focal length magnification range corresponding to the first camera may be a to b. This focal length magnification range is understood as the terminal using the first camera for image acquisition (e.g., taking a picture) when the terminal adjusts the focal length magnification value to be greater than or equal to a and less than or equal to b, where a and b are both numerical values ​​representing the focal length magnification, and b is greater than or equal to a. As another example, the focal length magnification range corresponding to the second camera may be b to c. This focal length magnification range is understood as the terminal using the second camera for image acquisition (e.g., taking a picture) when the terminal adjusts the focal length magnification value to be greater than or equal to b and less than or equal to c, where b and c are both numerical values ​​representing the focal length magnification, and c is greater than or equal to b.

[0063] It is understandable that the focal length magnification corresponding to 'b' is also the focal length magnification at which the camera switches. At this focal length magnification, the terminal switches from the first camera to the second camera. Therefore, the target mapping relationship needs to be determined before this (the focal length magnification is adjusted to the focal length magnification corresponding to 'b').

[0064] In some embodiments, a target mapping relationship is determined when the terminal's focal length magnification meets a focal length magnification threshold. The focal length magnification threshold is determined based on the focal length magnification range corresponding to the first camera and / or the focal length magnification range corresponding to the second camera. For example, continuing from the above embodiments, if the focal length magnification range corresponding to the first camera is a to b, and the focal length magnification range corresponding to the second camera is b to c, the focal length magnification threshold can be set to e to f, where e and f are numerical values ​​representing focal length magnification, e is greater than or equal to a, f is less than or equal to b, and e is less than or equal to f. When the terminal's focal length magnification value falls within the range of e to f, the terminal determines the target mapping relationship.

[0065] It is understandable that by setting a focal length magnification threshold, the terminal can determine the target mapping relationship when the focal length magnification threshold is met, thereby achieving the determination of the target mapping relationship before switching cameras.

[0066] In some embodiments, “get,” “obtain,” “receive,” “transmit,” “bidirectional transmission,” “send and / or receive” can be used interchangeably and can be interpreted as receiving from other entities, but are not limited thereto.

[0067] In some embodiments, terms such as "certain," "preset," "default," "set," "indicated," "a certain," "any," and "first" can be used interchangeably. "Certain A," "preset A," "default A," "set A," "indicated A," "a certain A," "any A," and "first A" can be interpreted as A pre-defined in a protocol or the like, or as A obtained through setting, configuration, or instruction, or as specific A, a certain A, any A, or first A, but are not limited thereto.

[0068] The photographing method disclosed in this embodiment may include at least one of steps S11 to S15. For example, step S13 may be implemented as a separate embodiment, step S14 may be implemented as a separate embodiment, step S11+step S11+step S13 may be implemented as a separate embodiment, and step S11+step S11+step S13+step S14 may be implemented as a separate embodiment, but is not limited thereto.

[0069] In some embodiments, steps S11, S12, S13 and S14 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0070] In some embodiments, steps S11, S12, S13 and S15 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0071] In some embodiments, steps S11, S12, S14 and S15 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0072] In some embodiments, steps S11, S13, S14 and S15 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0073] In some embodiments, steps S11, S13, S14 and S15 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0074] Figure 3 This is a flowchart illustrating a method for determining a target mapping relationship according to an exemplary embodiment, such as... Figure 3 As shown, it includes the following steps.

[0075] In step S21, the second feature point and the second feature are determined based on the first image and the second image.

[0076] In step S22, a set of matching points between the first image and the second image is determined based on the first feature point, the first feature, and the second feature point.

[0077] In step S23, the target mapping relationship is determined based on the matching point set.

[0078] In some embodiments, the second feature point is the corresponding projected pixel of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point.

[0079] In some embodiments, the set of matching points includes at least one pair of matching points, and each pair of matching points includes a first feature point and a second feature point.

[0080] It is understandable that the first feature point and the second feature point can be considered as pixels with the same name, and they are related. Therefore, the first feature point and its corresponding second feature point can be referred to as a matching point pair. And the set of matching points can include at least one matching point pair.

[0081] In this embodiment of the disclosure, by combining the first feature and the first feature point to determine the pixel point (i.e. the second feature point) corresponding to the first feature point in the second image, and constructing a matching point set based on the first feature point and the second feature point, the target mapping relationship is determined based on the correspondence in the matching point set. This can realize the determination of the target mapping relationship based on the matching relationship between the first feature point and the second feature point. Compared with the mapping relationship determined only based on the first feature point and the second feature point, the error of the target mapping relationship is reduced, thereby making the impact of the jump in the captured image corresponding to the second camera determined based on the target mapping relationship smaller, thus improving the user experience.

[0082] In some embodiments, the determination of the second feature can be achieved through a corresponding AI model. For example, it can be achieved through a model based on transformer network registration. For instance, the first image and the second image are input into the model based on transformer network registration, and based on the cross-attention mechanism in the network, the corresponding feature point of the first feature point in the second image (i.e., the second feature point) is obtained, and the correlation between the first feature point and the second feature point can be output simultaneously.

[0083] Optionally, the first feature point, the first feature point, and the second feature can be obtained using the same AI model. This includes, for example, using a model based on transformer network registration. The first and second images are input into the transformer network registration model, and feature extraction is performed using its CNN to obtain the first feature point. Based on the self-attention mechanism in the network, the association between the first feature point and surrounding pixels (i.e., the first feature) is obtained. Then, based on its cross-attention mechanism, the second feature point and the second feature are obtained. It is understood that by using a model based on transformer network registration, global attention enhancement of the image can be achieved, making the output first feature point, second feature point, first feature, and second feature more accurate. This allows for better matching of low-texture images acquired from a camera.

[0084] In some embodiments, after obtaining the first feature point, the second feature point, the first feature, and the second feature, more refined feature matching can be performed. For example, continuing from the above embodiments, the coordinates (i1, j1) of the first feature point obtained in the above embodiments are corresponding to the coordinates (i2, j2) of the second feature point. These are mapped to corresponding more refined feature maps, and their mapped neighborhoods are cropped. The cropped neighborhood is input into another AI model (e.g., a model based on transformer network registration, different from the model in the above embodiments, and with a more refined computational granularity than the AI ​​model in the above embodiments) for feature extraction, obtaining the extracted features corresponding to the first feature point and the extracted features corresponding to the second feature point, respectively. The matching probability between the extracted features corresponding to the first feature point and all extracted features corresponding to the second feature point is then calculated, and the sub-pixel precision matching point positions among all extracted features corresponding to the second feature point are determined based on the probability distribution.

[0085] The photographing method disclosed in this embodiment may include at least one of steps S21 to S23. For example, step S21 may be implemented as a separate embodiment, step S22 may be implemented as a separate embodiment, and step S21+step S22+step S23 may be implemented as a separate embodiment, but is not limited thereto.

[0086] In some embodiments, steps S21 and S22 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0087] In some embodiments, steps S21 and S23 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0088] In some embodiments, steps S22 and S23 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0089] Figure 4 This is a flowchart illustrating a method for determining a target mapping relationship according to an exemplary embodiment, such as... Figure 4 As shown, it includes the following steps.

[0090] In step S31, a first mapping relationship is determined based on a first subset of the matching point set, the first subset including at least one pair of matching points.

[0091] In step S32, based on the first feature point in the matching point set and the first mapping relationship, the position information of the first feature point in the second image based on the first mapping relationship is calculated.

[0092] In step S33, if the error between the location information and the location information corresponding to the second feature point meets the error threshold, then the first mapping relationship is determined as the target mapping relationship.

[0093] In this embodiment of the disclosure, the first mapping relationship is determined based on a subset of the pairing point set, and the first mapping relationship that meets the error requirement is taken as the target mapping relationship. The first mapping relationship that does not meet the error requirement can be eliminated, thereby reducing the solution error of the target mapping relationship. This makes the impact of the jump in the captured image corresponding to the second camera determined based on the target mapping relationship smaller, thereby improving the user experience.

[0094] In some embodiments, the set of matching points includes one or more subsets of matching points, each subset of matching points including at least one pair of matching points.

[0095] In some embodiments, the first mapping relationship can be confirmed based on matching point pairs in a first subset. For example, the number of matching point pairs in the first subset is 'a', where a is an integer greater than or equal to 1. Then, based on the position information (e.g., coordinate information) of the first feature point and the position information (e.g., coordinate information) of the second feature point in the matching point pair, the projection relationship between the first feature point and the second feature point in the corresponding first subset is determined (e.g., by calculating the projection relationship for each matching point pair separately and then performing a weighted average calculation).

[0096] In some embodiments, if the error between the location information and the location information corresponding to the second feature point does not meet the error threshold, a second subset is determined in the matching point set, and the first mapping relationship is re-determined based on the second subset, wherein the second subset is different from the first subset.

[0097] The photographing method disclosed in this embodiment may include at least one of steps S31 to S33. For example, step S31 may be implemented as a standalone embodiment, step S32 may be implemented as a standalone embodiment, and step S31+step S32+step S33 may be implemented as a standalone embodiment, but is not limited thereto.

[0098] In some embodiments, steps S31 and S32 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0099] In some embodiments, steps S31 and S33 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0100] In some embodiments, steps S32 and S33 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0101] Figure 5 This is a flowchart illustrating a method for determining a first feature according to an exemplary embodiment, such as... Figure 5 As shown, it includes the following steps.

[0102] In step S41, the first image is magnified and then the magnified image is cropped from the center to obtain the third image.

[0103] In step S42, the third image is grayscale registered based on the second image to obtain the fourth image.

[0104] In step S43, a first feature point is selected in the fourth image, and the pixels surrounding the first feature point are determined.

[0105] In step S44, the first feature is determined based on the first feature point and the pixels surrounding the first feature point.

[0106] In some embodiments, the first image is magnified, for example, by using a center magnification strategy.

[0107] In this embodiment of the disclosure, by performing feature amplification and grayscale registration on the first image before determining the first feature, the features in the first image can be enhanced, thereby reducing the risk of feature recognition due to unclear features, and thus improving the accuracy of the first feature recognition in the game.

[0108] It is understandable that the first feature obtained through steps S41 to S42 is a feature of the cropped region in the first image. Since steps S31 to S33 are based on global features in the first image, the first feature obtained through steps S41 to S42 needs to be restored to global features (including, for example, multiplying the feature by the inverse of the cropping matrix) before proceeding with steps S31 to S33.

[0109] The photographing method disclosed in this embodiment may include at least one of steps S41 to S44. For example, step S41 may be implemented as a standalone embodiment, step S42 may be implemented as a standalone embodiment, and step S41+S42+S43+S44 may be implemented as a standalone embodiment, but is not limited thereto.

[0110] In some embodiments, steps S41, S42, and S43 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0111] In some embodiments, steps S41, S42, and S44 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0112] In some embodiments, steps S41, S43, and S44 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0113] In some embodiments, steps S42, S43, and S44 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0114] Figure 6 This is a flowchart illustrating a method for determining a target image according to an exemplary embodiment, such as... Figure 6 As shown, it includes the following steps.

[0115] In step S51, the target mapping relationship is decomposed into a second mapping relationship and a third mapping relationship.

[0116] In step S52, based on the focal length magnification of the first camera, the pixel translation and pixel rotation amounts corresponding to the projection of the first image onto the second image are determined.

[0117] In step S53, the target image is determined based on the first image, pixel translation amount, and pixel rotation amount.

[0118] In some embodiments, the second mapping relationship represents the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image, and the third mapping relationship represents the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image.

[0119] It is understandable that by decomposing the target mapping relationship into a second mapping relationship and a third mapping relationship, the correspondence between the translation and rotation components in the target mapping relationship corresponding to the focal length magnification can be determined. The terminal can then process the first image based on the second and third mapping relationships to obtain the target image.

[0120] In some embodiments, the target mapping relationship can be decomposed into a second mapping relationship and a third mapping relationship using a singular value decomposition matrix.

[0121] The photographing method disclosed in this embodiment may include at least one of steps S51 to S53. For example, step S51 may be implemented as a standalone embodiment, step S52 may be implemented as a standalone embodiment, and step S51+step S52+step S53 may be implemented as a standalone embodiment, but is not limited thereto.

[0122] In some embodiments, steps S51 and S52 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0123] In some embodiments, steps S51 and S53 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0124] In some embodiments, steps S52 and S53 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0125] This disclosure also proposes a photographic method for more accurate alignment of dual-camera images in low-texture scenes, thereby ensuring a smooth transition when switching between lenses with different focal lengths.

[0126] In some embodiments, the photographing method includes the following -A) to -I), the method comprising:

[0127] -A) Open the terminal camera (camera app or other application that can take pictures), control the camera to capture the original image (Raw) information and send it to the terminal processor. After a certain image processing process, the image is sent to the terminal screen for display.

[0128] Optionally, the camera application interface of a typical terminal has a zoom function, allowing users to manually select the desired focal length. From the image I of the first camera... A Starting from the native magnification, zoom in initially using a center-magnification strategy, performing center cropping, denoted as... And the cropped image is interpolated to the native resolution, denoted as . The digitally cropped image is sent to the terminal screen for display.

[0129] -B) After magnification reaches the predetermined magnification range, begin calculating the first camera I. A With the second camera I B The projection matrix between them. As mentioned earlier, the native image from the first camera obtained by the mobile phone processor is digitally zoomed to match the current user's magnification. The digitally zoomed first camera image and the native image from the second camera are then converted to grayscale and histogram matched. The histogram matching target is the second camera. The two processed images are denoted as follows: and I B,gray .

[0130] -C) The matched grayscale image pairs from step -B) are sent from the terminal processor to the dedicated AI processing module in the terminal. A transformer network model and weights are used to calculate the matching point pairs, which are then sent back to the terminal processor as P.match .

[0131] Optionally, the transformer network registration model may include: a CNN layer for local feature extraction, a self-attention and cross-attention layer for coarse local feature transformation, a differentiable feature matching layer for feature matching, and a matching layer for fine-grained matching.

[0132] -D) Using the matching point pairs obtained in step -C), perform Random Sampling Consensus (RANSAC) processing to obtain the optimal matching point from multiple sets of observed matching point pairs through random iteration. (For example, four sets of matching points), calculated based on these four sets of matching points. with I B,gray The projection matrix H crop H crop Multiplying by the inverse of the first camera image cropping matrix yields I. A with I B The full-map projection matrix H full .

[0133] -E) Split the projection matrix from step -D). Typically, splitting the translation shift... full Rotation with z-axis theta full Two components.

[0134] -F) Within the native magnification range of the first and second cameras, the translation shift and rotation theta obtained above are adjusted according to the user's magnification ratio at that time. user Perform a linear split to obtain shift sub With theta sub The closer to the native magnification of the second camera, the greater the translation and rotation. Based on these two components, the sub-projection matrix H is recalculated. sub .

[0135] -G) will project the sub-projection matrix H sub Applying it to the image from the first camera, we obtain And then it is displayed on the terminal screen.

[0136] -H) When the native magnification of the second camera is reached, the uncropped image of the second camera is sent to the terminal screen for display.

[0137] -I) Repeat the above process for the second camera image I B With the third camera image I C The same process is performed to ensure that the image displayed on the terminal screen moves and rotates smoothly during zooming, without any abrupt changes in the field of view (FOV).

[0138] Based on the same concept, this disclosure also provides a photographing device.

[0139] It is understood that the photographing device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0140] Figure 7 This is a block diagram illustrating a photographing device 100 according to an exemplary embodiment. (Refer to...) Figure 7 The device includes a control unit 111, an acquisition unit 112, and a processing unit 113.

[0141] The control unit 111 is used to control the terminal to switch from the first camera to the second camera.

[0142] The acquisition unit 112 is used to acquire a first image captured by the first camera and a second image captured by the second camera.

[0143] The processing unit 113 is used to determine a first feature based on the first image. The first feature is the association feature between a first feature point in the first image and the pixels surrounding the first feature point. The first feature point is the pixel in the first image used to determine the target mapping relationship.

[0144] The processing unit 113 is also used to determine a target mapping relationship based on the first image, the second image, and the first feature. The target mapping relationship is used to characterize the positional projection relationship between pixels in the first image and pixels in the second image.

[0145] The processing unit 113 is also used to determine the target image based on the first image and the target mapping relationship, and use the target image as the image captured by the second camera.

[0146] In some embodiments, the processing unit 113 determines the target mapping relationship based on the first image, the second image, and the first feature in the following manner: determining a second feature point and a second feature based on the first image and the second image, wherein the second feature point is the corresponding projected pixel point of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point; determining a matching point set between the first image and the second image based on the first feature point, the first feature, and the second feature point, wherein the matching point set includes at least one matching point pair, and each matching point pair includes a first feature point and a second feature point; and determining the target mapping relationship based on the matching point set.

[0147] In some embodiments, the processing unit 113 determines the target mapping relationship based on the matching point set in the following manner: determining a first mapping relationship based on a first subset of the matching point set, the first subset including at least one pair of matching points; calculating the position information of the first feature point in the second image based on the first mapping relationship based on the first feature point in the matching point set; and determining the first mapping relationship as the target mapping relationship if the error between the position information and the position information corresponding to the second feature point meets an error threshold.

[0148] In some embodiments, the processing unit 113 determines the target image based on the first image and the target mapping relationship in the following manner: decomposing the target mapping relationship into a second mapping relationship and a third mapping relationship, wherein the second mapping relationship characterizes the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image, and the third mapping relationship characterizes the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image; determining the pixel translation amount and pixel rotation amount corresponding to the projection of the first image onto the second image based on the focal length magnification of the first camera; and determining the target image based on the first image, the pixel translation amount, and the pixel rotation amount.

[0149] In some embodiments, the processing unit 113 determines a first feature based on a first image in the following manner: the first image is magnified and the magnified image is cropped at the center to obtain a third image; the third image is grayscale registered based on a second image to obtain a fourth image; a first feature point is selected in the fourth image and the pixels around the first feature point are determined; the first feature is determined based on the first feature point and the pixels around the first feature point.

[0150] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0151] Figure 8This is a block diagram illustrating a device 200 for taking pictures according to an exemplary embodiment. For example, device 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0152] Reference Figure 8 The photographing device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0153] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0154] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0155] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0156] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0157] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0158] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0159] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0160] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0161] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0162] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0163] Figure 9 This is a block diagram illustrating a device 300 for taking a picture according to an exemplary embodiment. For example, device 300 may be provided as a server. (See also...) Figure 9 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions executable by the processing component 322, such as application programs. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the aforementioned screen display method.

[0164] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0165] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0166] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0167] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0168] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0169] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0170] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for taking photos, characterized in that, The method includes: The control terminal switches from the first camera to the second camera, acquires the first image captured by the first camera, and acquires the second image captured by the second camera; A first feature is determined based on the first image. The first feature is the association feature between a first feature point in the first image and the pixels surrounding the first feature point. The first feature point is the pixel in the first image used to determine the target mapping relationship. Based on the first image, the second image, and the first feature, a target mapping relationship is determined, wherein the target mapping relationship is used to characterize the positional projection relationship between pixels in the first image and pixels in the second image; The target image is determined based on the first image and the target mapping relationship, and the target image is used as the image captured by the second camera.

2. The method according to claim 1, characterized in that, Determining the target mapping relationship based on the first image, the second image, and the first feature includes: Based on the first image and the second image, a second feature point and a second feature are determined. The second feature point is the corresponding projected pixel point of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point. Based on the first feature point, the first feature, and the second feature point, a set of matching points between the first image and the second image is determined. The set of matching points includes at least one pair of matching points, and each pair of matching points includes the first feature point and the second feature point. Based on the set of matching points, the target mapping relationship is determined.

3. The method according to claim 2, characterized in that, Determining the target mapping relationship based on the matching point set includes: A first mapping relationship is determined based on a first subset of the set of matching points, wherein the first subset includes at least one pair of matching points. Based on the first feature point in the matching point set and the first mapping relationship, the position information of the first feature point in the second image based on the first mapping relationship is calculated. If the error between the location information and the location information corresponding to the second feature point meets the error threshold, then the first mapping relationship is determined as the target mapping relationship.

4. The method according to claim 1, characterized in that, Determining the target image based on the first image and the target mapping relationship includes: The target mapping relationship is decomposed into a second mapping relationship and a third mapping relationship. The second mapping relationship represents the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image. The third mapping relationship represents the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image. Based on the focal length magnification of the first camera, determine the pixel translation and pixel rotation amounts corresponding to the projection of the first image onto the second image; The target image is determined based on the first image, the pixel translation amount, and the pixel rotation amount.

5. The method according to claim 1, characterized in that, The step of determining the first feature based on the first image includes: The first image is magnified, and the magnified image is then cropped from the center to obtain the third image; The third image is registered in grayscale based on the second image to obtain the fourth image; In the fourth image, the first feature point is selected, and the pixels surrounding the first feature point are determined; The first feature is determined based on the first feature point and the pixels surrounding the first feature point.

6. A photographing device, characterized in that, The device includes: The control unit is used to control the terminal to switch from the first camera to the second camera; The acquisition unit is used to acquire a first image captured by the first camera and a second image captured by the second camera; The processing unit is configured to determine a first feature based on the first image, wherein the first feature is the association feature between a first feature point in the first image and the pixels surrounding the first feature point, and the first feature point is the pixel in the first image used to determine the target mapping relationship. The processing unit is further configured to determine a target mapping relationship based on the first image, the second image, and the first feature, wherein the target mapping relationship is used to characterize the positional projection relationship between pixels in the first image and pixels in the second image; The processing unit is further configured to determine a target image based on the first image and the target mapping relationship, and use the target image as the captured image corresponding to the second camera.

7. The photographing device according to claim 6, characterized in that, The processing unit determines the target mapping relationship based on the first image, the second image, and the first feature in the following manner: Based on the first image and the second image, a second feature point and a second feature are determined. The second feature point is the corresponding projected pixel point of the first feature point in the second image, and the second feature is the association feature between the first feature point and the second feature point. Based on the first feature point, the first feature, and the second feature point, a set of matching points between the first image and the second image is determined. The set of matching points includes at least one pair of matching points, and each pair of matching points includes the first feature point and the second feature point. Based on the set of matching points, the target mapping relationship is determined.

8. The photographing device according to claim 7, characterized in that, The processing unit determines the target mapping relationship based on the matching point set in the following manner: A first mapping relationship is determined based on a first subset of the set of matching points, wherein the first subset includes at least one pair of matching points. Based on the first feature point in the matching point set and the first mapping relationship, the position information of the first feature point in the second image based on the first mapping relationship is calculated. If the error between the location information and the location information corresponding to the second feature point meets the error threshold, the first mapping relationship is determined as the target mapping relationship.

9. The photographing device according to claim 6, characterized in that, The processing unit determines the target image based on the first image and the target mapping relationship in the following manner: The target mapping relationship is decomposed into a second mapping relationship and a third mapping relationship. The second mapping relationship represents the relationship between the focal length magnification and the pixel translation amount corresponding to the projection of a pixel in the first image onto the second image. The third mapping relationship represents the relationship between the focal length magnification and the pixel rotation amount corresponding to the projection of a pixel in the first image onto the second image. Based on the focal length magnification of the first camera, determine the pixel translation and pixel rotation amounts corresponding to the projection of the first image onto the second image; The target image is determined based on the first image, the pixel translation amount, and the pixel rotation amount.

10. The photographing device according to claim 6, characterized in that, The processing unit determines the first feature based on the first image in the following manner: The first image is magnified, and the magnified image is then cropped from the center to obtain the third image; The third image is registered in grayscale based on the second image to obtain the fourth image; In the fourth image, the first feature point is selected, and the pixels surrounding the first feature point are determined; The first feature is determined based on the first feature point and the pixels surrounding the first feature point.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the photographing method according to any one of claims 1 to 5.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the terminal's processor, enable the terminal to perform the photographing method according to any one of claims 1 to 5.

Images