Method, apparatus, electronic device and readable medium for generating annular object image

Abstract

An embodiment of the present invention provides a method, apparatus, electronic device, and readable medium for generating a panoramic object image. The method includes: in response to an operation of a photographing device for surrounding and photographing a target object, acquiring a plurality of object images corresponding to the target object in different acquisition directions; identifying the device pose corresponding to the object image based on the image feature points in the object image; determining the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image; adjusting the object images with device poses close to the ideal pose direction based on the ideal pose to obtain target images in the ideal pose; and generating a panoramic object image corresponding to the target object using the target images in the ideal pose. Thus, it is possible to obtain a panoramic object image with high quality by using a hand-held photographing device without the need for photographing auxiliary facilities such as a turntable.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular, to a method for generating a panoramic image, an apparatus for generating a panoramic image, an electronic device, and a computer-readable medium. Background Art

[0002] A panoramic image is an image obtained by using a panoramic shooting method. Generally speaking, if a panoramic image needs to be obtained, the shooting device can be set at a fixed position, the target object to be shot rotates on a turntable, and the shooting device takes images at a fixed frequency, and finally the obtained images are combined into a panoramic image. However, if the target object to be shot is relatively large, it often takes a high installation and construction cost to build a turntable, resulting in a high cost for panoramic shooting. Summary of the Invention

[0003] Embodiments of the present invention provide a method, an apparatus, an electronic device, and a computer-readable storage medium for generating a panoramic image, so as to obtain a panoramic image with high quality without setting up a turntable.

[0004] Embodiments of the present invention disclose a method for generating a panoramic image, including:

[0005] In response to an operation of a shooting device for performing a surrounding shooting on a target object, obtaining a plurality of object images corresponding to the target object in different acquisition directions, where there is partial overlapping image content between every two adjacent object images;

[0006] Based on image feature points in the object images, identifying a device pose corresponding to the object images;

[0007] Based on the device pose and the position of the target object in the object images, determining an ideal pose corresponding to each acquisition direction; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image;

[0008] Based on the ideal pose, adjusting the object images whose device poses are close to the ideal pose direction to obtain target images in the ideal pose;

[0009] Generating a panoramic image corresponding to the target object by using the target images in the ideal pose.

[0010] Optionally, the step of in response to an operation of a shooting device for performing a surrounding shooting on a target object, obtaining a plurality of object images corresponding to the target object in different acquisition directions includes:

[0011] In response to the operation of the shooting device for surrounding shooting of the target object, the lateral deviation angle of the shooting device is detected in real time, and the lateral deviation angle is the angle of rotation around the axis perpendicular to the ground;

[0012] Whenever the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, an object image collected by the shooting device is acquired, and a plurality of object images corresponding to the target object in different acquisition directions are obtained.

[0013] Optionally, the step of identifying the device pose corresponding to the object image based on the image feature points in the object image includes:

[0014] According to the partially overlapping image content existing between every two adjacent object images, the common image feature points between the adjacent object images are identified to form image matching pairs;

[0015] Based on the image matching pairs, an essential matrix between the adjacent object images is constructed;

[0016] Based on the essential matrix, the device pose of the object image is determined.

[0017] Optionally, the step of determining the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image includes:

[0018] Select one of the object images, and use the spatial position corresponding to the object image as the initial position;

[0019] Starting from the initial position, a preset number of ideal poses are generated by horizontally surrounding the center point.

[0020] Optionally, the step of adjusting the object image with a device pose similar to the ideal pose based on the ideal pose to obtain a target image in the ideal pose includes:

[0021] For one of the ideal poses, the object image with a device pose similar to the ideal pose is used as the image to be processed matching the ideal pose;

[0022] Obtain the first rotation matrix and the first position information recorded in the ideal pose, and obtain the device internal parameter matrix, the second rotation matrix, and the second position information recorded in the device pose corresponding to the image to be processed;

[0023] Based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix, a first transformation matrix is determined;

[0024] Based on the first position information and the second position information, a second transformation matrix is determined;

[0025] Adjust the object image by using the first transformation matrix and the second transformation matrix to obtain a target image in the ideal pose.

[0026] Optionally, the step of generating a panoramic image corresponding to the target object by using the target image in the ideal pose includes:

[0027] Identify the area where the target object is located in each target image;

[0028] Extract the image of the area where the target object is located in each target image to obtain a target object image;

[0029] Adjust the target object image so that the sizes of the target objects in the target object image are unified;

[0030] Generate a panoramic image corresponding to the target object by using the target object image.

[0031] Optionally, the method further includes:

[0032] Identify at least one area to be processed in the panoramic image and perform special effect processing on the area to be processed; wherein, the special effect processing includes at least one of scene replacement, information addition, and information deletion.

[0033] An embodiment of the present invention also provides a device for generating a panoramic image, including:

[0034] An image acquisition module, configured to acquire a plurality of object images corresponding to the target object in different acquisition directions in response to an operation of a shooting device for surrounding shooting of the target object;

[0035] A pose recognition module, configured to recognize the device pose corresponding to the object image based on the image feature points in the object image;

[0036] An ideal pose generation module, configured to determine the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image; wherein, the ideal pose is centered on the target object and horizontally surrounds the target object;

[0037] An image adjustment module, configured to adjust the object image with a device pose close to the ideal pose direction based on the ideal pose to obtain a target image in the ideal pose;

[0038] A panoramic image generation module, configured to generate a panoramic image corresponding to the target object by using the target image in the ideal pose.

[0039] Optionally, the image acquisition module includes:

[0040] An angle detection sub-module, which is configured to respond to an operation of a photographing device for surrounding photographing of a target object, and to detect in real time a lateral deviation angle of the photographing device, where the lateral deviation angle is an angle of rotation about an axis perpendicular to the ground;

[0041] An image acquisition sub-module, which is configured to acquire an object image collected by the photographing device every time the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, so as to obtain a plurality of object images corresponding to the target object in different acquisition directions.

[0042] Optionally, the pose recognition module includes:

[0043] An image matching sub-module, which is configured to identify common image feature points between adjacent object images according to partially overlapping image content existing between every two adjacent object images, so as to form an image matching pair;

[0044] An essential matrix construction sub-module, which is configured to construct an essential matrix between adjacent object images based on the image matching pair;

[0045] A pose recognition sub-module, which is configured to determine the device pose of the object image based on the essential matrix.

[0046] Optionally, the ideal pose generation module includes:

[0047] An initial position selection module, which is configured to select one of the object images and use the spatial position corresponding to the object image as the initial position;

[0048] An ideal position generation sub-module, which is configured to generate a preset number of ideal poses by horizontally surrounding the center point starting from the initial position.

[0049] Optionally, the image adjustment module includes:

[0050] An image selection sub-module, which is configured to, for one of the ideal poses, use an object image whose device pose is close to the direction of the ideal pose as a to-be-processed image matched with the ideal pose;

[0051] An information acquisition sub-module, which is configured to acquire a first rotation matrix and first position information recorded in the ideal pose, and to acquire a device internal parameter matrix, a second rotation matrix, and second position information recorded in the device pose corresponding to the to-be-processed image;

[0052] A first transformation matrix determination sub-module, which is configured to determine a first transformation matrix based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix;

[0053] A second transformation matrix determination sub-module, configured to determine a second transformation matrix based on the first position information and the second position information;

[0054] An image adjustment sub-module, configured to adjust the object image by using the first transformation matrix and the second transformation matrix to obtain a target image in the ideal pose.

[0055] Optionally, the annular object image generation module includes:

[0056] A region recognition module, configured to recognize the region where the target object is located in each of the target images;

[0057] An image extraction sub-module, configured to extract an image of the region where the target object is located in each of the target images to obtain a target object image;

[0058] A size unification sub-module, configured to adjust the target object image so that the sizes of the target objects in the target object image are unified;

[0059] An annular object image generation sub-module, configured to generate an annular object image corresponding to the target object by using the target object image.

[0060] Optionally, the apparatus further includes:

[0061] An special effect processing module, configured to recognize at least one region to be processed in the annular object image and perform special effect processing on the region to be processed; wherein, the special effect processing includes at least one of scene replacement, information addition, and information deletion.

[0062] An embodiment of the present invention further discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0063] The memory is configured to store a computer program;

[0064] The processor is configured to implement the method as described in the embodiment of the present invention when executing the program stored in the memory.

[0065] An embodiment of the present invention further discloses one or more computer-readable media, on which instructions are stored, and when executed by one or more processors, cause the processors to execute the method as described in the embodiment of the present invention.

[0066] The embodiments of the present invention include the following advantages:

[0067] In an embodiment of the present invention, in response to an operation of a shooting device for surrounding shooting of a target object, a plurality of object images corresponding to the target object in different acquisition directions are acquired, where there is partial overlapping image content between every two adjacent object images; based on image feature points in the object images, the device pose corresponding to the object images is identified; based on the device pose and the position of the target object in the object images, the ideal pose corresponding to each acquisition direction is determined; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image; based on the ideal pose, the object images with device poses close to the ideal pose are adjusted to obtain target images in the ideal pose; the target images in the ideal pose are used to generate a panoramic image corresponding to the target object. Thus, without the need for shooting auxiliary facilities such as a turntable, by using a hand-held shooting device, a panoramic image of relatively high quality can be obtained. Description of the Drawings

[0068] Figure 1 is a flowchart of the steps of a panoramic image generation method provided in an embodiment of the present invention;

[0069] Figure 2 is a schematic diagram of the device pose in a three-dimensional space provided in an embodiment of the present invention;

[0070] Figure 3 is a schematic diagram of the ideal pose in a three-dimensional space provided in an embodiment of the present invention;

[0071] Figure 4 is a flowchart of the steps of another panoramic image generation method provided in an embodiment of the present invention;

[0072] Figure 5 is a schematic diagram of a target image provided in an embodiment of the present invention;

[0073] Figure 6 is a structural block diagram of a panoramic image generation device provided in an embodiment of the present invention;

[0074] Figure 7 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0075] Figure 8 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Embodiments

[0076] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0077] In the embodiment of the present invention, in order to obtain a panoramic image of an object without setting a turntable, the adopted method is that the object to be photographed is kept at a fixed position, while the photographing device for the object to be photographed moves around the object to be photographed for photographing. In this photographing method, since the photographing device moves during the photographing process, situations may occur such as the photographing device not maintaining a fixed distance from the object, the photographing device not maintaining a fixed height, and the attitude of the photographing device changing, resulting in generally no way to obtain a panoramic image with high quality. However, in the embodiment of the present invention, the device pose of the photographing device is determined based on the object image itself obtained by the photographing device, and a number of ideal poses are generated. Based on the ideal poses, the object image is adjusted to obtain a number of target images equivalent to being photographed in the ideal poses. Then, a panoramic image is generated based on the target images, thereby realizing the acquisition of a panoramic image with high quality.

[0078] Referring to Figure 1 , a flowchart of the steps of a method for generating a panoramic image provided in an embodiment of the present invention is shown, which may specifically include the following steps:

[0079] Step 101, in response to an operation of the photographing device for surrounding and photographing the target object, obtain a number of object images corresponding to the target object in different acquisition directions, where there is partial overlapping image content between every two adjacent object images;

[0080] Specifically, when a user needs to obtain a panoramic image, the photographing device can be used to surround and photograph the target object. The photographing device can be a device with a camera such as a camera, a mobile phone, a tablet computer, a drone, etc., and the present invention does not limit this.

[0081] During the process of the photographing device moving around the target object, the photographing device can continuously photograph the target object. The photographing device itself or an electronic device communicatively connected to the photographing device can, in response to the operation of the photographing device for surrounding and photographing the target object, obtain a number of object images corresponding to the target object in different acquisition directions photographed by the photographing device. For the convenience of subsequent processing, there can be partial overlapping image content between every two adjacent object images.

[0082] In a specific implementation, in order to ensure that there is partial overlapping image content between every two adjacent object images, the photographing device can be controlled to photograph at a relatively high frequency to ensure that there is overlap between the acquired images. For example, the photographing device can be controlled to perform image photographing every preset time period, or the photographing device can be controlled to perform image photographing after moving a certain distance, or the photographing device can be controlled to perform image photographing after the lateral deviation angle increases by a certain value, etc., and the present invention does not limit this.

[0083] Step 102: Based on the image feature points in the object image, identify the device pose corresponding to the object image;

[0084] After obtaining the object image, the device pose of the imaging device when each object image is captured can be determined based on the object image. This device pose can be used as the position and orientation of the object image in three-dimensional space, so that the object image can be adjusted based on the device pose subsequently. Among them, the device pose can include the position and orientation of the device, which can usually be represented by a rotation matrix R and a translation matrix t.

[0085] Specifically, there may be image feature points in the object image. Compared with general object images, image feature points can have more information and will not change due to factors such as illumination, noise, and perspective change, having scale invariance and rotation invariance. For example, the image feature points can be the corner positions of the object in the object image, the edge positions of the object, the bright points in the dark area, the dark points in the bright area, etc. The relative pose between adjacent object images can be determined based on the position change of the image feature points in the adjacent object images. Subsequently, based on the relative pose between adjacent object images, the pose change of the imaging device corresponding to all object images can be determined, and finally the device pose corresponding to each object image can be determined. As a specific example of the present invention, the device pose in three-dimensional space can be as Figure 2 shown.

[0086] Step 103: Based on the device pose and the position of the target object in the object image, determine the ideal pose corresponding to each acquisition direction; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image;

[0087] Specifically, in order to obtain a high-quality panoramic image of the object, generally, it is necessary to keep a fixed distance between the imaging device and the target object, a fixed height from the ground, and maintain a fixed pose, and capture the target object from multiple directions. Thus, several ideal poses can be set horizontally around the target object with the target object as the center point. If the imaging device captures the target object in the ideal pose, since the imaging device is at a fixed distance from the target object, a fixed height from the ground, and maintains a fixed pose at this time, better object images can be obtained, and further high-quality panoramic images of the object can be generated.

[0088] Among them, the center point of the target object is determined based on the device pose of each object image and the position of the target object in each object image. Specifically, based on the device pose corresponding to the object image, the position and pose of the object image in the three-dimensional space can be known. At the same time, the target detection method can be used to detect the position of the target object in the object image, and a target detection frame is used to frame the target object on the object image. Subsequently, the focus of the camera is connected to the corner points of the target detection frame and extended towards the target object, and a visual cone pointing to the target object can be formed. Each object image can form a visual cone, so that in the three-dimensional space, the visual cones corresponding to each object image can overlap with each other to form a common area. Since the visual cone is generated based on the position of the target object in the object image, this common area can be considered as the area where the target object is located in the three-dimensional space, and the center point of this area is the center point of the target object. As a specific example of the present invention, the ideal pose in the three-dimensional space can be as Figure 3 shown.

[0089] Optionally, the center point of the target object can also be determined during the process of determining the device pose corresponding to the object image. Specifically, the image feature points in the object image actually point to a certain three-dimensional point in the three-dimensional space. During the process of determining the device pose corresponding to the object, based on the epipolar geometry, according to the coordinates of the common image feature points in each object image and the device pose corresponding to the object image, the position of the three-dimensional point corresponding to the image feature points in the three-dimensional space can be calculated. Thus, a three-dimensional point cloud formed by the three-dimensional points corresponding to the image feature points can be constructed in the three-dimensional space, and the center point of the target object can be determined based on the three-dimensional point cloud. However, generally speaking, there may be a certain deviation in the center point of the target object determined by this method. And the center point determined based on the common area of the overlapping visual cones can have higher accuracy.

[0090] Step 104, based on the ideal pose, adjust the object images whose device poses are close to the ideal pose direction to obtain target images in the ideal pose;

[0091] After determining the ideal pose, the object images can be adjusted based on the device pose and the ideal pose to obtain target images in the ideal pose, so that subsequently, high-quality panoramic images can be obtained based on the target images in the ideal pose.

[0092] Specifically, in order to obtain better target images, for an ideal pose, the object images adjacent to the device position and this ideal pose can be preferentially selected for adjustment, so that target images in the ideal pose can be obtained with fewer adjustments.

[0093] Step 105: Generate a panoramic image corresponding to the target object using the target image in the ideal pose.

[0094] After obtaining the target image in the ideal pose, based on the overlapping image content between adjacent target images, the target images in each ideal pose can be stitched together to generate a panoramic image corresponding to the target object. Thus, it is possible to complete the shooting of the panoramic image only by using a handheld shooting device without additionally setting up shooting auxiliary facilities such as a turntable. In a specific implementation, the user can simply hold a mobile phone and shoot around the target object to obtain the panoramic image.

[0095] In an embodiment of the present invention, in response to an operation of a shooting device for surrounding shooting of a target object, a plurality of object images in different acquisition directions corresponding to the target object are obtained, where there is partial overlapping image content between every two adjacent object images; based on the image feature points in the object images, the device pose corresponding to the object images is identified; based on the device pose and the position of the target object in the object images, the ideal pose corresponding to each acquisition direction is determined; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image; based on the ideal pose, the object images with a device pose close to the ideal pose direction are adjusted to obtain a target image in the ideal pose; a panoramic image corresponding to the target object is generated using the target image in the ideal pose. Thus, it is possible to obtain a panoramic image with high quality by using a handheld shooting device without shooting auxiliary facilities such as a turntable.

[0096] Refer to Figure 4 , which shows a flowchart of the steps of a method for generating a panoramic image provided in an embodiment of the present invention, and specifically may include the following steps:

[0097] Step 401: In response to an operation of a shooting device for surrounding shooting of a target object, the lateral deviation angle of the shooting device is detected in real time, and the lateral deviation angle is the angle of rotation around an axis perpendicular to the ground;

[0098] Specifically, when the user needs to obtain a panoramic image, the shooting device can be used to surround and shoot the target object. The shooting device can be a device with a camera such as a camera, a mobile phone, a tablet computer, a drone, etc., and the present invention does not limit this.

[0099] During the process of the photographing device moving around the target object, the photographing device itself or an electronic device communicatively connected to the photographing device can detect the lateral deviation angle of the photographing device in real time. The lateral deviation angle can refer to the angle of rotation around the axis perpendicular to the ground. During the circular object photographing process, the photographing device moves around the target object while the camera always aims at the target object, so that the movement degree of the photographing device and the acquisition direction of the photographing device can be determined according to the lateral deviation angle.

[0100] Step 402, whenever the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, obtain the object image collected by the photographing device, and obtain a plurality of object images corresponding to the target object with different acquisition directions, where there is partial overlapping image content between every two adjacent object images;

[0101] During the circular object photographing process, the user can control the photographing device to move around the object in the clockwise or counterclockwise direction. Thus, whenever the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, it can be considered that the photographing device is at a new acquisition direction at this time. The photographing device itself or an electronic device communicatively connected to the photographing device can control the photographing device to perform image acquisition in this acquisition direction and obtain the object image in this acquisition direction. Among them, the preset angle value can be determined according to actual needs. The smaller the preset angle value, the more object images are obtained, and the final circular object image can have better quality, but the computational processing amount is larger. While the larger the preset angle value, the fewer object images are obtained, the computational processing amount can be reduced, but the quality of the circular object image may be reduced to a certain extent. Generally speaking, the preset angle value can be set to 5°, 6°, 10°, 12°, etc., and the present invention does not limit this.

[0102] After the photographing device moves around the target object for one week, a plurality of object images corresponding to the target object with different acquisition directions can be obtained. There can be partial overlapping image content between every two adjacent object images for subsequent further image processing.

[0103] Step 403, based on the image feature points in the object image, identify the device pose corresponding to the object image;

[0104] After obtaining the object image, the device pose of the photographing device when each object image is taken can be determined based on the object image. This device pose can be used as the position and pose of the object image in the three-dimensional space, so that the object image can be adjusted based on the device pose subsequently. Among them, the device pose can include the position and the pose of the device, and it can usually be represented by a rotation matrix R and a translation matrix t.

[0105] Specifically, there may be image feature points in the object image. The image feature points may have more information content compared to those in a general object image, and will not change due to factors such as illumination, noise, and perspective change, having scale invariance and rotation invariance. For example, the image feature points may be the corner positions of the object in the object image, the edge positions of the object, the bright points in the dark areas, the dark points in the bright areas, etc. The relative pose between adjacent object images can be determined based on the position change of the image feature points in the adjacent object images. Subsequently, based on the relative pose between adjacent object images, the pose change of the imaging device corresponding to all object images can be determined, and finally the device pose corresponding to each object image can be determined.

[0106] In an embodiment of the present invention, the step of identifying the device pose corresponding to the object image based on the image feature points in the object image includes:

[0107] S11, according to the partially overlapping image content existing between every two adjacent object images, identify the common image feature points between the adjacent object images to form an image matching pair;

[0108] Specifically, there may be image feature points in the object image. The image feature points may have more information content compared to those in a general object image, and will not change due to factors such as illumination, noise, and perspective change, having scale invariance and rotation invariance. For example, the image feature points may be the corner positions of the object in the object image, the edge positions of the object, the bright points in the dark areas, the dark points in the bright areas, etc.

[0109] The image feature points in the object image actually point to a certain three-dimensional point in three-dimensional space. Since there may be overlapping image content between every two adjacent object images, there may be image feature points in the adjacent object images that all point to the same three-dimensional point. It can be considered that the image feature points in the adjacent object images that point to the same three-dimensional point are the same. At this time, there are common image feature points between the adjacent object images. The common image feature points that may exist between the adjacent object images can be marked as an image matching pair.

[0110] S12, based on the image matching pair, construct the fundamental matrix between the adjacent object images;

[0111] The fundamental matrix is the fundamental constraint between corresponding points in two images. It can reflect the intrinsic projective geometry of the epipolar geometry. The normalized eight-point algorithm can be used to construct the fundamental matrix F between the adjacent object images based on the image matching pair.

[0112] Specifically, the fundamental matrix F can also be expressed as the mapping of the image point p1 in one image to the epipolar line l2 in another image. Since the two feature points in the image matching pair lie on the same epipolar line, the fundamental matrix F between adjacent object images can be calculated based on the image matching pair. As there are 9 elements in the fundamental matrix F, and except for a constant factor, the remaining 8 elements are unknown. Therefore, 8 pairs of image matching pairs can be used to calculate the fundamental matrix F between adjacent object images.

[0113] S13. Determine the device pose of the object image based on the fundamental matrix.

[0114] After calculating the fundamental matrix, the essential matrix E can be further calculated based on the fundamental matrix and the internal parameter K of the imaging device. The relationship between the fundamental matrix F and the essential matrix E can be expressed as E = K' T FK. Here, K is the internal parameter of the imaging device. The internal parameter of the imaging device can be obtained based on the camera attributes such as focal length, camera center, etc. recorded in the object image. After obtaining the essential matrix E, the essential matrix E can be subjected to singular value decomposition (SVD), and during the decomposition process, ensure that the determinant value of the rotation matrix R is positive, that is, the rotation matrix R and the translation matrix t can be calculated, thereby obtaining the device pose of the object image.

[0115] Step 404. Determine the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image.

[0116] Specifically, in order to obtain a high-quality panoramic image, generally, it is necessary to keep a fixed distance between the imaging device and the target object, a fixed height from the ground, and maintain a fixed pose, and photograph the target object from multiple directions. Thus, several ideal poses can be set horizontally around the target object with the target object as the center point. If the imaging device photographs the target object in the ideal pose, since the imaging device is at a fixed distance from the target object, a fixed height from the ground, and maintains a fixed pose at this time, better object images can be obtained, and further high-quality panoramic images can be generated.

[0117] Among them, the center point of the target object is determined based on the device pose of each object image and the position of the target object in each object image. Specifically, based on the device pose corresponding to the object image, the position and orientation of the object image in the three-dimensional space can be known. At the same time, the target detection method can be used to detect the position of the target object in the object image, and a target detection frame is used to frame the target object on the object image. Subsequently, the focus of the camera is connected to the corner points of the target detection frame and extended towards the target object, and a visual cone pointing to the target object can be formed. Each object image can form a visual cone, so that in the three-dimensional space, the visual cones corresponding to each object image can overlap with each other to form a common area. Since the visual cone is generated based on the position of the target object in the object image, this common area can be considered as the area where the target object is located in the three-dimensional space, and the center point of this area is the center point of the target object.

[0118] Optionally, since in the process of determining the device pose of the shooting device, a coordinate system of a three-dimensional space can be determined based on the device pose of the object image at the same time. However, due to the shaking of the shooting device during the panoramic shooting, the device pose is not always perpendicular to the ground, resulting in the coordinate system of the determined three-dimensional space may not be perpendicular to the ground, which may lead to complex calculations in the subsequent construction of the ideal pose, or an ideal pose that cannot horizontally surround the target object cannot be constructed. Therefore, during the panoramic shooting, the gravity direction information of the shooting device can be obtained at the same time, and one of the coordinate axes in the coordinate system of the three-dimensional space can be adjusted to be the same as the gravity direction, so as to better construct the ideal pose subsequently.

[0119] In an embodiment of the present invention, the step of determining the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image includes:

[0120] S21, select an object image, and use the spatial position corresponding to the object image as the initial position;

[0121] In order to regenerate the ideal pose, an object image can be selected, and the spatial position corresponding to the object image is used as the initial position in the process of generating the ideal pose.

[0122] In a specific implementation, generally speaking, when a user performs panoramic shooting, a location with an appropriate distance from the target object is usually selected as the starting point, and the target object is placed at the center of the viewfinder to start shooting. Therefore, the first object image taken at the starting point usually has a relatively appropriate distance from the target object and is at a relatively appropriate height from the ground. Thus, the spatial position corresponding to the first object image can be used as the initial position in the process of generating the ideal pose.

[0123] Optionally, a ring surrounding the target object can also be determined based on the average distance between the object image and the center point of the target object, and the average height of the object image from the ground. Subsequently, any point on the ring surrounding the target object is selected as the initial position of the ideal pose.

[0124] S22, starting from the initial position, generate a preset number of ideal poses by horizontally surrounding the center point.

[0125] After determining the initial position, starting from the initial position, a preset number of ideal poses can be generated by horizontally surrounding the center point. Among them, the ideal poses can be evenly distributed in each acquisition direction around the target object. The preset number can be determined as 50, 60, 100, etc. according to actual needs, and the present invention does not limit this. To ensure that ring object images of better quality can be generated, the number of ideal poses can be less than the number of object images, so that subsequently, object images with device poses closer to the ideal poses can be preferably selected for adjustment.

[0126] In a specific implementation, after determining the initial position, the target distance between the initial position and the target object, and the target height from the ground can be obtained. Subsequently, based on the number of ideal poses to be determined, the horizontal rotation angle between adjacent ideal poses relative to the center point of the target object can be calculated. For example, in the case where 50 ideal poses need to be generated, the horizontal rotation angle between adjacent ideal poses relative to the center point of the target object can be 7.2°. Subsequently, starting from the initial position, horizontally surround the target object clockwise or counterclockwise. Every time the horizontal rotation angle increases by 7.2°, this position is determined as the position where the ideal pose is located. Thus, the ideal poses can be evenly distributed on a ring formed by horizontally surrounding the target object.

[0127] Step 405, based on the ideal poses, adjust the object images with device poses similar to the ideal poses to obtain target images in the ideal poses;

[0128] After determining the ideal poses, the object images can be adjusted based on the device poses and the ideal poses to obtain target images in the ideal poses, so that subsequently, based on the target images in the ideal poses, ring object images of higher quality can be obtained.

[0129] In an embodiment of the present invention, the step of adjusting the object images with device poses similar to the ideal poses based on the ideal poses to obtain target images in the ideal poses includes:

[0130] S31, for an ideal pose, use the object image with a device pose similar to the ideal pose as the image to be processed matching the ideal pose;

[0131] Specifically, for an ideal pose, an object image with a device pose close to the ideal pose direction can be searched for, and the object image can be used as the image to be processed that matches the ideal pose. If there are multiple object images close to the ideal pose, the object image with the device position closest to the ideal pose can be used as the image to be processed.

[0132] S32. Obtain the first rotation matrix and the first position information recorded in the ideal pose, and obtain the internal parameter matrix of the device corresponding to the image to be processed, the second rotation matrix and the second position information recorded in the device pose;

[0133] After determining the image to be processed, the object image can be adjusted based on the device pose and the ideal pose. Specifically, a homography transformation can be performed on the image based on the ideal pose and the device pose.

[0134] Thus, the first rotation matrix and the first position information recorded in the ideal pose can be obtained, and the internal parameter matrix of the device corresponding to the image to be processed, the second rotation matrix and the second position information recorded in the device pose can be obtained.

[0135] Specifically, the ideal pose and the device pose can be expressed by the rotation matrix R and the translation matrix t. Thus, the rotation matrix R of the ideal pose is the first rotation matrix, and based on the translation matrix t of the ideal pose, the distance from the ideal pose to the center point of the target object can be determined and used as the first position information. The rotation matrix R of the device pose corresponding to the image to be processed is the second rotation matrix, and the translation matrix t of the device pose can determine the distance from the device pose to the center point of the target object and use it as the second position information. The internal parameter matrix of the object image can be obtained based on the camera attributes such as focal length and camera center recorded in the object image. Specifically, the object image usually has EXIF (Exchangeable Image File format) information, which records information such as focal length and camera center.

[0136] S33. Determine the first transformation matrix based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix;

[0137] Specifically, in order to perform a homography transformation on the image, it is necessary to determine how the image should be rotated and how to scale it. Thus, the first transformation matrix can be determined first based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix to determine how to rotate the image.

[0138] Specifically, the first transformation matrix can be calculated using the following formula:

[0139] H = k @ R ideal @ inv(R origin)@inv(k)

[0140] Among them, k represents the internal parameter matrix of the imaging device, inv0 represents finding the inverse matrix of the matrix, and R origin represents the second rotation matrix, and R ideal represents the first rotation matrix.

[0141] S34. Based on the first position information and the second position information, determine the second transformation matrix;

[0142] After that, based on the first position information and the second position information, the second transformation matrix can be determined. The second transformation matrix can be calculated using the following formula:

[0143]

[0144]

[0145] Among them, d_origin represents the second position information, and d_ideal represents the first position information.

[0146] S35. Use the first transformation matrix and the second transformation matrix to adjust the object image to obtain the target image in the ideal pose.

[0147] After determining the first transformation matrix and the second transformation matrix, the first transformation matrix and the second transformation matrix can be used to perform a homography transformation on the object image, and the target image in the ideal pose can be calculated.

[0148] Specifically, multiplying the first transformation matrix by the second transformation matrix gives the homography matrix. Using the homography matrix to adjust the object image can obtain the target image in the ideal pose.

[0149] Step 406. Use the target image in the ideal pose to generate a panoramic image corresponding to the target object.

[0150] After obtaining the target image in the ideal pose, based on the overlapping image content between adjacent target images, each target image in the ideal pose can be stitched together to generate a panoramic image corresponding to the target object.

[0151] In an embodiment of the present invention, the step of using the target image in the ideal pose to generate a panoramic image corresponding to the target object includes:

[0152] S41. Identify the area where the target object is located in each target image;

[0153] Specifically, as Figure 5 shown, Figure 5The embodiment of the present invention provides a schematic diagram of a target image. After transforming an object image into a target image in an ideal pose, since the image content of a part of the region does not appear in the object image, a part of the black background may appear in the target image. If the target image is directly used to generate a panoramic image, it may lead to poor panoramic image effects. To avoid the adverse effects of the black background on the panoramic image and avoid the adverse effects of different sizes of target objects that may exist in the target image and other information that does not need to be concerned about on the generation effect of the panoramic image, the target image can be further processed. Thus, the region where the target object is located in each target image can be identified first by using target detection.

[0154] S42, Extract the image of the region where the target object is located in each target image to obtain a target object image;

[0155] After determining the region where the target object is located in each target image, a target detection box can be used to frame the target object in the target image. The target image can be further cropped based on the target detection box to extract a target object image mainly containing the target object, so as to exclude the information in the image that does not need to be concerned about.

[0156] Optionally, since the target object can usually be at the center of the processed target image, after determining the target detection box, the target detection box can also be adjusted so that it is symmetric up and down and left and right around the center position of the target image, and then the target object image is extracted.

[0157] S43, Adjust the target object image so that the sizes of the target objects in the target object image are unified;

[0158] To avoid different sizes of target objects in the target object image, the target object image can also be adjusted according to actual needs so that the sizes of the target objects in the target object image are unified. For example, the target object image can be enlarged or reduced according to actual needs.

[0159] S44, Use the target object image to generate a panoramic image corresponding to the target object.

[0160] After the processing of the target object image is completed, based on the overlapping image content between adjacent target object images, each target object image in the ideal pose can be spliced to generate a panoramic image corresponding to the target object.

[0161] In an embodiment of the present invention, the method further includes:

[0162] S51. Identify at least one area to be processed in the panoramic image and perform special effect processing on the area to be processed; wherein, the special effect processing includes at least one of scene replacement, information addition, and information deletion.

[0163] Specifically, after generating the panoramic image, further special effect processing can be performed on the panoramic image to improve the display effect of the panoramic image.

[0164] If scene replacement processing needs to be performed on the panoramic image, the target object in the panoramic image can be identified in the panoramic image, and the area outside the target object can be used as the area to be processed, and the area to be processed can be replaced with a preset background, such as a studio picture, a landscape picture, etc.

[0165] If information addition processing needs to be performed on the panoramic image, such as adding advertising information, adding descriptive information associated with the target object, etc., a suitable area in the panoramic image can be selected as the area to be processed, and the information can be added to the area to be processed accordingly.

[0166] If information deletion processing needs to be performed on the panoramic image, such as deleting personal privacy information such as license plate information, the area where the information to be deleted is located can be identified in the panoramic image, and the original information can be covered with other text or images so that the information to be deleted does not appear in the panoramic image.

[0167] Optionally, according to actual needs, resolution improvement processing, filter addition, brightness adjustment, resolution adjustment, etc. can also be performed on the panoramic image, and the present invention does not limit this.

[0168] In an embodiment of the present invention, in response to an operation of a shooting device for surrounding shooting of a target object, the lateral deviation angle of the shooting device is detected in real time, where the lateral deviation angle is an angle of rotation about an axis perpendicular to the ground; whenever the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, an object image collected by the shooting device is acquired, and a plurality of object images corresponding to the target object in different acquisition directions are obtained, where there is partial overlapping image content between every two adjacent object images; based on the image feature points in the object images, the device pose corresponding to the object images is identified; based on the device pose and the position of the target object in the object images, the ideal pose corresponding to each acquisition direction is determined; where the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image; based on the ideal pose, the object images with a device pose close to the ideal pose are adjusted to obtain target images in the ideal pose; the target images in the ideal pose are used to generate a panoramic image corresponding to the target object. Thus, without the need for shooting auxiliary facilities such as a turntable, by using a hand-held shooting device, a panoramic image of high quality can be obtained.

[0169] It should be noted that for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0170] Referring to Figure 6 , a structural block diagram of a panoramic image generation device provided in an embodiment of the present invention is shown, which may specifically include the following modules:

[0171] An image acquisition module 601, configured to acquire a plurality of object images corresponding to the target object in different acquisition directions in response to an operation of a shooting device for surrounding shooting of the target object;

[0172] A pose recognition module 602, configured to identify the device pose corresponding to the object images based on the image feature points in the object images;

[0173] An ideal pose generation module 603, configured to determine the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object images; where the ideal pose horizontally surrounds the target object with the target object as the center point;

[0174] An image adjustment module 604, configured to adjust an object image whose device pose is similar to the ideal pose based on the ideal pose, so as to obtain a target image in the ideal pose;

[0175] A panoramic image generation module 605, configured to generate a panoramic image corresponding to the target object by using the target image in the ideal pose.

[0176] Optionally, the image acquisition module includes:

[0177] An angle detection sub-module, configured to respond to an operation of the photographing device to perform a surrounding photographing on a target object, and to detect in real time a lateral deviation angle of the photographing device, where the lateral deviation angle is an angle of rotation about an axis perpendicular to the ground;

[0178] An image acquisition sub-module, configured to acquire an object image collected by the photographing device every time the lateral deviation angle increases by a preset angle value in the clockwise direction or the counterclockwise direction, so as to obtain a plurality of object images in different acquisition directions corresponding to the target object.

[0179] Optionally, the pose recognition module includes:

[0180] An image matching sub-module, configured to identify common image feature points between adjacent object images according to partially overlapping image content existing between every two adjacent object images, so as to form an image matching pair;

[0181] An essential matrix construction sub-module, configured to construct an essential matrix between adjacent object images based on the image matching pair;

[0182] A pose recognition sub-module, configured to determine the device pose of the object image based on the essential matrix.

[0183] Optionally, the ideal pose generation module includes:

[0184] An initial position selection module, configured to select one of the object images, and use the spatial position corresponding to the object image as the initial position;

[0185] An ideal position generation sub-module, configured to horizontally surround the center point from the initial position to generate a preset number of ideal poses.

[0186] Optionally, the image adjustment module includes:

[0187] An image selection sub-module, configured to, for one of the ideal poses, use an object image whose device pose is similar to the ideal pose as a to-be-processed image matching the ideal pose;

[0188] An information acquisition sub-module, configured to acquire the first rotation matrix and the first position information recorded in the ideal pose, and acquire the device internal parameter matrix, the second rotation matrix, and the second position information recorded in the device pose corresponding to the to-be-processed image;

[0189] A first transformation matrix determination sub-module, configured to determine a first transformation matrix based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix;

[0190] A second transformation matrix determination sub-module, configured to determine a second transformation matrix based on the first position information and the second position information;

[0191] An image adjustment sub-module, configured to adjust the object image by using the first transformation matrix and the second transformation matrix to obtain a target image in the ideal pose.

[0192] Optionally, the panoramic image generation module includes:

[0193] A region recognition module, configured to recognize the region where the target object is located in each of the target images;

[0194] An image extraction sub-module, configured to extract the image of the region where the target object is located in each of the target images to obtain a target object image;

[0195] A size unification sub-module, configured to adjust the target object image so that the sizes of the target objects in the target object image are unified;

[0196] A panoramic image generation sub-module, configured to generate a panoramic image corresponding to the target object by using the target object image.

[0197] Optionally, the device further includes:

[0198] An effect processing module, configured to recognize at least one to-be-processed region in the panoramic image and perform effect processing on the to-be-processed region; wherein, the effect processing includes at least one of scene replacement, information addition, and information deletion.

[0199] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, please refer to the partial description of the method embodiment.

[0200] In addition, an embodiment of the present invention further provides an electronic device, as Figure 7 shown, including a processor 701, a communication interface 702, a memory 703, and a communication bus 704. Among them, the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704,

[0201] A memory 703 for storing computer programs;

[0202] A processor 701, when executing the program stored on the memory 703, implements the following steps:

[0203] In response to an operation of a photographing device to perform a circumferential photographing of a target object, obtain a plurality of object images corresponding to the target object in different acquisition directions, where there is partial overlapping image content between every two adjacent object images;

[0204] Based on the image feature points in the object images, identify the device pose corresponding to the object images;

[0205] Based on the device pose and the position of the target object in the object images, determine the ideal pose corresponding to each acquisition direction; wherein, the ideal pose horizontally circumscribes the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image;

[0206] Based on the ideal pose, adjust the object images whose device poses are close to the ideal pose direction to obtain target images in the ideal pose;

[0207] Generate a panoramic object image corresponding to the target object by using the target images in the ideal pose.

[0208] Optionally, the step of, in response to an operation of a photographing device to perform a circumferential photographing of a target object, obtaining a plurality of object images corresponding to the target object in different acquisition directions includes:

[0209] In response to an operation of a photographing device to perform a circumferential photographing of a target object, real-time detect the lateral deviation angle of the photographing device, where the lateral deviation angle is the angle of rotation around an axis perpendicular to the ground;

[0210] Whenever the lateral deviation angle increases by a preset angle value in the clockwise direction or the counterclockwise direction, obtain the object image collected by the photographing device to obtain a plurality of object images corresponding to the target object in different acquisition directions.

[0211] Optionally, the step of, based on the image feature points in the object images, identifying the device pose corresponding to the object images includes:

[0212] According to the partial overlapping image content existing between every two adjacent object images, identify the common image feature points between the adjacent object images to form image matching pairs;

[0213] Based on the image matching pairs, construct an essential matrix between adjacent object images;

[0214] Based on the basic matrix, determine the device pose of the object image.

[0215] Optionally, the step of determining the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image includes:

[0216] Select one of the object images, and use the spatial position corresponding to the object image as the initial position;

[0217] Starting from the initial position, horizontally surround the center point to generate a preset number of ideal poses.

[0218] Optionally, the step of adjusting the object image with a device pose similar to the ideal pose based on the ideal pose to obtain a target image in the ideal pose includes:

[0219] For one of the ideal poses, use the object image with a device pose similar to the ideal pose as the image to be processed that matches the ideal pose;

[0220] Obtain the first rotation matrix and the first position information recorded in the ideal pose, and obtain the device internal parameter matrix, the second rotation matrix, and the second position information recorded in the device pose corresponding to the image to be processed;

[0221] Based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix, determine the first transformation matrix;

[0222] Based on the first position information and the second position information, determine the second transformation matrix;

[0223] Use the first transformation matrix and the second transformation matrix to adjust the object image to obtain a target image in the ideal pose.

[0224] Optionally, the step of generating a panoramic image corresponding to the target object using the target image in the ideal pose includes:

[0225] Identify the area where the target object is located in each of the target images;

[0226] Extract the image of the area where the target object is located in each of the target images to obtain a target object image;

[0227] Adjust the target object image so that the sizes of the target objects in the target object images are unified;

[0228] Use the target object images to generate a panoramic image corresponding to the target object.

[0229] Optionally, the method further includes:

[0230] identifying at least one area to be processed in the annular object image, and performing special effect processing on the area to be processed; wherein the special effect processing includes at least one of scene replacement, information addition, and information deletion.

[0231] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0232] The communication interface is used for communication between the above terminal and other devices.

[0233] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0234] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0235] As Figure 8 shown, in another embodiment provided by the present invention, a computer-readable storage medium 801 is further provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the method for generating an annular object image described in the above embodiment.

[0236] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions, which, when running on a computer, causes the computer to execute the method for generating a ring object image described in the above embodiments.

[0237] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0238] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0239] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the corresponding part of the method embodiment for the relevant content.

[0240] The above description is only for the preferred embodiments of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A method for generating a panoramic image of an object, characterized in that, it includes: In response to an operation of a photographing device for surrounding photographing of a target object, obtaining a plurality of object images corresponding to the target object in different acquisition directions, wherein there is partial overlapping image content between every two adjacent object images; Based on the image feature points in the object images, identifying the device pose corresponding to the object images; Based on the device pose and the position of the target object in the object images, determining the ideal pose corresponding to each acquisition direction; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point, and the center point is determined based on the device pose of each object image and the position of the target object in each object image; Based on the ideal pose, adjusting the object images with device poses similar to the ideal pose direction to obtain target images in the ideal pose; Generating a panoramic image corresponding to the target object by using the target images in the ideal pose; Wherein, the step of adjusting the object images with device poses similar to the ideal pose direction based on the ideal pose to obtain target images in the ideal pose includes: For one ideal pose, taking the object image with a device pose similar to the ideal pose direction as the to-be-processed image matching the ideal pose; Obtaining the first rotation matrix and the first position information recorded in the ideal pose, and obtaining the device internal parameter matrix, the second rotation matrix, and the second position information recorded in the device pose corresponding to the to-be-processed image; Based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix, determining the first transformation matrix; Based on the first position information and the second position information, determining the second transformation matrix; Using the first transformation matrix and the second transformation matrix to adjust the object image to obtain a target image in the ideal pose; Wherein, the step of generating a panoramic image corresponding to the target object by using the target images in the ideal pose includes: Identifying the area where the target object is located in each target image; Extracting the image of the area where the target object is located in each target image to obtain a target object image; Adjusting the target object image so that the size of the target object in the target object image is unified; Using the target object image to generate a panoramic image corresponding to the target object; Wherein, the method further includes: Identifying at least one area to be processed in the panoramic image and performing special effect processing on the area to be processed; wherein, the special effect processing includes at least one of scene replacement, information addition, and information deletion.

2. The method according to claim 1, characterized in that, the step of, in response to an operation of a photographing device for surrounding photographing of a target object, obtaining a plurality of object images corresponding to the target object in different acquisition directions includes: In response to an operation of a photographing device for surrounding photographing of a target object, real-time detecting the lateral deviation angle of the photographing device, and the lateral deviation angle is the angle of rotation around an axis perpendicular to the ground; Whenever the lateral deviation angle increases by a preset angle value in the clockwise or counterclockwise direction, obtain the object images collected by the imaging device, and obtain a plurality of object images corresponding to the target object in different acquisition directions.

3. The method according to claim 1, wherein, the step of identifying the device pose corresponding to the object image based on the image feature points in the object image includes: Identifying the common image feature points between adjacent object images according to the partially overlapping image content existing between every two adjacent object images, and forming image matching pairs; Based on the image matching pairs, constructing an essential matrix between adjacent object images; Based on the essential matrix, determining the device pose of the object image.

4. The method according to claim 1, wherein, the step of determining the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image includes: Selecting one of the object images and taking the spatial position corresponding to the object image as the initial position; Starting from the initial position, generating a preset number of ideal poses by horizontally surrounding the center point.

5. A device for generating a panoramic image of an object, wherein, it includes: An image acquisition module, configured to obtain a plurality of object images corresponding to the target object in different acquisition directions in response to an operation of the imaging device for surrounding shooting of the target object; A pose recognition module, configured to recognize the device pose corresponding to the object image based on the image feature points in the object image; An ideal pose generation module, configured to determine the ideal pose corresponding to each acquisition direction based on the device pose and the position of the target object in the object image; wherein, the ideal pose horizontally surrounds the target object with the target object as the center point; An image adjustment module, configured to adjust the object image with a device pose similar to the ideal pose based on the ideal pose to obtain a target image in the ideal pose; A panoramic image generation module, configured to generate a panoramic image corresponding to the target object by using the target image in the ideal pose; wherein, the image adjustment module includes: An image selection sub-module, configured to, for one ideal pose, use the object image with a device pose similar to the ideal pose as the to-be-processed image matching the ideal pose; An information acquisition sub-module, configured to obtain the first rotation matrix and the first position information recorded in the ideal pose, and obtain the device internal parameter matrix, the second rotation matrix, and the second position information recorded in the device pose corresponding to the to-be-processed image; A first transformation matrix determination sub-module, configured to determine a first transformation matrix based on the first rotation matrix, the second rotation matrix, and the device internal parameter matrix; A second transformation matrix determination sub-module, configured to determine a second transformation matrix based on the first position information and the second position information; An image adjustment sub-module, configured to adjust the object image by using the first transformation matrix and the second transformation matrix to obtain a target image in the ideal pose; wherein, the panoramic image generation module includes: An area recognition module for recognizing the area where the target object is located in each of the target images; An image extraction sub-module for extracting the image of the area where the target object is located in each of the target images to obtain a target object image; A size unification sub-module for adjusting the target object image so that the sizes of the target objects in the target object image are unified; A panoramic object image generation sub-module for generating a panoramic object image corresponding to the target object by using the target object image; Wherein, the device further includes: A special effect processing module for recognizing at least one area to be processed in the panoramic object image and performing special effect processing on the area to be processed; wherein, the special effect processing includes at least one of scene replacement, information addition, and information deletion.

6. An electronic device, characterized in that, it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; the memory is used for storing a computer program; when the processor executes the program stored on the memory, it implements the method according to any one of claims 1-4.

7. A computer-readable medium, on which instructions are stored, and when executed by one or more processors, cause the processors to execute the method according to any one of claims 1-4.

Images