Panoramic image processing method and device based on train transformation room, equipment and medium

Abstract

The invention discloses a panoramic image processing method and device based on a train transformation room, equipment and a medium, and the method comprises the steps: obtaining a first panoramic image and a second panoramic image of the train transformation room, carrying out the image registration through employing an APAP algorithm, and obtaining a to-be-spliced panoramic image after registration; projecting the panoramic image to be spliced to a central cylindrical surface by using a cylindrical orthographic projection algorithm; calling a space-time filter, and splicing the panoramic image to be spliced to obtain a spliced panoramic image; performing edge elimination on the spliced panoramic image to obtain a target panoramic image; and importing the target panoramic image into the space editor, and displaying the target panoramic image in the virtual panoramic space. Therefore, through the APAP algorithm, panoramic image registration can be quickly realized, the accuracy is relatively high, and the target panoramic image is displayed by using the virtual panoramic space constructed by the space editor, so that the train transformation room is displayed in the virtual panoramic space, and the virtual room roaming of a user is realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a panoramic image processing method, device, equipment and medium based on a train conversion room. Background Art

[0002] The train renovation market has broad prospects and will usher in huge development opportunities in the next few years. As people's requirements for travel quality become higher and higher, the train decoration market will receive more and more attention and favor. At the same time, train renovation houses have also brought better development to the tourism industry and provided users with more convenient and thoughtful services for travel. Train renovation houses refer to the renovation of abandoned train carriages to make them living or office spaces.

[0003] In order to ensure the quality of travel, when customizing a room, users need to check the configuration and facilities of the train-converted room, so the display of the train-converted room configuration is particularly important. However, based on the existing technology alone, users cannot quickly and intuitively obtain the configuration of the train-converted room. Therefore, a method is urgently needed to quickly obtain the configuration of the train-converted room and have a good display effect. Summary of the invention

[0004] The present application provides a panoramic image processing method, device, equipment and medium based on a train conversion room, which can quickly obtain the configuration of the train conversion room and intuitively display it to the user, thereby improving the user experience.

[0005] In a first aspect, the present application provides a panoramic image processing method based on a train conversion room, the method comprising:

[0006] Acquire a panoramic image sequence of the train renovation room, wherein the panoramic image sequence includes multiple frames of panoramic images;

[0007] Sequentially acquiring a first panoramic image and a second panoramic image which are adjacent frames to each other in the panoramic image sequence; wherein the first panoramic image and the second panoramic image have an overlapping area;

[0008] Performing image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain a registered panoramic image to be stitched;

[0009] Using a cylindrical projection algorithm, projecting the panoramic image to be stitched onto a central cylindrical surface;

[0010] Calling a spatiotemporal filter to stitch the panoramic images to be stitched, to obtain a stitched panoramic image;

[0011] Eliminating the edges of the panoramic image to be stitched according to the panoramic image to be stitched, to obtain a target panoramic image;

[0012] The target panoramic image is imported into a space editor, and the target panoramic image is displayed in a virtual panoramic space.

[0013] Optionally, the method further comprises:

[0014] Back-projecting the target panoramic image in the virtual panoramic space into a central projection image through a back-projection algorithm of a cylindrical panoramic image;

[0015] The central projection image is displayed on a computer device so as to adjust the target panoramic image displayed in the virtual panoramic space based on the central projection image.

[0016] Optionally, the performing image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain the registered panoramic image to be stitched includes:

[0017] Extracting feature points in the first panoramic image and feature points in the second panoramic image using a scale-invariant feature transform (SIFT) algorithm;

[0018] Matching feature points in the first panoramic image with feature points in the second panoramic image to obtain initial feature point pairs;

[0019] The initial feature point pairs are screened using a random sample consistent RANSAC algorithm to obtain a screened target feature point pair; the target feature point pair represents a correspondence between a first feature point in the first panoramic image and a second feature point in the second panoramic image;

[0020] A corresponding homography matrix is ​​acquired according to the target feature points, and the first panoramic image and the second panoramic image are registered based on the homography matrix to obtain the registered panoramic image to be stitched.

[0021] Optionally, the random sample consistent RANSAC algorithm is used to screen the initial feature point pairs to obtain the screened target feature point pairs, including:

[0022] Randomly select a set of samples from the initial feature point pairs;

[0023] Obtaining a mathematical model by fitting the sample set, wherein the mathematical model includes a homography matrix mathematical model or an affine transformation matrix mathematical model;

[0024] Eliminating the sample set from the initial feature point pairs to obtain remaining feature point pairs;

[0025] Substituting the remaining feature point pairs into the fitted mathematical model, calculating the errors between the remaining feature points and the mathematical model, and obtaining error results;

[0026] If the error result is less than a preset error threshold, it is determined to be an inlier point, and the number of inliers among the remaining feature points is counted;

[0027] Calculating the score of the mathematical model according to the ratio of the number of the inner points to the total number of the feature points of the initial feature point pair;

[0028] Returning to execute the step of randomly selecting a set of sample sets from the initial feature point pairs until a preset number of iterations is reached;

[0029] The mathematical model with the highest score was selected as the initial screening model;

[0030] Recalculating and setting the parameters of the initial screening model based on all the internal points in the initial feature point pair to obtain a target screening model;

[0031] Screening out the inner points matching the target screening model as the screened target feature point pairs;

[0032] Output the filtered target feature point pairs.

[0033] Optionally, acquiring a corresponding homography matrix according to the target feature points, registering the first panoramic image and the second panoramic image based on the homography matrix, and obtaining the registered panoramic image to be stitched includes:

[0034] If the number of the target feature points is greater than a preset number threshold and is evenly distributed, a direct linear transformation DLT algorithm is used to estimate a transparent transformation matrix for the target feature point pair to solve a global homography matrix for the target feature point pair;

[0035] Based on the global homography matrix, image registration is performed on the first panoramic image and the second panoramic image to obtain the registered panoramic image to be stitched.

[0036] Optionally, acquiring a corresponding homography matrix according to the target feature points, registering the first panoramic image and the second panoramic image based on the homography matrix, and obtaining the registered panoramic image to be stitched includes:

[0037] If the number of the target feature points is not greater than a preset number threshold and the distribution is uneven, the first panoramic image and the second panoramic image are correspondingly divided into a plurality of grids; each of the plurality of grids represents a local area of ​​the first panoramic image or the second panoramic image;

[0038] Obtaining a grid having the target feature point pair from the plurality of grids, and calculating a local homography matrix of the grid having the target feature point pair;

[0039] Based on the local homography matrix of the grid where the target feature point pair exists, the first global image and the second panoramic image are registered to obtain the registered panoramic image.

[0040] Optionally, calling a spatiotemporal filter to stitch the images to be stitched to obtain a stitched panoramic image includes:

[0041] Obtaining a first overlapping area of ​​the panoramic image to be stitched;

[0042] Determining a transformation relationship between the panoramic images to be stitched according to the first overlapping area;

[0043] Selecting a reference image and a non-reference image from the first panoramic image and the second panoramic image;

[0044] Calling a spatiotemporal filter to transform the non-reference image into an image coordinate system of the reference image according to a transformation parameter in the transformation relationship;

[0045] The images to be stitched are stitched together to obtain the stitched panoramic image.

[0046] Optionally, performing edge elimination on the stitched image according to the panoramic image to be stitched to obtain a target panoramic image includes:

[0047] Obtaining a second overlapping area between the panoramic image to be stitched and the stitched panoramic image;

[0048] Performing Gaussian pyramid decomposition on the second overlapping area to obtain a Gaussian pyramid decomposition result;

[0049] Constructing a Laplacian pyramid according to the Gaussian pyramid decomposition result;

[0050] At each level of the Laplacian pyramid, edges of the spliced ​​images are fused by weighted fusion to obtain a fused Laplacian pyramid;

[0051] The image is reconstructed layer by layer from the fused Laplacian pyramid until the target panoramic image is obtained.

[0052] In a second aspect, the present application further provides a panoramic image processing device based on a train conversion room, the device comprising:

[0053] An acquisition unit, used for acquiring a panoramic image sequence of the train renovation room, wherein the panoramic image sequence includes a plurality of panoramic image frames;

[0054] The acquisition unit is further used to sequentially acquire a first panoramic image and a second panoramic image which are adjacent frames to each other in the panoramic image sequence; wherein the first panoramic image and the second panoramic image have an overlapping area;

[0055] A registration unit, configured to perform image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain a registered panoramic image to be stitched;

[0056] A projection unit, used for projecting the panoramic image to be stitched onto a central cylindrical surface by using a cylindrical orthographic projection algorithm;

[0057] A stitching unit, used for calling a spatiotemporal filter to stitch the panoramic images to be stitched, so as to obtain a stitched panoramic image;

[0058] An edge elimination unit, configured to eliminate the edges of the stitched panoramic image according to the panoramic image to be stitched, so as to obtain a target panoramic image;

[0059] The display unit is used to import the target panoramic image into the space editor and display the target panoramic image in the virtual panoramic space.

[0060] Optionally, the device further comprises: a back-projection unit;

[0061] The back projection unit is specifically used for:

[0062] Back-projecting the target panoramic image in the virtual panoramic space into a central projection image through a back-projection algorithm of a cylindrical panoramic image;

[0063] The central projection image is displayed on a computer device so as to adjust the target panoramic image displayed in the virtual panoramic space based on the central projection image.

[0064] Optionally, the registration unit comprises:

[0065] A feature point extraction module, used to extract feature points in the first panoramic image and feature points in the second panoramic image by using a scale-invariant feature transform (SIFT) algorithm;

[0066] a matching module, configured to match feature points in the first panoramic image with feature points in the second panoramic image to obtain an initial feature point pair;

[0067] A screening module, used to screen the initial feature point pairs using a random sample consistent RANSAC algorithm to obtain a screened target feature point pair; the target feature point pair represents a correspondence between a first feature point in the first panoramic image and a second feature point in the second panoramic image;

[0068] A registration module is used to obtain a corresponding homography matrix according to the target feature points, and to register the first panoramic image and the second panoramic image based on the homography matrix to obtain the registered panoramic image to be stitched.

[0069] Optionally, the screening module is specifically used to:

[0070] Randomly select a set of samples from the initial feature point pairs;

[0071] Obtaining a mathematical model by fitting the sample set, wherein the mathematical model includes a homography matrix mathematical model or an affine transformation matrix mathematical model;

[0072] Eliminating the sample set from the initial feature point pairs to obtain remaining feature point pairs;

[0073] Substituting the remaining feature point pairs into the fitted mathematical model, calculating the errors between the remaining feature points and the mathematical model, and obtaining error results;

[0074] If the error result is less than a preset error threshold, it is determined to be an inlier point, and the number of inliers among the remaining feature points is counted;

[0075] Calculating the score of the mathematical model according to the ratio of the number of the inner points to the total number of the feature points of the initial feature point pair;

[0076] Returning to execute the step of randomly selecting a set of sample sets from the initial feature point pairs until a preset number of iterations is reached;

[0077] The mathematical model with the highest score was selected as the initial screening model;

[0078] Recalculating and setting the parameters of the initial screening model based on all the internal points in the initial feature point pair to obtain a target screening model;

[0079] Screening out the inner points matching the target screening model as the screened target feature point pairs;

[0080] Output the filtered target feature point pairs.

[0081] Optionally, the registration module is specifically used to:

[0082] If the number of the target feature points is greater than a preset number threshold and is evenly distributed, a direct linear transformation DLT algorithm is used to estimate a transparent transformation matrix for the target feature point pair to solve a global homography matrix for the target feature point pair;

[0083] Based on the global homography matrix, image registration is performed on the first panoramic image and the second panoramic image to obtain the registered panoramic image to be stitched.

[0084] Optionally, the registration module is specifically used to:

[0085] If the number of the target feature points is not greater than a preset number threshold and the distribution is uneven, the first panoramic image and the second panoramic image are correspondingly divided into a plurality of grids; each of the plurality of grids represents a local area of ​​the first panoramic image or the second panoramic image;

[0086] Obtaining a grid having the target feature point pair from the plurality of grids, and calculating a local homography matrix of the grid having the target feature point pair;

[0087] Based on the local homography matrix of the grid where the target feature point pair exists, the first global image and the second panoramic image are registered to obtain the registered panoramic image.

[0088] Optionally, the splicing unit is specifically used for:

[0089] Obtaining a first overlapping area of ​​the panoramic image to be stitched;

[0090] Determining a transformation relationship between the panoramic images to be stitched according to the first overlapping area;

[0091] Selecting a reference image and a non-reference image from the first panoramic image and the second panoramic image;

[0092] Calling a spatiotemporal filter to transform the non-reference image into an image coordinate system of the reference image according to a transformation parameter in the transformation relationship;

[0093] The images to be stitched are stitched together to obtain the stitched panoramic image.

[0094] Optionally, the edge elimination unit is specifically used to:

[0095] Obtaining a second overlapping area between the panoramic image to be stitched and the stitched panoramic image;

[0096] Performing Gaussian pyramid decomposition on the second overlapping area to obtain a Gaussian pyramid decomposition result;

[0097] Constructing a Laplacian pyramid according to the Gaussian pyramid decomposition result;

[0098] At each level of the Laplacian pyramid, edges of the spliced ​​images are fused by weighted fusion to obtain a fused Laplacian pyramid;

[0099] The image is reconstructed layer by layer from the fused Laplacian pyramid until the target panoramic image is obtained.

[0100] In a third aspect, the present application further provides an electronic device, the electronic device comprising a processor and a memory:

[0101] The memory is used to store computer programs;

[0102] The processor is used to execute the method provided in the first aspect according to the computer program.

[0103] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method provided in the first aspect above.

[0104] It can be seen that this application has the following beneficial effects:

[0105] The present application provides a panoramic image processing method, device, equipment and medium based on a train conversion room. In the method, a panoramic image sequence of the train conversion room is obtained, and the panoramic image sequence includes multiple frames of panoramic images; a first panoramic image and a second panoramic image that are adjacent frames in the panoramic image sequence are obtained in sequence; wherein the first panoramic image and the second panoramic image have an overlapping area; the first panoramic image and the second panoramic image are registered with the APAP algorithm to obtain the registered panoramic image to be spliced; the cylindrical forward projection algorithm is used to project the panoramic image to be spliced ​​onto a central cylindrical surface; a spatiotemporal filter is called to splice the panoramic images to be spliced ​​to obtain a spliced ​​panoramic image; the edges of the spliced ​​panoramic image are eliminated according to the panoramic image to be spliced ​​to obtain a target panoramic image; the target panoramic image is imported into a space editor to display the target panoramic image in a virtual panoramic space. In this way, compared with the problems of slow speed and low accuracy of the midpoint matching method in the prior art, the embodiment of the present application can quickly align the first panoramic image and the second panoramic image with high accuracy through the APAP algorithm; and, the cylindrical projection algorithm is used to perform central cylindrical projection, which can simplify the subsequent stitching process and have a better stitching effect, and then the edges of the stitched panoramic image are eliminated to obtain a target panoramic image with better display effect, and then the space editor is used to construct a virtual panoramic space, and the target panoramic image is displayed in the virtual panoramic space, so as to realize the display of the train renovation room in the virtual panoramic space, that is, the train renovation room is displayed in a three-dimensional form, thereby realizing the user's virtual room roaming and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0107] Figure 1 A schematic flow chart of a panoramic image processing method based on a train conversion room in an embodiment of the present application;

[0108] Figure 2 A schematic diagram of the structure of a panoramic image processing device 200 based on a train conversion room provided in an embodiment of the present application;

[0109] Figure 3 A schematic diagram of the structure of an electronic device 300 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0110] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0111] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, it should be noted that, for ease of description, only the parts related to the present application are shown in the accompanying drawings, not all structures.

[0112] At present, users cannot quickly and intuitively obtain the configuration of the train renovation room based on the existing technology alone. The applicant's research found that the above problems are mainly caused by the fact that the existing technology usually adopts the point matching method to achieve image registration. However, this method is slow and has low accuracy, and often requires manual selection of initial matching points. It cannot adapt to the fusion of large amounts of images. At the same time, this method cannot achieve seamless splicing. Therefore, users cannot quickly and intuitively obtain the configuration of the train renovation room based on the existing technology alone.

[0113] Based on this, the present application provides a panoramic image processing method, device, equipment and medium based on a train converted room. In the method, a panoramic image sequence of the train converted room is obtained, and the panoramic image sequence includes multiple frames of panoramic images; a first panoramic image and a second panoramic image that are adjacent frames in the panoramic image sequence are obtained in sequence; wherein the first panoramic image and the second panoramic image have an overlapping area; the first panoramic image and the second panoramic image are registered with the APAP algorithm to obtain the registered panoramic image to be stitched; the cylindrical orthographic projection algorithm is used to project the panoramic image to be stitched onto the central cylindrical surface; a spatiotemporal filter is called to stitch the panoramic images to be stitched to obtain a stitched panoramic image; the edges of the stitched panoramic image are eliminated according to the panoramic image to be stitched to obtain a target panoramic image; the target panoramic image is imported into a space editor to display the target panoramic image in a virtual panoramic space.

[0114] In this way, compared with the problems of slow speed and low accuracy of the midpoint matching method in the prior art, the embodiment of the present application can quickly align the first panoramic image and the second panoramic image with high accuracy through the APAP algorithm; and, the cylindrical projection algorithm is used to perform central cylindrical projection, which can simplify the subsequent stitching process and have a better stitching effect, and then the edges of the stitched panoramic image are eliminated to obtain a target panoramic image with better display effect, and then the space editor is used to construct a virtual panoramic space, and the target panoramic image is displayed in the virtual panoramic space, so as to realize the display of the train renovation room in the virtual panoramic space, that is, the train renovation room is displayed in a three-dimensional form, thereby realizing the user's virtual room roaming and improving the user experience.

[0115] To facilitate understanding of the specific implementation of the panoramic image processing method based on the train conversion room provided in the embodiment of the present application, it will be described below with reference to the accompanying drawings.

[0116] It should be noted that the subject implementing the panoramic image processing method based on the train-converted room can be the panoramic image processing device based on the train-converted room provided in the embodiment of the present application, and the panoramic image processing device based on the train-converted room can be carried in an electronic device or a functional module of an electronic device. The electronic device in the embodiment of the present application can be any device capable of implementing the panoramic image processing method based on the train-converted room in the embodiment of the present application, for example, it can be an Internet of Things (IoT) device.

[0117] See also Figure 1 , a flow chart of a panoramic image processing method based on a train conversion room is provided. In an embodiment of the present application, for example, the following steps may be included:

[0118] S1, obtain a panoramic image sequence of the train renovation room.

[0119] The panoramic image sequence includes multiple frames of panoramic images.

[0120] It should be noted that a panoramic camera, an ordinary camera or a video camera can be used to obtain a panoramic image of the train conversion room, and then the panoramic image can be downloaded to a computer device. The embodiment of the present application does not limit the method of obtaining the panoramic image of the train conversion room.

[0121] S2, sequentially acquiring a first panoramic image and a second panoramic image which are adjacent frames in a panoramic image sequence.

[0122] The first panoramic image and the second panoramic image have an overlapping area.

[0123] S3, performing image registration on the first panoramic image and the second panoramic image using an APAP algorithm to obtain a registered panoramic image to be stitched.

[0124] Among them, the APAP (As-Projective-As-Possible Image Stitching) algorithm is used to solve the image registration problem with parallax.

[0125] In a possible implementation, S3 provided in the embodiment of the present application may include:

[0126] S31, extracting feature points in the first panoramic image and feature points in the second panoramic image using a Scale-Invariant Feature Transform (SIFT) algorithm.

[0127] Among them, the feature point is usually a key point in the image. In the specific implementation process, the key point may be, for example, a corner point or an edge point, because the corner point or the edge point has good stability and invariance in image transformation.

[0128] S32, matching feature points in the first panoramic image with feature points in the second panoramic image to obtain initial feature point pairs.

[0129] In a specific implementation process, the similarity or distance between feature points may be calculated, and a point pair with the highest similarity or the smallest distance may be selected as an initial feature point pair.

[0130] S33, using a Random Sample Consensus (RANSAC) algorithm to screen the initial feature point pairs to obtain screened target feature point pairs.

[0131] The target feature point pair represents a correspondence between a first feature point in the first panoramic image and a second feature point in the second panoramic image.

[0132] It should be noted that because there are still many erroneous feature points after matching the feature points in the first panoramic image with the feature points in the second panoramic image, it is necessary to screen the feature point pairs. The screened feature point pairs usually indicate that the feature points in the first panoramic image and the feature points in the second panoramic image can correspond one-to-one.

[0133] In a possible implementation, S33 provided in the embodiment of the present application may include:

[0134] S3301, randomly select a set of samples from the initial feature point pairs.

[0135] S3302, obtaining a mathematical model according to the sample set fitting, where the mathematical model includes a homography matrix mathematical model or an affine transformation matrix mathematical model.

[0136] S3303, remove the sample set from the initial feature point pairs to obtain the remaining feature point pairs.

[0137] S3304, substituting the remaining feature point pairs into the fitted mathematical model, calculating the errors between the remaining feature points and the mathematical model, and obtaining error results.

[0138] S3305: If the error result is less than the preset error threshold, it is determined to be an inlier point, and the number of inliers among the remaining feature points is counted.

[0139] S3306, calculating the score of the mathematical model according to the ratio of the number of internal points to the total number of feature points of the initial feature point pairs.

[0140] S3307, return to execute randomly selecting a set of sample sets from the initial feature point pairs until the preset number of iterations is reached.

[0141] S3308, select the mathematical model with the highest score as the initial screening model.

[0142] S3309, recalculate and set the parameters of the initial screening model based on all the internal points in the initial feature point pair to obtain the target screening model.

[0143] S3310, filter out the inner points that match the target screening model as the filtered target feature point pairs.

[0144] S3311, output the filtered target feature point pairs.

[0145] Thus, the process of using the RANSAC algorithm for outlier filtering in the APAP algorithm is an iterative process, which finds the best fitting model through continuous random sampling, model fitting, consistency checking, and iterative updating.

[0146] In the embodiment of the present application, the RANSAC algorithm is used to screen the initial feature point pairs to exclude the mismatched feature points. The RANSAC algorithm calculates an initial transformation model by randomly selecting feature point sample pairs, and iteratively selects the transformation model with the largest number of inliers as the final fitting model. Feature point pairs that do not conform to the final fitting model will be regarded as outlier feature points and will be eliminated.

[0147] In the specific implementation process, first, the parameters required by the RANSAC algorithm need to be initialized, including the number of iterations, the number of samples, and the preset error threshold. Then, a randomly selects a small number of samples from the initial feature point pairs, and estimates a mathematical model based on the small number of samples. Then, b calculates the error with the mathematical model for the remaining samples. In the specific implementation process, the distance from the remaining samples to the mathematical model is calculated, and the samples with a distance less than the distance threshold are regarded as inliers, otherwise they are regarded as outliers. After that, c calculates the score of the mathematical model based on the ratio of the number of inliers to the total number of all feature points. Then, d repeats the above steps ac until the preset number of iterations is reached, and obtains the mathematical model with the highest score as the initial screening model. (5) Finally, all the inliers corresponding to the initial screening model are used to calculate the parameters to obtain a final target screening model, and the inliers matching the target screening model are screened out as the target feature point pairs.

[0148] The code of the RANSAC algorithm may be as follows:

[0149] import random

[0150] defransac(data,model,n,k,t,d):

[0151] #data: The data set to be processed, usually a collection of points;

[0152] #model: a class instance or object containing fit and evaluate methods;

[0153] #fit method is used to fit the model according to sample data;

[0154] The #evaluate method is used to evaluate the error or distance between a given data point and the model;

[0155] #n: The number of sample points used to fit the model in each iteration;

[0156] #k: maximum number of iterations;

[0157] #t: used to select the current optimal model (the model with the smallest error and less than t) during the iteration process;

[0158] #d: The error threshold for determining whether a data point is an inlier. Under the final selected model, if the error of a data point is less than d, it is considered an inlier.

[0159] best_model=None#Used to store the best model currently found;

[0160] est_error = float('inf') # used to store the error of the current best model;

[0161] for iin range(k):#Perform k iterations

[0162] sample = random.sample(data,n) #Randomly select n points from the data set as samples

[0163] maybe_model = model.fit(sample) # Fit the model using sample data

[0164] error=model.evaluate(data,maybe_model)#Fit the model using sample data#If the error of the current model is less than t and is the smallest error found so far, update the best model and best error

[0165] iferror <t and error<best_error:

[0166] best_model = maybe_model

[0167] best_error=error

[0168] #Use the best model finally selected to determine which data points are internal points

[0169] inliers=[]

[0170] forpoint in data:

[0171] ifmodel.evaluate(point,best_model) <d:

[0172] inliers.append(point)

[0173] return best_model, inliers

[0174] According to the above code, the basic assumption of the RANSAC algorithm is that the sample contains correct data (inliers, data that can be described by the model) and abnormal data (outliers, data that deviates far from the normal range and cannot fit the mathematical model). The RANSAC algorithm calculates the model by randomly selecting samples and iteratively selects the model with the largest number of inliers as the final fitting model, thereby effectively identifying and eliminating mismatched data and improving the accuracy of data processing.

[0175] S34, acquiring a corresponding homography matrix according to the target feature points, and registering the first panoramic image and the second panoramic image based on the homography matrix to obtain a registered panoramic image to be stitched.

[0176] It should be noted that the homography matrix includes a global homography matrix and a local homography matrix, which need to be selected according to the number and distribution of target feature point pairs to achieve registration of the first panoramic image and the second panoramic image, thereby obtaining the registered panoramic image to be stitched.

[0177] In a possible implementation, S34 provided in the embodiment of the present application may include:

[0178] S3411, if the number of target feature points is greater than a preset number threshold and is evenly distributed, a direct linear transform DLT algorithm is used to estimate a transparent transformation matrix for the target feature point pair to solve a global homography matrix for the target feature point pair.

[0179] S3412: Perform image registration on the first panoramic image and the second panoramic image based on the global homography matrix to obtain a registered panoramic image to be stitched.

[0180] In another possible implementation, S34 provided in the embodiment of the present application may further include:

[0181] S3421: If the number of target feature points is not greater than a preset number threshold and the distribution is uneven, the first panoramic image and the second panoramic image are correspondingly divided into a plurality of grids.

[0182] It should be noted that the grid division of the first panoramic image and the second panoramic image is corresponding, wherein each of the multiple grids represents a local area of ​​the first panoramic image or the second panoramic image, that is, each grid represents a local area in the global image.

[0183] S3422, obtaining a grid having a target feature point pair from a plurality of grids, and calculating a local homography matrix of the grid having the target feature point pair.

[0184] S3423, based on the local homography matrix of the grid where the target feature point pairs exist, register the first global image and the second panoramic image to obtain a registered panoramic image.

[0185] In an embodiment of the present application, the image is divided into multiple small grids, and a corresponding local homography matrix is ​​calculated for each small grid where a target feature point exists. Finally, based on the calculated local homography matrix, the first global image and the second panoramic image are aligned.

[0186] In the specific implementation process, the global homography matrix can be used to align most of the image content first, and then the local homography matrix can be used to process the details of the stitching area, that is, first execute S3411~S3412, and then execute S3421~S3423, thereby improving the effect of image registration.

[0187] S4, using a cylindrical projection algorithm, projecting the panoramic image to be stitched onto a central cylindrical surface.

[0188] In the embodiment of the present application, the images are stored in the form of cylindrical panoramic images, which are obtained by rotating the camera in the same horizontal plane for one circle and obtaining a continuous panoramic image sequence with overlapping areas, and then transformed by a cylindrical orthographic projection algorithm. Cylindrical panoramic images are simpler to obtain than cubic panoramas and spherical panoramas.

[0189] In the embodiments of the present application, on the one hand, projecting the global image sequence into a cylindrical coordinate system can make the overlapping areas of adjacent images more consistent, thereby simplifying the stitching process; on the other hand, the cylindrical projection transformation can reduce distortion and distortion during image stitching and improve stitching accuracy.

[0190] In a possible implementation manner, all registered panoramic images to be stitched in the panoramic image sequence may be obtained according to the above S2 and S3, so as to project them onto the central cylindrical surface.

[0191] S5, calling the spatiotemporal filter to stitch the panoramic image to be stitched, and obtaining a stitched panoramic image.

[0192] In the embodiment of the present application, a spatiotemporal filter is called to stitch the panoramic image to be stitched on the central cylindrical surface to obtain a stitched panoramic image.

[0193] In a possible implementation, S5 provided in the embodiment of the present application may include:

[0194] S51, obtaining a first overlapping area of ​​the panoramic image to be stitched.

[0195] S52: Determine a transformation relationship between the panoramic images to be stitched according to the first overlapping area.

[0196] S53: Select a reference image and a non-reference image from the first panoramic image and the second panoramic image.

[0197] S54, calling a spatiotemporal filter to transform the non-reference image into an image coordinate system of the reference image according to the transformation parameters in the transformation relationship.

[0198] S55, completing stitching of the images to be stitched, and obtaining a stitched panoramic image.

[0199] It should be noted that the splicing process in S51 to S52 can be regarded as being completed in a frame-to-frame synthesis manner.

[0200] In a possible implementation, all panoramic images in a panoramic image sequence can be simultaneously transformed to the same coordinate system for registration in a batch processing manner, and then different spatiotemporal filters are selected to stitch the images to obtain a stitched image. The coordinate system can be automatically selected according to certain rules or specified by the user. The coordinate system remains fixed during the stitching process. This method selects a frame in the panoramic image sequence as a reference image, selects a frame image from the image sequence as the reference image, and transforms other frame images in the panoramic image sequence into the reference image coordinate system through the transformation parameters of the transformation relationship between the reference image and the previously calculated transformation parameters, thereby realizing image stitching.

[0201] S6, performing edge elimination on the stitched panoramic image according to the panoramic image to be stitched, to obtain a target panoramic image.

[0202] It should be noted that there are many edge traces in the stitched panoramic image obtained after stitching. In this case, image fusion is used to eliminate the edge traces.

[0203] In the specific implementation process, the idea of ​​multi-band blending is adopted, and the method adopted is to directly perform Laplacian pyramid decomposition on the stitched panoramic image. Then, in a possible implementation method, S6 provided in the embodiment of the present application may include:

[0204] S61, obtaining a second overlapping area between the panoramic image to be stitched and the stitched panoramic image.

[0205] S62, performing Gaussian pyramid decomposition on the second overlapping area to obtain a Gaussian pyramid decomposition result.

[0206] Specifically, starting from the original image G0, a series of image layers G1 and G2 with gradually reduced resolutions are generated through Gaussian smoothing and downsampling operations.

[0207] S63, constructing a Laplacian pyramid according to the Gaussian pyramid decomposition result.

[0208] It should be noted that the Laplacian pyramid is a multi-resolution image representation method based on the Gaussian pyramid.

[0209] Specifically, for each layer of the Gaussian pyramid, the corresponding layer of the Laplacian pyramid is generated through upsampling (interpolation) and subtraction operations with the previous layer image.

[0210] S64, at each level of the Laplacian pyramid, edges of the spliced ​​images are fused by weighted fusion to obtain a fused Laplacian pyramid.

[0211] S65, reconstructing the image layer by layer from the fused Laplacian pyramid until the target panoramic image is obtained.

[0212] In this way, the edges of the stitched panoramic image are eliminated to obtain the target panoramic image.

[0213] In the embodiment of the present application, the panoramic image to be stitched includes the first panoramic image and the second panoramic image after registration, and the overlapping area of ​​the panoramic image to be stitched and the stitched panoramic image is obtained, that is, the overlapping area based on the first panoramic image and the second panoramic image after registration and the panoramic image to be stitched is the second overlapping area. Gaussian pyramid decomposition is performed on the second overlapping area to obtain a Gaussian pyramid decomposition result, a Laplacian pyramid is constructed based on the Gaussian pyramid decomposition result, and weighted fusion is performed to reconstruct the Laplacian pyramid after weighted summation, and finally a target panoramic image is obtained.

[0214] In the specific implementation process, G0 is set as the original image, and G1 represents the reduce operation on G0, where the definition of reduced is shown in the following formula (1):

[0215]

[0216] In the above formula (1), it is described how to generate an element G of the lth level (lower resolution) from the l-1th level (higher resolution) of the pyramid. l (i,j).

[0217] Among them, G l-1 Represents the l-1th layer of the pyramid, which is higher than the current layer G l A layer with higher resolution. (i,j) is the current layer G l The pixel coordinates of G. (2i+m,2j+n) are calculated in l (i,j), from G l-1 The 2i and 2j here are because when downsampling, the size of the image is usually halved (here it is assumed that both the horizontal and vertical directions are halved). m and n are the offsets of the convolution kernel, which are used to traverse G l-1 w(m,n) is the weight of the convolution kernel, which is used to l-1 The pixel values ​​in are weighted summed. It means to double sum all elements in the convolution kernel. n ranges from 1 to 5. In practice, m and n can be set to the corresponding range.

[0218] After expanding G1, OSubtracting them, we can get the first layer L0 of the Laplacian pyramid. Similarly, the L2, L3 and other layers of the Laplacian pyramid can also be calculated in this way.

[0219] S7, importing the target panoramic image into the space editor, and displaying the target panoramic image in the virtual panoramic space.

[0220] In an embodiment of the present application, after generating the target panoramic image, the space editor constructs a virtual panoramic space and displays the target panoramic image in the virtual panoramic space, thereby realizing the display of the train converted room in the virtual panoramic space, that is, displaying the train converted room in a three-dimensional form, thereby realizing virtual room roaming.

[0221] In a possible implementation, the method provided in the embodiment of the present application may further include:

[0222] S8, back-projecting the target panoramic image in the virtual panoramic space into a central projection image through a back-projection algorithm of the cylindrical panoramic image; displaying the central projection image on a computer device so as to adjust the target panoramic image displayed in the virtual panoramic space based on the central projection image.

[0223] In an embodiment of the present application, through the back-projection algorithm of the cylindrical panoramic image, the visible part of the target panoramic image is back-projected as a central projection image and displayed on a computer device, so that the staff can adjust the target panoramic image displayed in the virtual panoramic space based on the central projection image, and can achieve fixed-point rotation display on the seamlessly stitched target panoramic image.

[0224] It should be noted that the back-projection algorithm is used to convert it into a central projection image that is easy to display on the screen, which makes it convenient for the staff to make corresponding edits, thereby further improving the display of the target panoramic image corresponding to the train renovation room and improving the user experience.

[0225] In this way, compared with the problems of slow speed and low accuracy of the midpoint matching method in the prior art, the embodiment of the present application can quickly align the first panoramic image and the second panoramic image with high accuracy through the APAP algorithm; and, the cylindrical projection algorithm is used to perform central cylindrical projection, which can simplify the subsequent stitching process and have a better stitching effect, and then the edges of the stitched panoramic image are eliminated to obtain a target panoramic image with better display effect, and then the space editor is used to construct a virtual panoramic space, and the target panoramic image is displayed in the virtual panoramic space, so as to realize the display of the train renovation room in the virtual panoramic space, that is, the train renovation room is displayed in a three-dimensional form, thereby realizing the user's virtual room roaming and improving the user experience.

[0226] See also Figure 2The embodiment of the present application further provides a panoramic image processing device 200 based on a train conversion room, and the device 200 includes:

[0227] An acquisition unit 201 is used to acquire a panoramic image sequence of the train renovation room, wherein the panoramic image sequence includes multiple frames of panoramic images;

[0228] The acquisition unit 201 is further configured to sequentially acquire a first panoramic image and a second panoramic image which are adjacent frames to each other in the panoramic image sequence; wherein the first panoramic image and the second panoramic image have an overlapping area;

[0229] A registration unit 202 is used to perform image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain a registered panoramic image to be stitched;

[0230] The projection unit 203 is used to project the panoramic image to be stitched onto a central cylindrical surface using a cylindrical orthographic projection algorithm;

[0231] A stitching unit 204 is used to call a spatiotemporal filter to stitch the panoramic images to be stitched, so as to obtain a stitched panoramic image;

[0232] An edge removal unit 205 is used to remove the edges of the stitched panoramic image according to the panoramic image to be stitched, so as to obtain a target panoramic image;

[0233] The display unit 206 is used to import the target panoramic image into the space editor and display the target panoramic image in the virtual panoramic space.

[0234] Optionally, the device 200 further includes: a back projection unit 207;

[0235] The back projection unit 207 is specifically used for:

[0236] Back-projecting the target panoramic image in the virtual panoramic space into a central projection image through a back-projection algorithm of a cylindrical panoramic image;

[0237] The central projection image is displayed on a computer device so as to adjust the target panoramic image displayed in the virtual panoramic space based on the central projection image.

[0238] Optionally, the registration unit 202 includes:

[0239] A feature point extraction module, used to extract feature points in the first panoramic image and feature points in the second panoramic image by using a scale-invariant feature transform (SIFT) algorithm;

[0240] a matching module, configured to match feature points in the first panoramic image with feature points in the second panoramic image to obtain an initial feature point pair;

[0241] A screening module, used to screen the initial feature point pairs using a random sample consistent RANSAC algorithm to obtain a screened target feature point pair; the target feature point pair represents a correspondence between a first feature point in the first panoramic image and a second feature point in the second panoramic image;

[0242] A registration module is used to obtain a corresponding homography matrix according to the target feature points, and to register the first panoramic image and the second panoramic image based on the homography matrix to obtain the registered panoramic image to be stitched.

[0243] Optionally, the screening module is specifically used to:

[0244] Randomly select a set of samples from the initial feature point pairs;

[0245] Obtaining a mathematical model by fitting the sample set, wherein the mathematical model includes a homography matrix mathematical model or an affine transformation matrix mathematical model;

[0246] Eliminating the sample set from the initial feature point pairs to obtain remaining feature point pairs;

[0247] Substituting the remaining feature point pairs into the fitted mathematical model, calculating the errors between the remaining feature points and the mathematical model, and obtaining error results;

[0248] If the error result is less than a preset error threshold, it is determined to be an inlier point, and the number of inliers among the remaining feature points is counted;

[0249] Calculating the score of the mathematical model according to the ratio of the number of the inner points to the total number of the feature points of the initial feature point pair;

[0250] Returning to execute the step of randomly selecting a set of sample sets from the initial feature point pairs until a preset number of iterations is reached;

[0251] The mathematical model with the highest score was selected as the initial screening model;

[0252] Recalculating and setting the parameters of the initial screening model based on all the internal points in the initial feature point pair to obtain a target screening model;

[0253] Screening out the inner points matching the target screening model as the screened target feature point pairs;

[0254] Output the filtered target feature point pairs.

[0255] Optionally, the registration module is specifically used to:

[0256] If the number of the target feature points is greater than a preset number threshold and is evenly distributed, a direct linear transformation DLT algorithm is used to estimate a transparent transformation matrix for the target feature point pair to solve a global homography matrix for the target feature point pair;

[0257] Based on the global homography matrix, image registration is performed on the first panoramic image and the second panoramic image to obtain the registered panoramic image to be stitched.

[0258] Optionally, the registration module is specifically used to:

[0259] If the number of the target feature points is not greater than a preset number threshold and the distribution is uneven, the first panoramic image and the second panoramic image are correspondingly divided into a plurality of grids; each of the plurality of grids represents a local area of ​​the first panoramic image or the second panoramic image;

[0260] Obtaining a grid having the target feature point pair from the plurality of grids, and calculating a local homography matrix of the grid having the target feature point pair;

[0261] Based on the local homography matrix of the grid where the target feature point pair exists, the first global image and the second panoramic image are registered to obtain the registered panoramic image.

[0262] Optionally, the splicing unit 204 is specifically used for:

[0263] Obtaining a first overlapping area of ​​the panoramic image to be stitched;

[0264] Determining a transformation relationship between the panoramic images to be stitched according to the first overlapping area;

[0265] Selecting a reference image and a non-reference image from the first panoramic image and the second panoramic image;

[0266] Calling a spatiotemporal filter to transform the non-reference image into an image coordinate system of the reference image according to a transformation parameter in the transformation relationship;

[0267] The images to be stitched are stitched together to obtain the stitched panoramic image.

[0268] Optionally, the edge elimination unit 205 is specifically configured to:

[0269] Obtaining a second overlapping area between the panoramic image to be stitched and the stitched panoramic image;

[0270] Performing Gaussian pyramid decomposition on the second overlapping area to obtain a Gaussian pyramid decomposition result;

[0271] Constructing a Laplacian pyramid according to the Gaussian pyramid decomposition result;

[0272] At each level of the Laplacian pyramid, edges of the spliced ​​images are fused by weighted fusion to obtain a fused Laplacian pyramid;

[0273] The image is reconstructed layer by layer from the fused Laplacian pyramid until the target panoramic image is obtained.

[0274] It should be noted that the specific implementation method of the device 200 and the technical effects achieved can be found in Figure 1 The relevant description in the method shown.

[0275] In addition, the present application embodiment also provides an electronic device 300, such as Figure 3 As shown, the electronic device 300 includes a processor 301 and a memory 302:

[0276] The memory 302 is used to store computer programs;

[0277] The processor 301 is used to execute according to the computer program Figure 1 The method provided.

[0278] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method provided by the embodiment of the present application.

[0279] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0280] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, in which the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the goals of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0281] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. It should be noted that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.

Claims

1. A panoramic image processing method based on a train conversion room, characterized in that: The method comprises: Acquire a panoramic image sequence of the train renovation room, wherein the panoramic image sequence includes multiple frames of panoramic images; Sequentially acquiring a first panoramic image and a second panoramic image which are adjacent frames to each other in the panoramic image sequence; wherein the first panoramic image and the second panoramic image have an overlapping area; Performing image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain a registered panoramic image to be stitched; Using a cylindrical projection algorithm, projecting the panoramic image to be stitched onto a central cylindrical surface; Calling a spatiotemporal filter to stitch the panoramic images to be stitched, to obtain a stitched panoramic image; Eliminating the edges of the panoramic image to be stitched according to the panoramic image to be stitched, to obtain a target panoramic image; The target panoramic image is imported into a space editor, and the target panoramic image is displayed in a virtual panoramic space.

2. The method according to claim 1, characterized in that The using the APAP algorithm to perform image registration on the first panoramic image and the second panoramic image to obtain the registered panoramic image to be stitched includes: Extracting feature points in the first panoramic image and feature points in the second panoramic image using a scale-invariant feature transform (SIFT) algorithm; Matching feature points in the first panoramic image with feature points in the second panoramic image to obtain initial feature point pairs; The initial feature point pairs are screened using a random sample consistent RANSAC algorithm to obtain a screened target feature point pair; the target feature point pair represents a correspondence between a first feature point in the first panoramic image and a second feature point in the second panoramic image; A corresponding homography matrix is ​​acquired according to the target feature points, and the first panoramic image and the second panoramic image are registered based on the homography matrix to obtain the registered panoramic image to be stitched.

3. The method according to claim 2, characterized in that The random sample consistent RANSAC algorithm is used to screen the initial feature point pairs to obtain the screened target feature point pairs, including: Randomly select a set of samples from the initial feature point pairs; Obtaining a mathematical model by fitting the sample set, wherein the mathematical model includes a homography matrix mathematical model or an affine transformation matrix mathematical model; Eliminating the sample set from the initial feature point pairs to obtain remaining feature point pairs; Substituting the remaining feature point pairs into the fitted mathematical model, calculating the errors between the remaining feature points and the mathematical model, and obtaining error results; If the error result is less than a preset error threshold, it is determined to be an inlier point, and the number of inliers among the remaining feature points is counted; Calculating the score of the mathematical model according to the ratio of the number of the inner points to the total number of the feature points of the initial feature point pair; Returning to execute the step of randomly selecting a set of sample sets from the initial feature point pairs until a preset number of iterations is reached; The mathematical model with the highest score was selected as the initial screening model; Recalculating and setting the parameters of the initial screening model based on all the internal points in the initial feature point pair to obtain a target screening model; Screening out the inner points matching the target screening model as the screened target feature point pairs; Output the filtered target feature point pairs.

4. The method according to claim 2, characterized in that: The step of acquiring a corresponding homography matrix according to the target feature points, registering the first panoramic image and the second panoramic image based on the homography matrix, and obtaining the registered panoramic image to be stitched includes: If the number of the target feature points is greater than a preset number threshold and is evenly distributed, a direct linear transformation DLT algorithm is used to estimate a transparent transformation matrix for the target feature point pair to solve a global homography matrix for the target feature point pair; Based on the global homography matrix, image registration is performed on the first panoramic image and the second panoramic image to obtain the registered panoramic image to be stitched.

5. The method according to claim 2, characterized in that: The step of acquiring a corresponding homography matrix according to the target feature points, registering the first panoramic image and the second panoramic image based on the homography matrix, and obtaining the registered panoramic image to be stitched includes: If the number of the target feature points is not greater than a preset number threshold and the distribution is uneven, the first panoramic image and the second panoramic image are correspondingly divided into a plurality of grids; each of the plurality of grids represents a local area of ​​the first panoramic image or the second panoramic image; Obtaining a grid having the target feature point pair from the plurality of grids, and calculating a local homography matrix of the grid having the target feature point pair; Based on the local homography matrix of the grid where the target feature point pair exists, the first global image and the second panoramic image are registered to obtain the registered panoramic image.

6. The method according to claim 1, characterized in that The calling of the spatiotemporal filter to stitch the images to be stitched to obtain a stitched panoramic image includes: Obtaining a first overlapping area of ​​the panoramic image to be stitched; Determining a transformation relationship between the panoramic images to be stitched according to the first overlapping area; Selecting a reference image and a non-reference image from the first panoramic image and the second panoramic image; Calling a spatiotemporal filter to transform the non-reference image into an image coordinate system of the reference image according to a transformation parameter in the transformation relationship; The images to be stitched are stitched together to obtain the stitched panoramic image.

7. The method according to claim 1, characterized in that The step of performing edge elimination on the stitched image according to the panoramic image to be stitched to obtain a target panoramic image includes: Obtaining a second overlapping area between the panoramic image to be stitched and the stitched panoramic image; Performing Gaussian pyramid decomposition on the second overlapping area to obtain a Gaussian pyramid decomposition result; Constructing a Laplacian pyramid according to the Gaussian pyramid decomposition result; At each level of the Laplacian pyramid, edges of the spliced ​​images are fused by weighted fusion to obtain a fused Laplacian pyramid; The image is reconstructed layer by layer from the fused Laplacian pyramid until the target panoramic image is obtained.

8. A panoramic image processing device based on a train conversion room, characterized in that: The device comprises: An acquisition unit, used for acquiring a panoramic image sequence of the train renovation room, wherein the panoramic image sequence includes a plurality of panoramic image frames; The acquisition unit is further used to sequentially acquire a first panoramic image and a second panoramic image which are adjacent frames to each other in the panoramic image sequence; wherein the first panoramic image and the second panoramic image have an overlapping area; A registration unit, configured to perform image registration on the first panoramic image and the second panoramic image by using an APAP algorithm to obtain a registered panoramic image to be stitched; A projection unit, used for projecting the panoramic image to be stitched onto a central cylindrical surface by using a cylindrical orthographic projection algorithm; A stitching unit, used for calling a spatiotemporal filter to stitch the panoramic images to be stitched, so as to obtain a stitched panoramic image; An edge elimination unit, configured to eliminate the edges of the stitched panoramic image according to the panoramic image to be stitched, so as to obtain a target panoramic image; The display unit is used to import the target panoramic image into the space editor and display the target panoramic image in the virtual panoramic space.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 7 according to the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

Images