Panoramic stitching optimization method and system based on distortion correction

Abstract

The present application relates to the technical field of image correction, and a panorama splicing optimization method and system based on distortion correction, comprising: receiving an ordered image sequence created by a user, dividing an overlapping area corresponding to a splicing node into a plurality of target sub-areas in a vertical direction, establishing an optimal splicing seam for each target sub-area among the plurality of target sub-areas, connecting the splicing seams of the plurality of sub-areas in sequence to obtain a global splicing seam, fusing two splicing images corresponding to the splicing node by using the global splicing seam to obtain a seamless image, constructing a pixel coordinate system with the upper left corner of the seamless panorama as an origin, constructing a target rectangle by using the seamless panorama and a fitting quadrilateral, calculating a target transformation matrix by using four fitting vertex coordinates and four target vertex coordinates, and remapping the seamless panorama by using the target transformation matrix to obtain a corrected standard rectangular panorama. The present application can improve the picture quality of distortion correction and the efficiency of panorama splicing optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image correction, in particular to a panorama stitching optimization method and system based on distortion correction. BACKGROUND

[0002] The panorama stitching technology based on distortion correction can eliminate the visual discontinuity at the image overlap, correct the irregular panorama into a standard rectangle, ensure the integrity and continuity of the image content, improve the visual consistency and practical value of the panorama, meet the demand for standard image format in subsequent image analysis, target detection and other scenes, and has important significance in security monitoring, unmanned aerial vehicle aerial photography, virtual reality and other fields.

[0003] At present, the traditional panorama stitching based on distortion correction often adopts a global single stitching seam, without considering the difference in gradient and texture in the overlapping area, which is easy to cause the stitching seam to pass through the high detail area and produce obvious stitching marks.

[0004] Although the above method can correct the distorted image, the calculation of the optimal stitching seam depends on the color difference degree and ignores the texture information, which is difficult to adapt to complex texture scenes, and the size of the traditional target rectangle is set by experience without considering the proportion of the effective pixels of the panorama and the fitted quadrilateral, which is easy to cause the image to be stretched too much or the effective information to be lost after correction. Therefore, how to improve the picture quality of distortion correction and the efficiency of panorama stitching optimization has become a problem to be solved. SUMMARY

[0005] The present application provides a panorama stitching optimization method based on distortion correction and a computer readable storage medium, which aims to improve the picture quality of distortion correction and the efficiency of panorama stitching optimization.

[0006] To achieve the above purpose, the present application provides a panorama stitching optimization method based on distortion correction, which comprises:

[0007] receiving an ordered image sequence created by a user, wherein the ordered image sequence comprises: S1, S2, …, S i , …, S m image, and each two adjacent stitching images contain an overlapping area;

[0008] constructing m-1 stitching nodes using the ordered image sequence, wherein the m-1 stitching nodes comprise: (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each stitching node contains two stitching images;

[0009] The following operations are performed on each of the m-1 stitching nodes:

[0010] The overlapping area corresponding to the stitching node is divided into a plurality of target sub-areas in the vertical direction;

[0011] An optimal stitching seam is established for each of the plurality of target sub-areas, obtaining a plurality of sub-area stitching seams;

[0012] The plurality of sub-area stitching seams are sequentially connected to obtain a global stitching seam, and the two stitching images corresponding to the stitching node are fused using the global stitching seam to obtain a seamless image. After each stitching node obtains the corresponding seamless image, a seamless panoramic image is obtained;

[0013] A pixel coordinate system is constructed with the upper left corner of the seamless panoramic image as the origin, wherein the pixel coordinate system includes an x-axis and a y-axis, the edges of the seamless panoramic image are extracted and fitted into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral includes four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 );

[0014] A target rectangle is constructed using the seamless panoramic image and the fitted quadrilateral, wherein the target rectangle includes four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 );

[0015] A target transformation matrix is calculated using the four fitted vertex coordinates and the four target vertex coordinates, and the seamless panoramic image is remapped using the target transformation matrix to obtain a corrected standard rectangular panoramic image.

[0016] Optionally, the overlapping area corresponding to the stitching node is divided into a plurality of target sub-areas in the vertical direction, comprising:

[0017] The overlapping area is divided into a plurality of initial sub-areas in the vertical direction, wherein each initial sub-area is identified with an area bit sequence, and the area bit sequence is 1, 2, 3, …, n;

[0018] The following operations are performed on each of the plurality of initial sub-areas:

[0019] calculating gradient values of the initial sub-regions to obtain region gradient values;

[0020]

[0021] calculating a gradient accumulation threshold value using the region gradient values;

[0022] sorting the region gradient values in ascending order of region ordinal to obtain a gradient value sequence;

[0023] sequentially traversing the region gradient values starting from the region gradient value of region ordinal 1 in the gradient value sequence and accumulating the traversed region gradient values to obtain an accumulated gradient value;

[0024] when the accumulated gradient value is greater than or equal to the accumulated gradient threshold value, delimiting one or more initial sub-regions corresponding to one or more region ordinals in the current traversal as a target sub-region;

[0025] setting the accumulated gradient value to zero and continuing to traverse the subsequent part of the gradient sequence until the traversal is completed to obtain a plurality of target sub-regions.

[0026] Optionally, the calculating of the gradient accumulation threshold value using the region gradient values comprises:

[0027] calculating the standard deviation and the mean of the region gradient values to obtain a gradient value standard deviation and a gradient mean;

[0028] calculating the gradient accumulation threshold value using the gradient value standard deviation and the gradient mean, wherein the gradient accumulation threshold value is as follows:

[0029] ,

[0030] wherein G' represents the gradient accumulation threshold value, G avg represents the gradient mean, G std represents the gradient value standard deviation, G0 represents a preset lower limit of the gradient mean, γ represents a preset adjustment coefficient, avg represents a mean identifier, std represents a standard deviation identifier, and max() represents a maximum value.

[0031] Optionally, the establishing of the optimal splicing seam for each target sub-region in the plurality of target sub-regions to obtain a plurality of sub-region splicing seams comprises:

[0032] extracting target sub-regions from the plurality of target sub-regions in sequence and performing the following operations on the extracted target sub-regions:

[0033] dividing the target sub-region into a plurality of pixel screening rows and performing the following operations on each pixel screening row in the plurality of pixel screening rows:​

[0034] Set the pixel points corresponding to the pixel screening row, and obtain a plurality of pixel points;

[0035] The following operations are performed on each pixel point in the plurality of pixel points:

[0036] Calculate the color difference degree and the texture difference degree corresponding to the pixel point;

[0037] Correlate the color difference degree and the texture difference degree to obtain a pixel node, wherein the pixel node corresponds to the pixel point one-to-one;

[0038] Summarize the pixel nodes to obtain a pixel node set;

[0039] Match the target pixel point using the pixel node set, wherein the target pixel point corresponds to the pixel screening row one-to-one;

[0040] Summarize the target pixel points to obtain a plurality of target pixel points, and use the plurality of target pixel points to form an optimal splicing seam;

[0041] Use the optimal splicing seam as a sub-region splicing seam, and summarize the sub-region splicing seams to obtain a plurality of sub-region splicing seams.

[0042] Optionally, the matching of the target pixel point using the pixel node set comprises:

[0043] Calculate the mean value of the color difference degree corresponding to all pixel nodes in the pixel node set to obtain a reference color difference degree;

[0044] Calculate the mean value of the texture difference degree corresponding to all pixel nodes in the pixel node set to obtain a reference texture difference degree;

[0045] Extract the pixel nodes from the pixel node set in sequence, and perform the following operations on the extracted pixel nodes:

[0046] Calculate the pixel energy value using the pixel node, the reference color difference degree, and the reference texture difference degree;

[0047] Summarize the pixel energy values to obtain a plurality of pixel energy values, and use the pixel point corresponding to the smallest pixel energy value in the plurality of pixel energy values as the target pixel point.

[0048] Optionally, the construction of the target rectangle using the seamless panorama and the fitted quadrilateral comprises:

[0049] Confirm the stretching ratio value interval using the seamless panorama, wherein the stretching ratio value interval includes a minimum stretching ratio value and a maximum stretching ratio value;

[0050] Confirm S width values and C height values using the fitted quadrilateral;

[0051] S width values and C height values are combined to form S×C distance nodes, wherein each distance node contains a width value and a height value;

[0052] The following operations are performed on each distance node in the S×C distance nodes:

[0053] The width value and the height value are extracted from the distance node;

[0054] The ratio of the width value to the height value is calculated to obtain a fitting ratio;

[0055] If the fitting ratio is within the stretching ratio interval, the distance node corresponding to the fitting ratio is confirmed as a candidate distance node;

[0056] The candidate distance nodes are summarized to obtain a candidate distance node set;

[0057] The target distance node is confirmed by using the candidate distance node set and the seamless panorama;

[0058] The target rectangle is constructed by using the width value and the height value corresponding to the target distance node.

[0059] Optionally, the target distance node is confirmed by using the candidate distance node set and the seamless panorama, including:

[0060] The effective pixels corresponding to the seamless panorama are counted to obtain a total number of effective pixels;

[0061] The candidate distance nodes are extracted from the candidate distance node set in sequence, and if the candidate distance node satisfies a preset preferred condition, the candidate distance node is used as a preferred distance node;

[0062] The preferred distance nodes are summarized to obtain a preferred distance node set;

[0063] The product of the height value and the width value corresponding to each preferred distance node in the preferred distance node set is calculated to obtain a preferred value, wherein the preferred value corresponds to the preferred distance node in a one-to-one manner;

[0064] The preferred values are summarized to obtain a preferred value set;

[0065] The preferred distance node corresponding to the smallest preferred value in the preferred value set is used as a target preferred distance node;

[0066] Otherwise, the product of the height value and the width value corresponding to each candidate distance node in the candidate distance node set is calculated to obtain a candidate value, wherein the candidate value corresponds to the candidate distance node in a one-to-one manner;

[0067] The candidate values are summarized to obtain a candidate value set;

[0068] The candidate distance node corresponding to the largest candidate value in the candidate value set is used as a target candidate distance node;

[0069] The target distance node is a target preferred distance node or a target candidate distance node.

[0070] Optionally, the preferred condition is as follows:

[0071] ,

[0072] wherein w represents a width value corresponding to the candidate distance node, h represents a height value corresponding to the candidate distance node, A ref represents the total number of effective pixels, η represents a preset efficiency factor, and ref represents an effective identifier.

[0073] Optionally, the calculating the target transformation matrix by using the four fitting vertex coordinates and the four target vertex coordinates comprises:

[0074] normalizing the four fitting vertex coordinates and the four target vertex coordinates respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates;

[0075] constructing a linear equation set by using a preset algorithm based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, and generating an initial transformation matrix by using the linear equation set;

[0076] performing a hard constraint processing on the initial transformation matrix to obtain the target transformation matrix.

[0077] To achieve the above object, the present application further provides a panoramic stitching optimization system based on distortion correction, comprising:

[0078] a stitching node construction module configured to receive an ordered image sequence created by a user, wherein the ordered image sequence comprises S1, S2, …, S i , …, S m m stitching images, and each two adjacent stitching images contain an overlapping area;

[0079] constructing m-1 stitching nodes by using the ordered image sequence, wherein the m-1 stitching nodes comprise (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each stitching node contains two stitching images;

[0080] a stitching area division module configured to perform the following operation on each stitching node in the m-1 stitching nodes:

[0081] dividing the overlapping area corresponding to the stitching node into a plurality of target sub-areas in a vertical direction;

[0082] a global seamless splicing module, configured to establish an optimal splicing seam for each target sub-region in the plurality of target sub-regions, to obtain a plurality of sub-region splicing seams;

[0083] connecting the plurality of sub-region splicing seams in sequence to obtain a global splicing seam, fusing two splicing images corresponding to a splicing node by using the global splicing seam to obtain a seamless image, and obtaining a seamless panorama after each splicing node obtains a corresponding seamless image;

[0084] a post-splicing correction module, configured to construct a pixel coordinate system with the upper left corner of the seamless panorama as an origin, extract edges of the seamless panorama and fit the edges into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 );

[0085] constructing a target rectangle by using the seamless panorama and the fitted quadrilateral, wherein the target rectangle comprises four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 );

[0086] calculating a target transformation matrix by using the four fitted vertex coordinates and the four target vertex coordinates, and performing remapping on the seamless panorama by using the target transformation matrix to obtain a corrected standard-rectangle panorama.

[0087] To solve the above problems, the application further provides an electronic device, which comprises:

[0088] a memory, configured to store at least one instruction;

[0089] a processor, configured to execute the instruction stored in the memory to implement the distortion-correction-based panorama splicing optimization method.

[0090] To solve the above problems, the application further provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the distortion-correction-based panorama splicing optimization method.

[0091] The present application receives an ordered image sequence created by a user, wherein the ordered image sequence comprises: S1, S2, …, S i , …, S m , and each two adjacent spliced images contain an overlapping area, constructs m-1 spliced nodes by using the ordered image sequence, wherein the m-1 spliced nodes comprise: (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each spliced node contains two spliced images, divides the overlapping area corresponding to the spliced node into a plurality of target sub-regions in the vertical direction, it can be seen that the present application divides the overlapping area into a plurality of initial sub-regions, quantifies the texture intensity of each initial sub-region by calculating the gradient value of the initial sub-region, and dynamically calculates the gradient accumulation threshold. According to the gradient accumulation threshold, the initial sub-regions are intelligently merged to form the target sub-regions, which can finely register the high-texture region to improve the splicing quality, and coarsely merge the smooth region to optimize the efficiency, on the basis of ensuring the splicing quality, the efficiency of the overlapping area splicing is improved, for each target sub-region in the plurality of target sub-regions, an optimal splicing seam is established, a plurality of sub-region splicing seams are obtained, it can be seen that the present application independently calculates the optimal splicing seam for each target sub-region, uses a pixel screening mechanism that minimizes the in-row normalized energy, eliminates the dimensional difference of color and texture, ensures the continuity and consistency of the splicing seam in vision, realizes the parallel optimization processing of multiple sub-regions, improves the precision and efficiency of large-scale image splicing, and improves the image fusion quality in complex scenes. The plurality of sub-region splicing seams are connected in order to obtain a global splicing seam, the two spliced images corresponding to the splicing node are fused by using the global splicing seam to obtain a seamless image, and when each splicing node obtains a corresponding seamless image, a seamless panorama is obtained. Taking the upper left corner of the seamless panorama as the origin, a pixel coordinate system is constructed, wherein the pixel coordinate system comprises an x-axis and a y-axis, the edge of the seamless panorama is extracted and fitted into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 ), a target rectangle is constructed by using the seamless panorama and the fitted quadrilateral, wherein the target rectangle comprises four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2x t2 ), T3(x t3 , y t3 ), T4(x t4 , y t4 ), the target transformation matrix is calculated by using four fitting vertex coordinates and four target vertex coordinates, the seamless panorama picture is remapped by using the target transformation matrix, and the corrected standard rectangular panorama picture is obtained, it can be seen that the intelligent generation of the target rectangle is realized by setting the stretching ratio interval and the total number of effective pixels, the two-stage screening strategy (preferably the minimum area, the bottom selection of the maximum area) is used to automatically balance the cutting efficiency and content retention of the seamless panorama picture, the distorted edge is maximally removed on the premise of ensuring the lossless core picture of the seamless panorama picture, and the composition quality and picture neatness are improved. Therefore, the picture quality of distortion correction and the efficiency of panorama splicing optimization can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0092] Figure 1 A flowchart of a panorama splicing optimization method based on distortion correction provided by an embodiment of the present application is shown;

[0093] Figure 2 A function module diagram of a panorama splicing optimization system based on distortion correction provided by an embodiment of the present application is shown;

[0094] Figure 3 A structural diagram of an electronic device for implementing the panorama splicing optimization method based on distortion correction provided by an embodiment of the present application is shown.

[0095] Explanation of reference signs:

[0096] 1, electronic device; 10, processor; 11, memory; 12, bus.

[0097] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0098] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0099] Embodiments of the present application provide a panorama splicing optimization method based on distortion correction. The execution subject of the panorama splicing optimization method based on distortion correction includes but is not limited to at least one of the electronic devices that can be configured to execute the method provided by the embodiments of the present application, such as a server and a terminal. In other words, the panorama splicing optimization method based on distortion correction can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster.

[0100] Reference Figure 1 The diagram shown is a flowchart illustrating a panoramic stitching optimization method based on distortion correction according to an embodiment of the present invention. In this embodiment, the panoramic stitching optimization method based on distortion correction includes:

[0101] S1. Receive an ordered image sequence created by the user, wherein the ordered image sequence includes: S1, S2, ..., S... i S m The images are stitched together, and every two adjacent stitched images contain overlapping areas.

[0102] It should be explained that an ordered image sequence refers to a set of stitched images arranged in the order they were captured, S1, S2, ..., S... i S m All of these represent stitched images within an ordered image sequence. A stitched image is a basic image unit that constitutes an ordered image sequence; it is a single original image that is to be synthesized into a panoramic image. The overlapping area is the portion where two adjacent stitched images overlap, and it forms the basis for image stitching.

[0103] S2. Construct m-1 stitching nodes using an ordered image sequence, where the m-1 stitching nodes include: (S1, S2), (S2, S3), ..., (S... i-1 S i ), ..., (S m-1 S m Each stitching node contains two stitched images.

[0104] For example, suppose a user takes a 180-degree panoramic view of an office, rotating the camera from left to right during the shooting process, taking four photos in sequence. The ordered image sequence is (S1, S2, S3, S4). Thus, m-1=3 stitching nodes can be constructed: stitching node (S1, S2), stitching node (S2, S3), and stitching node (S3, S4). Taking stitching node (S1, S2) as an example, the construction of stitching node (S1, S2) is to handle the alignment and fusion of overlapping areas between the stitched images S1 and S2.

[0105] S3. Divide the overlapping area corresponding to the splicing node into multiple target sub-regions in the vertical direction.

[0106] It should be explained that dividing the overlapping area corresponding to the splicing node into multiple target sub-regions in the vertical direction includes:

[0107] The overlapping region is divided into multiple initial sub-regions in the vertical direction. Each initial sub-region is identified by a region position number, which is 1, 2, 3, ..., n.

[0108] performing the following operations on each of the plurality of initial sub-regions:

[0109] calculating a gradient value of the initial sub-region to obtain a region gradient value;

[0110] summarizing the region gradient values to obtain a plurality of region gradient values, wherein the region gradient values correspond to region positions one by one;

[0111] calculating a gradient accumulation threshold value using the plurality of region gradient values;

[0112] sorting the plurality of region gradient values in ascending order of region position to obtain a gradient value sequence;

[0113] sequentially traversing the region gradient values from the region gradient value of region position 1 in the gradient value sequence and accumulating the traversed region gradient values to obtain an accumulated gradient value;

[0114] when the accumulated gradient value is greater than or equal to the accumulated gradient threshold value, delimiting one or more initial sub-regions corresponding to one or more region positions that have been traversed as a target sub-region;

[0115] setting the accumulated gradient value to zero and continuing to traverse the subsequent part of the gradient sequence until the traversal is completed to obtain a plurality of target sub-regions.

[0116] It can be understood that the initial sub-regions are a series of image blocks obtained by uniformly and continuously mechanically dividing the overlapping region in the vertical direction. It should be noted that the division method of the initial sub-regions includes but is not limited to fixed sub-region height method, fixed division number ratio method, etc. For example, only the fixed sub-region height method is taken as an example here: assuming that the vertical pixel height of the overlapping region is 200 pixels, in the vertical direction, the vertical pixel height of each initial sub-region is set to 20 pixels, so the overlapping region is divided into 10 initial sub-regions with a vertical pixel height of 20 pixels. By calculating the gradient value of each initial sub-region, the texture richness of the image in the initial sub-region is determined, thereby providing a basis for decision-making for the generation of subsequent target sub-regions.

[0117] Specifically, the gradient accumulation threshold value is calculated using the plurality of region gradient values, including:

[0118] calculating the standard deviation and the mean of the plurality of region gradient values to obtain a gradient value standard deviation and a gradient mean;

[0119] calculating the gradient accumulation threshold value using the gradient value standard deviation and the gradient mean, wherein the gradient accumulation threshold value is as follows:

[0120] ,

[0121] wherein G' represents the gradient accumulation threshold value, G represents the gradient value standard deviation, and G represents the gradient mean.avg Gavg represents the gradient mean value std Gstd represents the gradient value standard deviation, G0 represents the preset gradient mean value lower limit, γ represents the preset adjustment coefficient, avg represents the mean identifier, std represents the standard deviation identifier, max() represents the maximum value.

[0122] It should be understood that the region gradient value is the sum of the gradient amplitudes of all pixels in the corresponding initial sub-region, which is used to measure the total texture intensity of the corresponding initial sub-region. Optionally, the method for obtaining the region gradient value includes but is not limited to a Sobel operator, a first-order difference of image gray values, etc. The gradient value standard deviation is used to measure the degree of dispersion between the region gradient values corresponding to all initial sub-regions and the gradient mean value. The smaller the gradient value standard deviation, the more uniform the texture richness at different positions in the entire overlapping region. The gradient mean value is the arithmetic mean of the region gradient values corresponding to all initial sub-regions.

[0123] It can be understood that the preset gradient mean value lower limit is a safety gradient value set by a person. The purpose of setting the gradient mean value lower limit is to prevent the gradient mean value from being too small when the overall overlapping region is too smooth, resulting in a too large gradient cumulative threshold value. After traversing all region gradient values, the cumulative gradient value obtained still cannot trigger the division condition, and finally the entire overlapping region is incorrectly merged into a target sub-region, causing the panoramic stitching optimization method to fail. The gradient cumulative threshold value is a gradient critical value used to determine when to merge one or more initial sub-regions being sequentially traversed and demarcate as a target sub-region. The preset adjustment coefficient is a preset parameter (with a value between 0 and 1) used to control the sensitivity of the gradient cumulative threshold value. Optionally, the method for obtaining the adjustment coefficient includes but is not limited to an empirical value method, a grid search method, etc.

[0124] It can be understood that the greater the texture richness of the initial sub-region, the higher the corresponding region gradient value. The region with high texture richness (high region gradient value) usually contains a large amount of details. If the target sub-region is too large, it may cause a large stitching error due to local deformation.

[0125] For example, assuming that the overlapping region is divided into 5 initial sub-regions, denoted as: {a1, a2, a3, a4, a5}, a1, a2, a3, a4, a5 correspond to region bit sequences 1, 2, 3, 4, 5 respectively, and correspond to region gradient values {a1-56, a2-8, a3-50, a4-60, a5-5} respectively, if the preset adjustment coefficient is 0.8 and the preset gradient mean lower limit is 10, the gradient accumulation threshold can be calculated as 55 (55 is a divisor of the actual gradient accumulation threshold), when the initial sub-region a1 with the region bit sequence 1 is traversed, the corresponding accumulated gradient value is 56 which is greater than the gradient accumulation threshold 55, the region bit sequence 1 is confirmed as a target sub-region, the accumulated gradient value is set to zero, and the subsequent part of the gradient sequence is continued to traverse, when the initial sub-region a3 with the region bit sequence 3 is traversed, the accumulated gradient value is 8+50=58 which is greater than the gradient accumulation threshold 55, the initial sub-region a2 with the region bit sequence 2 and the initial sub-region a3 with the region bit sequence 3 are confirmed as a target sub-region, and the initial sub-region a4 with the region bit sequence 4 is confirmed as a target sub-region in the same way, when the initial sub-region a5 is traversed, the accumulated gradient value is 5 which is less than the gradient accumulation threshold, but at this time the traversal is completed, and therefore the initial sub-region a5 with the region bit sequence 5 is confirmed as a target sub-region. In the embodiment of the present application, the overlapping region is divided into a plurality of initial sub-regions, the gradient values of the initial sub-regions are calculated to quantify the texture intensity, and the gradient accumulation threshold is dynamically calculated. The initial sub-regions are intelligently merged according to the gradient accumulation threshold to form target sub-regions, which can finely register the high-texture region to improve the stitching quality, and coarsely merge the smooth region to optimize the efficiency, thereby improving the efficiency of the overlapping region stitching while ensuring the stitching quality.

[0126] S4, establishing an optimal stitching seam for each target sub-region in the plurality of target sub-regions to obtain a plurality of sub-region stitching seams.

[0127] It should be explained that the step of establishing an optimal stitching seam for each target sub-region in the plurality of target sub-regions to obtain a plurality of sub-region stitching seams comprises:

[0128] Target sub-regions are sequentially extracted from the plurality of target sub-regions, and the following operations are performed on the extracted target sub-regions:

[0129] The target sub-region is divided into a plurality of pixel screening rows, and the following operations are performed on each pixel screening row in the plurality of pixel screening rows:

[0130] A plurality of pixel points corresponding to the pixel screening row are set to obtain the plurality of pixel points;

[0131] The following operations are performed on each pixel point in the plurality of pixel points:

[0132] Calculate the color difference degree and the texture difference degree corresponding to the pixel point;

[0133] Correlate the color difference degree and the texture difference degree to obtain a pixel node, wherein the pixel node corresponds to the pixel point one by one;

[0134] Summarize the pixel nodes to obtain a pixel node set;

[0135] Match the target pixel point by using the pixel node set, wherein the target pixel point corresponds to the pixel screening row one by one;

[0136] Summarize the target pixel points to obtain a plurality of target pixel points, and the plurality of target pixel points form an optimal splicing seam;

[0137] Take the optimal splicing seam as a sub-region splicing seam, and summarize the sub-region splicing seams to obtain a plurality of sub-region splicing seams.

[0138] Specifically, the matching of the target pixel point by using the pixel node set comprises:

[0139] Calculate the mean value of the color difference degree corresponding to all pixel nodes in the pixel node set to obtain a reference color difference degree;

[0140] Calculate the mean value of the texture difference degree corresponding to all pixel nodes in the pixel node set to obtain a reference texture difference degree;

[0141] Extract the pixel nodes from the pixel node set one by one, and perform the following operations on the extracted pixel nodes:

[0142] Calculate the pixel energy value by using the pixel node, the reference color difference degree and the reference texture difference degree, wherein the pixel energy value is as follows:

[0143] ,

[0144] Wherein, E represents the pixel energy value, E k co represents the reference color difference degree, E k te represents the reference texture difference degree, E co represents the color difference degree corresponding to the pixel node, E te represents the texture difference degree corresponding to the pixel node, co represents the color identifier, and te represents the texture identifier;

[0145] Summarize the pixel energy values to obtain a plurality of pixel energy values, and the pixel point corresponding to the smallest pixel energy value in the plurality of pixel energy values is the target pixel point.

[0146] It can be understood that the optimal splicing seam is a continuous pixel path, which extends from the top to the bottom of the target sub-region through the overlapping region corresponding to the target sub-region, and the color and texture difference of all pixels on the pixel path are the smallest in the pixel screening row, so as to ensure that the visual trace of the image spliced and fused along the seam is the least obvious. Wherein, the pixel screening row is a set of pixels in the horizontal direction, only containing a row of transverse pixels, and the pixel is an indivisible minimum unit with unique position coordinates and specific color value in the pixel screening row, which is the basic element for constituting the pixel screening row and participating in the calculation of the splicing seam.

[0147] For example, assuming that there are currently three target sub-regions: B1, B2, and B3, and taking the target sub-region B1 as an example: the target sub-region B1 is divided into four pixel screening rows, b1, b2, b3, and b4. If the pixel screening row b1 contains 10 pixel points, the pixel energy value corresponding to each pixel point in the 10 pixel points is calculated, and the pixel point corresponding to the smallest pixel energy value is taken as the target pixel point. The reason is that when the pixel energy value of a certain pixel point is the smallest, it indicates that the visual difference of the splicing at the position of the pixel point is smaller, and the splicing trace is less obvious. The pixel energy value is a consistency measure value of the pixel point and the surrounding environment. The lower the pixel energy value, the less obvious the trace of the splicing at the pixel point, and the better the effect. The pixel energy value is a value calculated by using color difference degree and texture difference degree. The color difference degree is used to measure the color difference between the current pixel point and the surrounding area. The smaller the color difference degree, the higher the consistency of the pixel point and the surrounding color, and the less likely to produce color difference defects after splicing. Optionally, the process of obtaining the color difference degree includes but is not limited to RGB-SAD algorithm, CIELAB-ΔE algorithm, etc. Here, only the RGB-SAD algorithm is taken as an example: a symmetric neighborhood window is defined with the pixel point one as the center, a plurality of neighborhood pixel points (such as 3x3, including the pixel point one and eight neighborhood pixel points) are confirmed, for each neighborhood pixel point in the eight neighborhood pixel points: the absolute difference values of R, G, and B three channels and the corresponding channels of the pixel point one are calculated respectively, and then the sum of the three channel difference values is obtained to obtain the single-pixel difference sum of the neighborhood pixel point and the pixel point one. The final result is the color difference degree of the pixel point one. The texture difference degree is used to measure the texture difference between the current pixel point and the surrounding area. The smaller the texture difference degree, the more matched the texture structure of the pixel point and the surrounding texture, and the less likely to produce texture difference after splicing. Optionally, the process of obtaining the texture difference degree includes but is not limited to gradient amplitude difference method, gradient similarity measure method, etc. Here, only the gradient amplitude difference method is taken as an example: a symmetric neighborhood window is defined with the pixel point one as the center, a plurality of neighborhood pixel points (such as 3x3, including the pixel point one and eight neighborhood pixel points) are confirmed, for each neighborhood pixel point in the eight neighborhood pixel points: the gradient amplitude is calculated by using the gradient operator, the gradient amplitude absolute difference between the pixel point one and each neighborhood pixel point is calculated, and then the sum of all gradient amplitude absolute differences is calculated, and the result is the texture difference degree of the pixel point one.

[0148] It can be understood that the reference color difference degree is a color difference degree benchmark value for normalizing the color difference degree of the pixel point and converting it into a dimensionless relative proportion value, thereby eliminating the dimension effect. The reference texture difference degree is a texture difference degree benchmark value for normalizing the texture difference degree of the pixel point and converting it into a dimensionless relative proportion value, thereby eliminating the dimension effect.

[0149] It can be understood that the target pixel point is the pixel point with the minimum pixel energy value in the pixel screening row. According to the above example, the target sub-region B1 is divided into four pixel screening rows, b1, b2, b3 and b4. After the four pixel screening rows obtain the corresponding target pixel points, four target pixel points are obtained. The optimal stitching seam is composed of the four target pixel points. The optimal stitching seam is used as the sub-region stitching seam corresponding to the target sub-region B1. The acquisition process of the sub-region stitching seam corresponding to other target sub-regions is the same as that of the sub-region stitching seam corresponding to the target sub-region B1, and thus is not described herein. The embodiment of the present application independently calculates the optimal stitching seam for each target sub-region. The pixel screening mechanism with the minimum normalized energy in the pixel screening row is used to eliminate the dimensional difference of color and texture, ensure the continuity and consistency of the stitching seam in vision, realize the parallel optimization processing of multiple sub-regions, improve the precision and efficiency of large-scale image stitching, and improve the image fusion quality in a complex scene.

[0150] S5, sequentially connecting the multiple sub-region stitching seams to obtain a global stitching seam, fusing two stitching images corresponding to a stitching node by using the global stitching seam to obtain a seamless image, and obtaining a seamless panorama after each stitching node obtains a corresponding seamless image.

[0151] Understandably, the sub-region stitching seam is a pixel-level boundary line between two adjacent target sub-regions. Multiple sub-region stitching seams are connected sequentially. Optionally, a graph traversal vertex chain closure technique can be used to achieve the sequential connection of sub-regions and obtain a global stitching seam. Specifically, each target sub-region is considered a vertex of a graph, and the sub-region stitching seam between adjacent target sub-regions is considered an edge connecting the vertices. Starting from the first target sub-region, other target sub-regions are connected sequentially according to the principle of minimum cost, forming a non-repeating connection chain. The target sub-region at the end of the connection chain is connected to the first target sub-region, closing the vertex chain. The stitching seams on the chain constitute the global stitching seam. The global stitching seam is a complete pixel-level boundary line formed by connecting all sub-region stitching seams between adjacent stitching nodes in topological order. The process of using the global stitching seam to fuse the two stitched images corresponding to a stitching node refers to using the global stitching seam as a mask boundary, performing pixel-level gradient blending on the stitched images on both sides within a narrow band on both sides, making the seam invisible to the naked eye and outputting a seamless image. Optionally, multi-band fusion technology can be used to achieve the fusion of stitched images. The specific process is as follows: the stitched images on both sides of the global stitching seam are decomposed into different frequency bands (low frequency, mid frequency, and high frequency). The low-frequency band (reflecting overall brightness) is averaged and fused to maintain global consistency. The mid-to-high frequency bands (reflecting details and textures) are weighted and fused according to their distance from the global stitching seam. More pixels from the opposite side are taken at locations closer to the global stitching seam, and more pixels from the same side are taken at locations farther from the global stitching seam. Finally, the fusion results of each frequency band are reconstructed to obtain a seamless image, achieving an effect where the seam is invisible to the naked eye. The seamless image is a single local panoramic image after stitching seam mask gradient fusion. The seamless panoramic image is a complete panoramic image formed by stitching together the first and last stitches after all stitching nodes have completed seam mask gradient fusion.

[0152] S6. Construct a pixel coordinate system with the top left corner of the seamless panoramic image as the origin. The pixel coordinate system includes the x-axis and y-axis. Extract the edges of the seamless panoramic image and fit them into a quadrilateral to obtain the fitted quadrilateral. The fitted quadrilateral includes the coordinates of four fitted vertices.

[0153] Understandably, the pixel coordinate system is a two-dimensional Cartesian coordinate system constructed with the top-left corner of the seamless panoramic image as the origin, the horizontal direction to the right as the positive x-axis, and the vertical direction downwards as the positive y-axis. Each coordinate (x, y) corresponds to a pixel in the seamless panoramic image. Specifically, the coordinates of the four fitted vertices of the fitted quadrilateral, P1(x...y...), are... p1 y p1 P2(x) p2 y p2 P3(x) p3 y p3 P4(x) p4 y p4) respectively correspond to four pixel points in the seamless panorama. The process of extracting the edges of the seamless panorama and fitting them into a quadrangle can be realized by using Canny edge detection + Douglas-Peucker polygon approximation method, and the specific process is as follows: the seamless panorama is converted into grayscale and denoised by using Canny edge detection, and then the edge intensity and direction are calculated, after irrelevant pixels are removed, continuous edge pixels are screened out by double threshold, and then the Douglas-Peucker polygon approximation method is used to form an initial graph with the edge pixels, key vertices are found according to the threshold, the graph is recursively divided, and finally redundant vertices are removed to obtain a fitted quadrangle composed of four vertices. The fitted quadrangle is a convex quadrangle formed by four straight line segments to frame the image edge closest to the edge contour.

[0154] S7, constructing a target rectangle by using the seamless panorama and the fitted quadrangle, wherein the target rectangle includes four target vertex coordinates.

[0155] It should be explained that the construction of the target rectangle by using the seamless panorama and the fitted quadrangle includes:

[0156] confirming a stretching ratio interval by using the seamless panorama, wherein the stretching ratio interval includes a minimum stretching ratio and a maximum stretching ratio;

[0157] confirming S width values and C height values by using the fitted quadrangle;

[0158] combining the S width values and the C height values to obtain S*C distance nodes, wherein each distance node includes a width value and a height value;

[0159] performing the following operations on each distance node in the S*C distance nodes:

[0160] extracting the width value and the height value from the distance node;

[0161] calculating the ratio of the width value to the height value to obtain a fitted ratio;

[0162] if the fitted ratio is within the stretching ratio interval, confirming the distance node corresponding to the fitted ratio as a candidate distance node;

[0163] summarizing the candidate distance nodes to obtain a candidate distance node set;

[0164] confirming a target distance node by using the candidate distance node set and the seamless panorama;

[0165] constructing a target rectangle by using the width value and the height value corresponding to the target distance node.

[0166] It can be understood that the stretching ratio interval is a height-width ratio range determined according to the seamless panorama, which limits the shape of the target rectangle, avoids important image content distortion caused by excessive stretching or compression, and the target rectangle is a rectangle region with moderate height-width ratio, which can maximize the retention of important visual content and has no obvious geometric distortion, which is calculated according to the fitting quadrilateral and the seamless panorama.

[0167] For example, assuming that two width values (w1=800, w2=1000) and two height values (h1=600, h2=750) are extracted from the fitting quadrilateral, which can be combined into four distance nodes: Node1 (w=800, h=600), Node2 (w=800, h=750), Node3 (w=1000, h=600), Node4 (w=1000, h=750), and the fitting ratio of each distance node is calculated as: Node1: 4 / 3, Node2: 16 / 15, Node3: 5 / 3, Node4: 4 / 3, and if the stretching ratio interval is set to [1.2, 1.6], the fitting ratio of distance nodes Node1 and Node4 corresponding to the stretching ratio interval [1.2, 1.6] can be obtained, and distance nodes Node1 and Node4 are confirmed as candidate distance nodes, and the candidate distance node set is {Node1 (w=800, h=600), Node4 (w=1000, h=750)}.

[0168] It should be explained that the target distance node is confirmed by using the candidate distance node set and the seamless panorama, which includes:

[0169] The number of effective pixels corresponding to the seamless panorama is counted to obtain the total number of effective pixels;

[0170] The candidate distance nodes are extracted from the candidate distance node set in sequence, and if the candidate distance node meets the preset preferred condition, the candidate distance node is used as the preferred distance node;

[0171] The preferred distance nodes are summarized to obtain the preferred distance node set;

[0172] The product of the height value and the width value corresponding to each preferred distance node in the preferred distance node set is calculated to obtain the preferred value, wherein the preferred value corresponds to the preferred distance node one by one;

[0173] The preferred values are summarized to obtain the preferred value set;

[0174] The preferred distance node corresponding to the smallest preferred value in the preferred value set is used as the target preferred distance node;

[0175] Otherwise, a product of a height value and a width value corresponding to each candidate distance node in the candidate distance node set is calculated to obtain a candidate value, wherein the candidate value corresponds to the candidate distance node in one-to-one manner;

[0176] The candidate values are summarized to obtain a candidate value set;

[0177] The candidate distance node corresponding to the largest candidate value in the candidate value set is taken as a target candidate distance node;

[0178] The target distance node is taken as the target distance node.

[0179] In detail, the preferred condition is as follows:

[0180] ,

[0181] wherein w represents a width value corresponding to the candidate distance node, h represents a height value corresponding to the candidate distance node, A ref represents the total number of effective pixels, η represents a preset efficiency factor, and ref represents an effective identifier.

[0182] It can be understood that the total number of effective pixels is the sum of the number of all pixel points participating in the final imaging in the seamless panorama, that is, the number of pixels contained in the actual image region after excluding the invalid region (such as the black frame, the part of extreme distortion which cannot be repaired). The preset efficiency factor is a coefficient for adjusting the screening degree of the area of the candidate distance node, and the coefficient is a positive number greater than 1.

[0183] For example, it is assumed that the total number of effective pixels A refFor 1000000 pixels, the efficiency factor is 2, and the candidate distance node set is {Node1 (w=800, h=600), Node4 (w=1000, h=750)}. It can be seen that the candidate distance node Node1 does not meet the preferred condition (800*600 is less than 1000000 / 2), and the candidate distance node Node4 meets the preferred condition (1000*750 is greater than 1000000 / 2). Therefore, the preferred distance node set is {Node4 (w=1000, h=750)}, and at this time, the preferred distance node set contains only one preferred distance node, and the corresponding preferred value is 1000*750=75000. Therefore, the preferred distance node Node4 (w=1000, h=750) is taken as the target preferred distance node. It should be noted that there are usually multiple preferred distance nodes in the preferred distance node set, and the preferred value corresponding to each preferred distance node is calculated. The preferred distance node corresponding to the smallest preferred value in the preferred value set is taken as the target preferred distance node. The purpose is to: ensure that the core content of the seamless panorama is not lost, and to maximize the elimination of invalid, redundant and distorted edge regions, so as to obtain a panorama with compact composition and prominent main body. At the same time, the worst edge region is automatically discarded, effectively improving the overall texture and average quality of the panorama. Similarly, if there is no candidate distance node in the candidate distance node set that meets the preferred condition, the candidate value is calculated, and the candidate distance node corresponding to the largest candidate value in the candidate value set is taken as the target candidate distance node. The purpose is: when an ideal rectangle cannot be generated, the information of the seamless panorama is maximized to ensure that the core content of the seamless panorama is not lost. The target distance node is an optimal distance node finally selected from the candidate distance node set according to the proportional relationship between its area and the effective content of the panorama, and the width and height values corresponding to the target distance node are used to construct a target rectangle.

[0184] It should be noted that the process of constructing a target rectangle using the width value and the height value corresponding to the target distance node is: taking the width value and the height value corresponding to the target distance node as the length and the width respectively, determining a rectangular region with this size (length and width) and a central position on the seamless panorama. The rectangular region is the target rectangle. The target rectangle contains four target point coordinates, which are the coordinates of the four vertices of the target rectangle defined by the width value and the height value corresponding to the target distance node in the pixel coordinate system, which are the target and reference system for subsequent transformation. The embodiment of the present application realizes the intelligent generation of the target rectangle by setting the stretching ratio interval and the total number of effective pixels. By using a two-level screening strategy (preferred minimum area and guaranteed maximum area), the cutting efficiency and content retention of the seamless panorama are automatically balanced. On the premise of ensuring that the core picture of the seamless panorama is not lost, the distorted edge is maximally eliminated, and the composition quality and picture neatness are improved.

[0185] S8, calculating a target transformation matrix by using the four fitted vertex coordinates and the four target vertex coordinates, and re-mapping the seamless panorama by using the target transformation matrix to obtain a corrected standard rectangular panorama.

[0186] It should be explained that the calculating of the target transformation matrix by using the four fitted vertex coordinates and the four target vertex coordinates comprises:

[0187] The four fitted vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitted vertex coordinates and four normalized target vertex coordinates.

[0188] Based on the four normalized fitted vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set.

[0189] The initial transformation matrix is subjected to a hard constraint processing to obtain the target transformation matrix.

[0190] It can be understood that the purpose of normalizing the four fitted vertex coordinates and the four target vertex coordinates respectively is to scale the fitted vertex coordinate values and the target vertex coordinates to the same small range (such as ±1), eliminate the numerical difference of the pixel coordinates, and ensure that the obtained target transformation matrix is more accurate. The linear equation set is a linear system containing eight equations constructed by using the normalized fitted vertex coordinates and the four normalized target vertex coordinates. Optionally, the DLT algorithm is used as the preset algorithm to realize this process, which is a prior art and will not be described here. The initial transformation matrix is a projection transformation matrix obtained by solving the linear equation set, which is used to describe the optimal projection transformation from the normalized fitted vertex to the normalized target vertex image plane. Optionally, the least square method and the singular value decomposition method can be used to calculate the initial transformation matrix, and the specific calculation process will not be described here. The hard constraint processing is a post-processing of the initial transformation matrix, which forces it to meet specific and strict mathematical constraints. For the projection transformation, the most important hard constraint is to maintain the straight line, and the target transformation matrix is obtained. The target transformation matrix is a projection transformation matrix obtained after the hard constraint processing.

[0191] It should be understood that the re-mapping process refers to the process of repositioning the position of the pixel points in the seamless panorama to the corresponding position of a new image according to the mapping relationship provided by the target transformation matrix, and the final obtained image is the corrected standard rectangular panorama. The corrected standard rectangular panorama is a seamless panorama image that completely conforms to the target rectangular shape after geometric correction.

[0192] To solve the problems in the background art, the present application receives an ordered image sequence created by a user, wherein the ordered image sequence comprises: S1, S2, …, Si , …, S m m-1 stitching nodes are constructed by using the ordered image sequence, wherein the m-1 stitching nodes comprise (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each stitching node comprises two stitching images, the overlapping area corresponding to the stitching node is divided into a plurality of target sub-regions in the vertical direction, it can be seen that the overlapping area is divided into a plurality of initial sub-regions, the texture intensity of each initial sub-region is quantified by calculating the gradient value of the initial sub-region, and the gradient accumulation threshold is dynamically calculated. According to the gradient accumulation threshold, the initial sub-regions are intelligently merged to form the target sub-regions, the fine registration of the high-texture region is performed to improve the stitching quality, and the coarse-grained merging of the smooth region is performed to optimize the efficiency, the efficiency of the overlapping region stitching is improved on the basis of ensuring the stitching quality, an optimal stitching seam is established for each target sub-region in the plurality of target sub-regions, a plurality of sub-region stitching seams are obtained, it can be seen that the optimal stitching seam is independently calculated for each target sub-region, the pixel screening mechanism of minimizing the in-row normalized energy is used, the dimensional difference of color and texture is eliminated, the continuity and consistency of the stitching seam in vision are ensured, the parallel optimization processing of the plurality of sub-regions is realized, the precision and efficiency of large-scale image stitching are improved, and the image fusion quality in a complex scene is improved. The global stitching seam is obtained by sequentially connecting the plurality of sub-region stitching seams, the two stitching images corresponding to the stitching node are fused by using the global stitching seam to obtain a seamless image, the seamless panoramic image is obtained after each stitching node obtains the corresponding seamless image, and a pixel coordinate system is constructed with the upper left corner of the seamless panoramic image as the origin, wherein the pixel coordinate system comprises an x-axis and a y-axis. The edge of the seamless panoramic image is extracted and fitted into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 ). A target rectangle is constructed by using the seamless panoramic image and the fitted quadrilateral, wherein the target rectangle comprises four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 .), the target transformation matrix is calculated by using four fitting vertex coordinates and four target vertex coordinates, the seamless panorama is remapped by using the target transformation matrix, and the corrected standard rectangular panorama is obtained, so that the intelligent generation of the target rectangle is realized by setting the stretching ratio interval and the total number of effective pixels, the cutting efficiency and content reservation of the seamless panorama are automatically balanced by using the two-stage screening strategy (preferably the minimum area, and the maximum area), the distorted edge is maximally removed under the premise of ensuring the lossless core picture of the seamless panorama, and the composition quality and picture neatness are improved.

[0193] As shown in Figure 2 , it is a function module diagram of the panoramic stitching optimization system based on distortion correction provided by an embodiment of the present application.

[0194] The panoramic stitching optimization system based on distortion correction 100 can be installed in an electronic device. According to the functions to be implemented, the panoramic stitching optimization system based on distortion correction 100 can include a stitching node construction module 101, a stitching area division module 102, a global seamless stitching module 103, and a post-stitching correction module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and are stored in the memory of the electronic device.

[0195] The stitching node construction module 101 is configured to receive an ordered image sequence created by a user, wherein the ordered image sequence includes S1, S2, …, S i , …, S m stitching images, and each two adjacent stitching images includes an overlapping area.

[0196] m-1 stitching nodes are constructed by using the ordered image sequence, wherein the m-1 stitching nodes include (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each stitching node includes two stitching images.

[0197] The stitching area division module 102 is configured to perform the following operation on each stitching node in the m-1 stitching nodes:

[0198] The overlapping area corresponding to the stitching node is divided into a plurality of target sub-areas in the vertical direction.

[0199] The global seamless stitching module 103 is configured to establish an optimal stitching seam for each target sub-area in the plurality of target sub-areas, and obtain a plurality of sub-area stitching seams.

[0200] The plurality of sub-region splicing seams are sequentially connected to obtain a global splicing seam, the two splicing images corresponding to the splicing node are fused by using the global splicing seam to obtain a seamless image, and when the corresponding seamless image is obtained for each splicing node, a seamless panorama is obtained;

[0201] The post-splicing correction module 104 is configured to construct a pixel coordinate system with the upper left corner of the seamless panorama as an origin, wherein the pixel coordinate system includes an x-axis and a y-axis, extract edges of the seamless panorama and fit the edges into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral includes four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 ).

[0202] A target rectangle is constructed by using the seamless panorama and the fitted quadrilateral, wherein the target rectangle includes four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 ).

[0203] A target transformation matrix is calculated by using the four fitted vertex coordinates and the four target vertex coordinates, and the seamless panorama is remapped by using the target transformation matrix to obtain a corrected standard rectangular panorama.

[0204] In detail, the modules in the panorama splicing optimization system 100 based on distortion correction in the embodiments of the present application use the same technical means as the panorama splicing optimization method based on distortion correction in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0205] As shown in Figure 3 , it is a structural schematic diagram of an electronic device for implementing the panorama splicing optimization method based on distortion correction according to an embodiment of the present application.

[0206] The electronic device 1 can include a processor 10, a memory 11, and a bus 12, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a panorama splicing optimization method based on distortion correction program.

[0207] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used to store application software and various data installed in the electronic device 1, such as the code of the panoramic stitching optimization method based on distortion correction, and can also be used to temporarily store data that has been output or will be output.

[0208] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control core of the electronic device, and is connected to various components of the electronic device through various interfaces and lines, and executes programs or modules stored in the memory 11 (such as the panoramic stitching optimization method based on distortion correction) and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0209] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0210] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0211] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components, and preferably the power source can be logically connected to the at least one processor 10 through a power management device, so that the power management device implements functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0212] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0213] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0214] The distortion correction-based panorama stitching optimization method program stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when executed in the processor 10, can implement:

[0215] receiving an ordered image sequence created by a user, wherein the ordered image sequence includes: S1, S2, …, S i , …, S m stitching images, and each two adjacent stitching images contain an overlapping area;

[0216] constructing m-1 stitching nodes using the ordered image sequence, wherein the m-1 stitching nodes include: (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , Sm ), and each stitching node comprises two stitching images;

[0217] performing the following operation on each of the m-1 stitching nodes:

[0218] dividing the overlapping area corresponding to the stitching node into a plurality of target sub-areas in a vertical direction;

[0219] establishing an optimal stitching seam for each of the plurality of target sub-areas to obtain a plurality of sub-area stitching seams;

[0220] connecting the plurality of sub-area stitching seams in sequence to obtain a global stitching seam, fusing the two stitching images corresponding to the stitching node by using the global stitching seam to obtain a seamless image, and obtaining a seamless panoramic image when each stitching node obtains a corresponding seamless image;

[0221] constructing a pixel coordinate system with the upper left corner of the seamless panoramic image as an origin, wherein the pixel coordinate system comprises an x-axis and a y-axis, extracting an edge of the seamless panoramic image and fitting the edge into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 );

[0222] constructing a target rectangle by using the seamless panoramic image and the fitted quadrilateral, wherein the target rectangle comprises four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 );

[0223] calculating a target transformation matrix by using the four fitted vertex coordinates and the four target vertex coordinates, and performing remapping on the seamless panoramic image by using the target transformation matrix to obtain a corrected standard rectangular panoramic image.

[0224] Specifically, the specific implementation method of the processor 10 on the above instructions can refer to the description of the related steps in the corresponding embodiments, which will not be described here. Figures 1 to 3

[0225] ​Further, the modules / units integrated in the electronic device 1, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0226] The application further provides a computer-readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0227] receiving an ordered image sequence created by a user, wherein the ordered image sequence includes S1, S2, …, S i , …, S m spliced images, and each two adjacent spliced images contain an overlapping area;

[0228] constructing m-1 splicing nodes by using the ordered image sequence, wherein the m-1 splicing nodes include (S1, S2), (S2, S3), …, (S i-1 , S i ), …, (S m-1 , S m ), and each splicing node contains two spliced images;

[0229] performing the following operations on each splicing node in the m-1 splicing nodes:

[0230] dividing the overlapping area corresponding to the splicing node into a plurality of target sub-areas in the vertical direction;

[0231] establishing an optimal splicing seam for each target sub-area in the plurality of target sub-areas, to obtain a plurality of sub-area splicing seams;

[0232] connecting the plurality of sub-area splicing seams in sequence to obtain a global splicing seam, fusing the two spliced images corresponding to the splicing node by using the global splicing seam to obtain a seamless image, and obtaining a seamless panorama after each splicing node obtains a corresponding seamless image;

[0233] constructing a pixel coordinate system with the upper left corner of the seamless panorama as the origin, wherein the pixel coordinate system includes an x-axis and a y-axis, extracting an edge of the seamless panorama and fitting the edge into a quadrilateral to obtain a fitted quadrilateral, and obtaining four fitted vertex coordinates P1(x p1 , y p1 ), P2(x p2 , yp2 ), P3(x p3 , y p3 ), P4(x p4 , y p4 );

[0234] The target rectangle is constructed by using the seamless panorama and the fitting quadrangle, wherein the target rectangle comprises four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 );

[0235] The target transformation matrix is calculated by using the four fitting vertex coordinates and the four target vertex coordinates, and the seamless panorama is remapped by using the target transformation matrix to obtain the corrected standard rectangular panorama.

[0236] In several embodiments provided by the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the above-described system embodiments are merely illustrative; actual implementation can have another division manner.

[0237] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs.

[0238] In addition, the functional modules in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0239] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A distortion correction based panorama stitching optimization method, characterized in that, The method comprises: receiving an ordered image sequence created by a user, wherein the ordered image sequence comprises: S1, S2, …, S i , …, S m stitched images, and each two adjacent stitched images contain an overlapping area; m-1 splicing nodes are constructed by using the ordered image sequence, wherein the m-1 splicing nodes include (S1, S2), (S2, S3), …, (S i-1 , S i ) , …, (S m-1 , S m ) , and each splicing node contains two splicing images; The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: wherein G' represents the gradient accumulation threshold, G avg represents the gradient mean, G std represents the gradient value standard deviation, G0represents a preset lower limit of the gradient mean, γ represents a preset adjustment coefficient, avg represents a mean identifier, std represents a standard deviation identifier, and max() represents a maximum value; The method comprises: The method comprises: A pixel coordinate system is constructed with the upper left corner of the seamless panorama as the origin, wherein the pixel coordinate system comprises an x-axis and a y-axis, an edge of the seamless panorama is extracted and fitted as a quadrangle to obtain a fitted quadrangle, wherein the fitted quadrangle comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 ). A target rectangle is constructed using a seamless panorama and a fitted quadrangle, wherein the target rectangle includes four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 ). The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: The method comprises: Counting effective pixels corresponding to the seamless panorama picture to obtain a total number of effective pixels; Extracting a candidate distance node from the candidate distance node set in sequence, and taking the candidate distance node as the preferred distance node if the candidate distance node meets a preset preferred condition; Aggregating the preferred distance nodes to obtain a preferred distance node set; Calculating a product of a height value and a width value corresponding to each preferred distance node in the preferred distance node set to obtain a preferred value, wherein the preferred value corresponds to the preferred distance node in a one-to-one manner; Aggregating the preferred values to obtain a preferred value set; Taking the preferred distance node corresponding to the smallest preferred value in the preferred value set as a target preferred distance node; Otherwise, calculating a product of a height value and a width value corresponding to each candidate distance node in the candidate distance node set to obtain a candidate value, wherein the candidate value corresponds to the candidate distance node in a one-to-one manner; Aggregating the candidate values to obtain a candidate value set; Taking the candidate distance node corresponding to the largest candidate value in the candidate value set as a target candidate distance node; Taking the target preferred distance node or the target candidate distance node as a target distance node; The preferred condition is as follows: wherein w represents a width value corresponding to the candidate distance node, h represents a height value corresponding to the candidate distance node, A ref represents the total number of valid pixels, η represents a preset efficiency factor, and ref represents an effective identifier. Calculating a target transformation matrix by using the four fitting vertex coordinates and the four target vertex coordinates, and performing remapping on the seamless panorama picture by using the target transformation matrix to obtain a corrected standard rectangular panorama picture.

2. The distortion correction based panorama stitching optimization method of claim 1, wherein, The method further includes the following steps. Extracting a target sub-region from the plurality of target sub-regions in sequence, and performing the following operations on the extracted target sub-region. Dividing the target sub-region into a plurality of pixel screening rows, and performing the following operations on each pixel screening row in the plurality of pixel screening rows. Setting a pixel point corresponding to the pixel screening row to obtain a plurality of pixel points. Performing the following operations on each pixel point in the plurality of pixel points. Calculating a color difference degree and a texture difference degree corresponding to the pixel point. Associating the color difference degree and the texture difference degree to obtain a pixel node, wherein the pixel node corresponds to the pixel point in a one-to-one manner. Aggregating the pixel nodes to obtain a pixel node set. Matching a target pixel point by using the pixel node set, wherein the target pixel point corresponds to the pixel screening row in a one-to-one manner. Aggregating the target pixel points to obtain a plurality of target pixel points, and taking the plurality of target pixel points to form an optimal splicing seam. Aggregating the sub-region splicing seams to obtain the plurality of sub-region splicing seams.

3. The distortion correction based panorama stitching optimization method of claim 2, wherein, The method further includes the following steps. Calculating a mean value of the color difference degrees corresponding to all the pixel nodes in the pixel node set to obtain a reference color difference degree. Calculating a mean value of the texture difference degrees corresponding to all the pixel nodes in the pixel node set to obtain a reference texture difference degree. Extracting a pixel node from the pixel node set in sequence, and performing the following operations on the extracted pixel node. Calculating a pixel energy value by using the pixel node, the reference color difference degree, and the reference texture difference degree. Aggregating the pixel energy values to obtain a plurality of pixel energy values, and taking a pixel point corresponding to a smallest pixel energy value in the plurality of pixel energy values as a target pixel point.

4. The distortion correction based panorama stitching optimization method of claim 3, wherein, The method further includes the following steps. The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix.

5. A system for optimizing the stitching of a panorama based on distortion correction according to any one of claims 1 to 4, characterized in that, The system comprises: The stitching node construction module is configured to receive an ordered image sequence created by a user, wherein the ordered image sequence comprises: S1, S2, …, S i , …, S m stitching images, and each two adjacent stitching images contain an overlapping area. m-1 splicing nodes are constructed by using the ordered image sequence, wherein the m-1 splicing nodes include (S1, S2), (S2, S3), …, (S i-1 , i S ), …, (S m-1 , m S ), and each splicing node contains two splicing images; The system comprises: The system comprises: The system comprises: The system comprises: The stitching post-correction module is configured to construct a pixel coordinate system with the top-left corner of the seamless panorama as the origin, wherein the pixel coordinate system comprises an x-axis and a y-axis, extract edges of the seamless panorama and fit them into a quadrilateral to obtain a fitted quadrilateral, wherein the fitted quadrilateral comprises four fitted vertex coordinates: P1(x p1 , y p1 ), P2(x p2 , y p2 ), P3(x p3 , y p3 ), and P4(x p4 , y p4 ). A target rectangle is constructed using a seamless panorama and a fitted quadrangle, wherein the target rectangle includes four target vertex coordinates: T1(x t1 , y t1 ), T2(x t2 , y t2 ), T3(x t3 , y t3 ), and T4(x t4 , y t4 ). The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix. The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix. The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix. The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix. The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial transformation matrix is subjected to hard constraint processing to obtain a target transformation matrix. The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The system comprises: The four fitting vertex coordinates and the four target vertex coordinates are normalized respectively to obtain four normalized fitting vertex coordinates and four normalized target vertex coordinates; Based on the four normalized fitting vertex coordinates and the four normalized target vertex coordinates, a linear equation set is constructed by using a preset algorithm, and an initial transformation matrix is generated by using the linear equation set; The initial

Images