A dynamic image sequence splicing optimization method, system, device, medium and product

Abstract

The application discloses a dynamic image sequence splicing optimization method, system, device, medium and product, relates to the technical field of image splicing, acquires a dynamic image sequence, carries out particle filtering segmentation on the dynamic image sequence, obtains a plurality of target segmentation masks, carries out confidence fusion on each target segmentation mask based on the dynamic image sequence, obtains a corresponding fusion segmentation mask, carries out gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask, and obtains a corresponding splicing image. The existing dynamic image sequence splicing technology is overcome. Generally, the position of a stitching line is optimized through a graph cut, dynamic programming and the like, and then, mean filtering, bilateral filtering, guided filtering and the like are combined to realize smooth fusion in an overlapping area, but in a dynamic image sequence with frequent target motion and complex and changeable background, the technical problem that splicing accuracy and calculation efficiency are difficult to be considered together exists.

Description

Technical Field

[0001] This invention relates to the field of image stitching technology, and in particular to a method, system, device, medium and product for optimizing dynamic image sequence stitching. Background Technology

[0002] With the rapid development of computer vision and multimedia technologies, dynamic image sequence stitching has become a core supporting technology in fields such as panoramic imaging, video surveillance, autonomous driving, virtual reality, and remote sensing. This technology registers, aligns, and fuses image / video frames acquired from multiple perspectives or in continuous time sequences to generate panoramic images or videos with a wider field of view and more complete content, effectively overcoming the physical limitations of a single camera's field of view. Currently, existing dynamic image sequence stitching technologies generally optimize the stitching line position through methods such as graph cut and dynamic programming, and then combine mean filtering, bilateral filtering, and guided filtering to achieve smooth fusion of overlapping areas. However, in dynamic image sequences with frequent target movement and complex and changing backgrounds, there is a difficulty in balancing stitching accuracy and computational efficiency, making it unsuitable for the high real-time requirements of dynamic image sequence stitching. Summary of the Invention

[0003] This invention provides a method, system, device, medium, and product for optimizing dynamic image sequence stitching. It solves the problem that existing dynamic image sequence stitching technologies generally optimize the stitching line position through graph cutting, dynamic programming, etc., and then combine mean filtering, bilateral filtering, guided filtering, etc. to achieve smooth fusion of overlapping areas. However, in dynamic image sequences with frequent target movement and complex and variable backgrounds, it is difficult to balance stitching accuracy and computational efficiency, and thus cannot meet the technical requirements of high real-time dynamic image sequence stitching.

[0004] The first aspect of this invention provides a dynamic image sequence stitching optimization method, comprising:

[0005] A dynamic image sequence is acquired, and particle filtering is performed on the dynamic image sequence to obtain multiple target segmentation masks;

[0006] Based on the dynamic image sequence, confidence fusion is performed on each of the target segmentation masks to obtain the corresponding fused segmentation mask;

[0007] The dynamic image sequence is fused in the gradient domain according to the fusion segmentation mask to obtain the corresponding stitched image.

[0008] Optionally, the step of performing particle filtering segmentation on the dynamic image sequence to obtain multiple target segmentation masks includes:

[0009] Based on the pre-acquired historical initial distribution parameters, an initial particle set corresponding to each image to be segmented in the dynamic image sequence is constructed, wherein each particle in the initial particle set corresponds to a candidate segmentation mask;

[0010] The likelihood of each particle is evaluated separately to obtain the total likelihood of each particle.

[0011] Each initial particle set is sorted in descending order according to its total likelihood, resulting in multiple particle sequences.

[0012] According to the preset sampling quantity, multiple particles are sequentially selected from each particle sequence as coarse-grained screening results, and the Kmeans clustering algorithm is used to cluster the remaining particles in each particle sequence to obtain multiple fine-grained clustering results.

[0013] The coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are respectively used as the target particle set;

[0014] Candidate segmentation masks corresponding to the maximum total likelihood values ​​are selected from each of the target particle sets as target segmentation masks.

[0015] By adopting the above technical solution, a particle set initialization method based on historical initial distribution parameters is introduced in the particle filtering segmentation process. This effectively utilizes temporal information to improve the rationality and relevance of the initial particle distribution. At the same time, by conducting a comprehensive likelihood assessment of the particles and sorting them in descending order of total likelihood, high-matching and low-matching particles can be quickly distinguished. Based on this, a two-layer optimization strategy of coarse-grained screening and K-means fine-grained clustering is combined. This not only ensures the accuracy of the segmentation results by directly selecting high-likelihood particles, but also enriches the diversity of particle distribution by clustering the remaining particles, avoiding particle depletion and local optima problems. Finally, the candidate segmentation mask with the highest total likelihood is selected as the target segmentation mask. This can improve the efficiency of dynamic image segmentation while effectively enhancing the stability and accuracy of the segmentation results, making the target segmentation mask more closely match the real target contour.

[0016] Optionally, the step of evaluating the likelihood of each particle to obtain the total likelihood of each particle includes:

[0017] Each particle is input into a preset color likelihood function to obtain multiple color likelihood values;

[0018] Each particle is input into a preset edge likelihood function to obtain multiple edge likelihood values;

[0019] Each particle is input into a preset motion likelihood function to obtain multiple motion likelihood values;

[0020] The color likelihood value, edge likelihood value, and motion likelihood value are normalized to obtain multiple first likelihood values, multiple second likelihood values, and multiple third likelihood values.

[0021] Based on preset feature weights, the first likelihood value, second likelihood value and third likelihood value corresponding to each particle are weighted and calculated to obtain the total likelihood of each particle.

[0022] By employing the above technical solution, likelihood functions are constructed from three dimensions—color, edge, and motion—to evaluate particles. This comprehensively characterizes the matching degree between particles and real targets from multiple perspectives, including target appearance features, contour features, and dynamic motion characteristics. It avoids the problem of misjudgment caused by single features being susceptible to interference from lighting, deformation, and occlusion. By normalizing the color likelihood values, edge likelihood values, and motion likelihood values, calculation biases caused by differences in the dimensions and numerical ranges of different features can be eliminated, ensuring fairness and rationality when various features participate in fusion. Furthermore, by combining preset feature weights to perform weighted calculations to obtain the total likelihood, the contribution of key features can be flexibly highlighted, further improving the reliability and discriminative power of likelihood calculation, making particle sorting and selection more accurate.

[0023] Optionally, the step of fusing confidence scores of each target segmentation mask based on the dynamic image sequence to obtain a corresponding fused segmentation mask includes:

[0024] Based on each target segmentation mask and a preset probability fusion function, the foreground probability corresponding to each pixel position in the dynamic image sequence is determined;

[0025] When the foreground probability corresponding to the pixel position is greater than or equal to the preset foreground threshold, the preset first reference value is determined as the mask value corresponding to the pixel position.

[0026] When the foreground probability corresponding to the pixel position is less than the foreground threshold, the preset second reference value is determined as the mask value corresponding to the pixel position.

[0027] The corresponding fusion segmentation mask is generated using each of the mask values.

[0028] By employing the above technical solution, the foreground probability of each pixel position is calculated based on dynamic image sequences and a probability fusion function. This fully utilizes temporal information to comprehensively determine the confidence level of multi-frame target segmentation masks, effectively suppressing the impact of single-frame segmentation noise, target jitter, and local missegmentation, thus improving the stability of foreground region judgment. By setting a foreground threshold and marking overlapping pixels of the foreground and background with a first and a second reference value respectively, clear and regular binary mask construction can be achieved. The logic is simple and easy to implement, ensuring the consistency of segmentation boundaries. Finally, a fused segmentation mask is generated based on the pixel mask values ​​at each pixel position. This can remove discrete noise and false regions while preserving the complete outline of the target, making the segmentation results more robust and continuous.

[0029] Optionally, the step of performing gradient domain fusion on the dynamic image sequence based on the fusion segmentation mask to obtain the corresponding stitched image includes:

[0030] The dynamic image sequence is divided according to the fused segmentation mask to obtain the foreground region and background overlap region corresponding to each image to be segmented;

[0031] The Laplacian value corresponding to the position of each pixel in each of the background overlapping regions is determined based on a preset Laplacian operator function.

[0032] Construct the corresponding sparse coefficient matrix using each of the Laplacian values;

[0033] Based on the multigrid method, the preset discretized Poisson equations are iteratively solved according to the sparse coefficient matrix to obtain the corresponding pixel value vector.

[0034] The pixel value vector is then used for image reconstruction to obtain the corresponding fused background image;

[0035] The fused background image and each of the foreground regions are combined to obtain the corresponding stitched image.

[0036] By employing the aforementioned technical solution, the overlapping regions of the foreground and background are accurately segmented using a fusion segmentation mask as a constraint. This ensures that the foreground target remains undisturbed during subsequent fusion processing, while simultaneously focusing on the overlapping background region for efficient stitching. Extracting background gradient features using the Laplacian operator and constructing a sparse coefficient matrix accurately characterizes the texture and structural changes of the overlapping background region, reducing data redundancy and computational complexity. Furthermore, combining the multi-grid method with iterative solutions to the discretized Poisson equations enables rapid convergence and the acquisition of smoothly transitioning pixel value vectors, effectively improving the computational efficiency of gradient domain fusion and avoiding the slow convergence and long processing time of traditional methods. After image reconstruction, the fused background image is combined with the foreground region, resulting in a natural, seamless, and color-different stitching effect at the background stitching point. While maintaining clear foreground target contours and complete details, this significantly enhances the visual coherence and realism of the overall stitched image, making the stitching effect of dynamic image sequences more stable and reliable.

[0037] Optionally, the probability fusion function is specifically:

[0038] ;

[0039] ;

[0040] in, For pixels The prospect probability, The total number of images to be segmented. Let the frame weights be the k-th image to be segmented. The time decay factor, Let x be the index of the image to be segmented, and y be the first coordinate index of the pixel. For the k-th pixel in the image to be segmented The probability of the future.

[0041] A second aspect of the present invention provides a dynamic image sequence stitching optimization system, comprising:

[0042] The acquisition module is used to acquire dynamic image sequences, perform particle filtering segmentation on the dynamic image sequences, and obtain multiple target segmentation masks;

[0043] The confidence fusion module is used to perform confidence fusion on each of the target segmentation masks based on the dynamic image sequence to obtain the corresponding fused segmentation mask.

[0044] The gradient domain fusion module is used to perform gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask to obtain the corresponding stitched image.

[0045] The third aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the dynamic image sequence stitching optimization method described above.

[0046] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the dynamic image sequence stitching optimization method as described above.

[0047] The fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the dynamic image sequence stitching optimization method as described above.

[0048] As can be seen from the above technical solutions, the present invention has the following advantages:

[0049] This invention acquires a dynamic image sequence, performs particle filtering segmentation on the sequence to obtain multiple target segmentation masks, fuses these masks based on confidence scores to obtain a fused segmentation mask, and then performs gradient domain fusion on the dynamic image sequence using the fused segmentation mask to obtain a stitched image. This overcomes the technical challenge of existing dynamic image sequence stitching techniques, which typically optimize stitching line positions using graph cuts and dynamic programming, and then combine mean filtering, bilateral filtering, and guided filtering to achieve smooth fusion of overlapping areas. However, these techniques often struggle to balance stitching accuracy and computational efficiency in dynamic image sequences with frequent target movement and complex, variable backgrounds. Compared with traditional dynamic image sequence stitching methods, this invention first uses particle filtering to segment the dynamic image sequence to obtain target segmentation masks, which can stably identify foreground targets in dynamic scenes and reduce segmentation errors caused by noise in single-frame images and target deformation. Then, confidence fusion is performed on each target segmentation mask based on the dynamic image sequence to obtain the corresponding fused segmentation mask, which further improves the accuracy and temporal consistency of foreground and background segmentation and avoids jitter and discontinuity in the segmentation results. Finally, gradient domain fusion is performed on the dynamic image sequence based on the fused segmentation mask to obtain the corresponding stitched image. This not only preserves the details and contours of the foreground target, but also enables a smooth transition in the background area, eliminates stitching gaps and color differences, and improves stitching accuracy while ensuring processing efficiency. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart of the steps of a dynamic image sequence stitching optimization method provided in Embodiment 1 of the present invention;

[0052] Figure 2 This is a flowchart of the steps of a dynamic image sequence stitching optimization method provided in Embodiment 2 of the present invention;

[0053] Figure 3 This is a structural block diagram of a dynamic image sequence stitching optimization system provided in Embodiment 3 of the present invention;

[0054] Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0055] This invention provides a method, system, device, medium, and product for optimizing dynamic image sequence stitching. It addresses the problem that existing dynamic image sequence stitching technologies generally optimize the stitching line position through graph cutting, dynamic programming, etc., and then combine mean filtering, bilateral filtering, guided filtering, etc. to achieve smooth fusion of overlapping areas. However, in dynamic image sequences with frequent target movement and complex and variable backgrounds, it is difficult to balance stitching accuracy and computational efficiency, and thus cannot meet the technical requirements of high real-time dynamic image sequence stitching.

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations, and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0057] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a dynamic image sequence stitching optimization method provided in Embodiment 1 of the present invention.

[0058] This invention provides a dynamic image sequence stitching optimization method, comprising:

[0059] Step 101: Obtain the dynamic image sequence, perform particle filtering segmentation on the dynamic image sequence, and obtain multiple target segmentation masks.

[0060] A dynamic image sequence refers to a collection of multiple frames of images that are continuously acquired in chronological order and contain moving targets and dynamic scene changes.

[0061] Target segmentation mask refers to the mask generated by selecting the optimal particles from the target particle set.

[0062] In this embodiment of the invention, a dynamic image sequence is acquired. Based on pre-acquired historical initial distribution parameters, an initial particle set corresponding to each image to be segmented in the dynamic image sequence is constructed, wherein each particle in the initial particle set corresponds to a candidate segmentation mask. The total likelihood of each particle is calculated based on a preset color likelihood function, a preset edge likelihood function, and a preset motion likelihood function. Each initial particle set is sorted in descending order according to the total likelihood, resulting in multiple particle sequences. Multiple particles are sequentially selected from each particle sequence as coarse-grained screening results according to a preset sampling quantity. The remaining particles in each particle sequence are then clustered using the K-means clustering algorithm to obtain multiple fine-grained clustering results. The coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are used as target particle sets. The candidate segmentation mask corresponding to the maximum total likelihood is selected from each target particle set as the target segmentation mask.

[0063] It is worth mentioning that by performing particle filtering segmentation on dynamic image sequences, multiple target segmentation masks can be obtained, which can transform image segmentation into a target state estimation problem. By utilizing the multi-hypothesis tracking characteristics of particle filtering, the deformation and rapid motion of dynamic image sequences can be accurately adapted, effectively improving the segmentation accuracy and real-time performance of target boundaries in dynamic scenes, reducing the computational complexity of the segmentation process, overcoming the shortcomings of traditional static segmentation algorithms such as lag, ambiguity, and poor adaptability, and providing stable and reliable target segmentation masks for subsequent image fusion.

[0064] Step 102: Filtrate the confidence of each target segmentation mask based on the dynamic image sequence to obtain the corresponding fused segmentation mask.

[0065] Fusion segmentation mask refers to a stable binary mask generated after confidence fusion.

[0066] In this embodiment of the invention, corresponding frame weights are assigned to each target segmentation mask according to a time decay factor. Based on a preset probabilistic fusion function, the foreground probability corresponding to each pixel position in the dynamic image sequence is determined according to the frame weights. Each foreground probability is compared with a preset foreground threshold. When the foreground probability corresponding to a pixel position is greater than or equal to the preset foreground threshold, a preset first benchmark value is determined as the mask value corresponding to the pixel position. When the foreground probability corresponding to a pixel position is less than the foreground threshold, a preset second benchmark value is determined as the mask value corresponding to the pixel position. A fused segmentation mask with stronger robustness and matching to the dynamic image sequence is generated using each mask value.

[0067] It is worth mentioning that by fusing the confidence of each target segmentation mask based on the dynamic image sequence, the corresponding fused segmentation mask can be obtained. This can make full use of the temporal continuity information of the dynamic image sequence, assign higher weights to recent target segmentation masks according to the time decay rule, suppress single-frame segmentation noise and false detection interference, significantly improve the robustness and consistency of the segmentation mask, and obtain a more stable and accurate fused segmentation mask. This effectively avoids mask breakage and distortion caused by target jitter, sudden changes in illumination or local occlusion.

[0068] Step 103: Perform gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask to obtain the corresponding stitched image.

[0069] A stitched image refers to a seamless panoramic image obtained by combining the foreground area with a blended background image.

[0070] In this embodiment of the invention, each image to be segmented in the dynamic image sequence is divided into a foreground region and a background overlapping region based on a fusion segmentation mask. The Laplacian value of each pixel within the background overlapping region is calculated using the five-point difference method, and a corresponding sparse coefficient matrix is ​​constructed using these Laplacian values. Based on the multi-grid method, a preset discretized Poisson equation system is iteratively solved using the sparse coefficient matrix to obtain the corresponding pixel value vector. The pixel value vector is reconstructed into a smooth, seamless fused background image. Finally, the foreground target region is kept unchanged and combined with the fused background image to obtain the corresponding stitched image.

[0071] It should be noted that the five-point difference method refers to a numerical method used to approximate the Laplacian operator and quickly obtain pixel gradient changes.

[0072] It is worth mentioning that by performing gradient domain fusion on dynamic image sequences based on the fusion segmentation mask, the corresponding stitched image can be obtained. The fusion segmentation mask can distinguish the overlapping areas of the foreground target and the background, and only perform gradient smoothing on the overlapping areas of the background. While fully preserving the foreground texture and edge details, it eliminates stitching seams, artifacts and color distortion, avoids the edge blurring and gradient inversion problems caused by traditional fusion, achieves a natural transition of the stitched area, and improves the accuracy of dynamic image sequence stitching.

[0073] In this embodiment of the invention, a dynamic image sequence is acquired, and particle filtering is applied to the dynamic image sequence to obtain multiple target segmentation masks. Confidence fusion is then performed on each target segmentation mask based on the dynamic image sequence to obtain a corresponding fused segmentation mask. Gradient domain fusion is then performed on the dynamic image sequence based on the fused segmentation mask to obtain the corresponding stitched image. This overcomes the technical problem that existing dynamic image sequence stitching techniques generally optimize the stitching line position through graph cuts, dynamic programming, etc., and then combine mean filtering, bilateral filtering, and guided filtering to achieve smooth fusion of overlapping areas. However, in dynamic image sequences with frequent target movement and complex and variable backgrounds, it is difficult to balance stitching accuracy and computational efficiency. Compared with traditional dynamic image sequence stitching methods, this invention first uses particle filtering to segment the dynamic image sequence to obtain target segmentation masks, which can stably identify foreground targets in dynamic scenes and reduce segmentation errors caused by noise in single-frame images and target deformation. Then, confidence fusion is performed on each target segmentation mask based on the dynamic image sequence to obtain the corresponding fused segmentation mask, which further improves the accuracy and temporal consistency of foreground and background segmentation and avoids jitter and discontinuity in the segmentation results. Finally, gradient domain fusion is performed on the dynamic image sequence based on the fused segmentation mask to obtain the corresponding stitched image. This not only preserves the details and contours of the foreground target, but also enables a smooth transition in the background area, eliminates stitching gaps and color differences, and improves stitching accuracy while ensuring processing efficiency.

[0074] Please see Figure 2 , Figure 2 This is a flowchart illustrating the steps of a dynamic image sequence stitching optimization method provided in Embodiment 2 of the present invention.

[0075] This invention provides a dynamic image sequence stitching optimization method, comprising:

[0076] Step 201: Obtain the dynamic image sequence, perform particle filtering segmentation on the dynamic image sequence, and obtain multiple target segmentation masks.

[0077] Further, step 201 includes the following sub-steps:

[0078] S11. Based on the pre-acquired historical initial distribution parameters, construct the initial particle set corresponding to each image to be segmented in the dynamic image sequence, wherein each particle in the initial particle set corresponds to a candidate segmentation mask.

[0079] The historical initial distribution parameters refer to the target particle set of the previous frame of the image to be segmented for each image to be segmented. For example, in a dynamic image sequence including the first frame, the second frame, and the third frame, the historical initial distribution parameters of the second frame are the target particle set of the first frame.

[0080] The image to be segmented refers to a single frame in a dynamic image sequence that needs to be divided into foreground and background.

[0081] The initial particle set refers to the set of all particles generated during the initialization phase of particle filtering segmentation for each image to be segmented.

[0082] A particle is a basic unit used to represent a possible state of a target area, containing information such as outline, color, and motion.

[0083] Candidate segmentation mask refers to a foreground target binarization template generated by a single particle, which provides candidate schemes for the segmentation result.

[0084] In this embodiment of the invention, an initial particle set corresponding to each image to be segmented in the dynamic image sequence is constructed based on pre-acquired historical initial distribution parameters. For example, the dynamic image sequence includes a first frame image to be segmented, a second frame image to be segmented, and a third frame image to be segmented. For the first frame image to be segmented, an initial particle set is generated using a preset target detection algorithm (such as a pre-trained YOLO model, SSD algorithm, etc.). For the second frame image to be segmented, an initial particle set is generated based on the target particle set corresponding to the first frame image to be segmented, using a preset state transition model. For the third frame image to be segmented, an initial particle set is generated based on the target particle set corresponding to the second frame image to be segmented, using the state transition model.

[0085] It should be noted that the state transition model is as follows:

[0086] ;

[0087] ;

[0088] in, Let i be the i-th particle in the t-th frame of the image to be segmented. Let i be the i-th particle in the target particle set corresponding to the (t-1)-th frame of the image to be segmented. The state change generated by the motion prediction model. It is Gaussian noise. The process noise covariance matrix is... It follows a normal distribution.

[0089] It should be noted that the particle state vector is:

[0090] ;

[0091] in, Let be the state vector of the i-th particle. These are the coordinates of the control point of the i-th particle, used to describe the shape of the target contour. Let be the histogram feature of the i-th particle in the HSV color space, used to characterize the target texture. The motion vector estimated by the optical flow algorithm for the i-th particle includes displacement and deformation parameters.

[0092] S12. Evaluate the likelihood of each particle separately to obtain the total likelihood of each particle.

[0093] Furthermore, S12 includes the following sub-steps:

[0094] S121. Input each particle into the preset color likelihood function to obtain multiple color likelihood values.

[0095] Color likelihood value refers to the numerical value calculated using the color likelihood function, which measures the degree of similarity between a particle and a target in terms of color characteristics.

[0096] In this embodiment of the invention, each particle is input into a preset color likelihood function, and multiple color likelihood values ​​are obtained by calculating the histogram Bach coefficients of the corresponding region of the particle and the reference target model in the HSV color space.

[0097] It should be noted that the color likelihood function is specifically:

[0098] ;

[0099] in, Let be the color likelihood value of the i-th particle. For color weighting factors, Let b be the color histogram of the b-th histogram interval for the i-th particle. Let B be the color histogram of the b-th histogram interval in the reference target model, where B is the histogram interval number and b is the histogram interval index.

[0100] S122. Input each particle into the preset edge likelihood function to obtain multiple edge likelihood values.

[0101] The edge likelihood value refers to the value calculated by the edge likelihood function. The larger the value, the better the particle outline matches the target edge.

[0102] In this embodiment of the invention, each particle is input into a preset edge likelihood function, and multiple edge likelihood values ​​are obtained by statistically analyzing the degree of overlap between the particle contour and the real edge of the image.

[0103] It should be noted that the marginal likelihood function is specifically:

[0104] ;

[0105] in, Let be the edge likelihood value of the i-th particle. As the edge weight factor, pixel position Distance to the nearest edge point Let i be the particle boundary of the i-th particle. This is the index of the pixel position.

[0106] S123. Input each particle into the preset motion likelihood function to obtain multiple motion likelihood values.

[0107] Motion likelihood value refers to the numerical value output by the motion likelihood function, which is used to measure the degree of fit between the particle's motion state and the target's actual motion.

[0108] In this embodiment of the invention, the motion likelihood value corresponding to each particle is determined based on a preset motion likelihood function.

[0109] It should be noted that the motion likelihood function is specifically:

[0110] ;

[0111] in, Let be the motion likelihood value of the i-th particle. For motion weighting factors, Let be the prediction vector for the i-th particle. The average motion vector of the i-th particle region is calculated through the optical flow field.

[0112] S124. Normalize each color likelihood value, each edge likelihood value, and each motion likelihood value to obtain multiple first likelihood values, multiple second likelihood values, and multiple third likelihood values.

[0113] The first likelihood value refers to the standard matching degree value obtained after normalizing the color likelihood value.

[0114] The second likelihood value refers to the standard matching degree value obtained by normalizing the marginal likelihood value.

[0115] The third likelihood value refers to the standard matching degree value obtained after normalizing the motion likelihood value.

[0116] In this embodiment of the invention, the color likelihood values, edge likelihood values ​​and motion likelihood values ​​are subjected to maximum and minimum normalization to eliminate the differences in numerical magnitude between different feature dimensions, so that the matching degree of each dimension is uniformly mapped to the same numerical range, resulting in multiple first likelihood values, multiple second likelihood values ​​and multiple third likelihood values.

[0117] It should be noted that the expression for the first likelihood value is:

[0118] ;

[0119] in, The first likelihood value, It is the minimum value among all color likelihood values. The maximum value among all color likelihood values. This represents the color likelihood value.

[0120] The expression for the second likelihood value:

[0121] ;

[0122] in, The second likelihood value, The marginal likelihood value, The minimum value for each marginal likelihood. It represents the maximum value among all marginal likelihood values.

[0123] The expression for the third likelihood value:

[0124] ;

[0125] in, The third likelihood value, It is the minimum value among all motion likelihood values. Let the motion likelihood value be... It represents the maximum value among all motion likelihood values.

[0126] S125. Based on the preset feature weights, perform weighted calculations on the first likelihood value, second likelihood value and third likelihood value corresponding to each particle to obtain the total likelihood corresponding to each particle.

[0127] Feature weights refer to pre-set and adaptively adjustable weighting coefficients for color, edge, and motion features, used to balance the contribution of each feature.

[0128] Total likelihood refers to the final quantitative value of the overall fit between particles. The higher the value, the better the particle matches the real target.

[0129] In this embodiment of the invention, based on preset weighting coefficients for color, edge, and motion features, the first likelihood value, second likelihood value, and third likelihood value corresponding to each particle are weighted and summed respectively, and the three-dimensional feature matching degree is integrated into a single quantitative index to obtain the total likelihood corresponding to each particle.

[0130] It should be noted that the expression for the total likelihood is as follows:

[0131] ;

[0132] in, Let be the total likelihood corresponding to the i-th particle. These are weighting coefficients for color features. These are the weighting coefficients for edge features. These are the weighting coefficients for motion characteristics.

[0133] It is worth mentioning that by constructing likelihood functions from three dimensions—color, edge, and motion—to evaluate particles, the matching degree between particles and real targets can be comprehensively characterized from multiple perspectives, including target appearance features, contour features, and dynamic motion characteristics. This avoids the problem of misjudgment caused by single features being easily affected by lighting, deformation, and occlusion. By normalizing the color likelihood values, edge likelihood values, and motion likelihood values, the calculation bias caused by differences in the dimensions and numerical ranges of different features can be eliminated, ensuring the fairness and rationality of each feature participating in the fusion. Furthermore, by combining preset feature weights to perform weighted calculations to obtain the total likelihood, the contribution of key features can be flexibly highlighted, further improving the reliability and discriminativeness of the likelihood calculation, making particle sorting and selection more accurate.

[0134] S13. Sort each initial particle set in descending order according to each total likelihood to obtain multiple particle sequences.

[0135] A particle sequence refers to an ordered sequence of particles formed by sorting a set of particles according to their total likelihood.

[0136] In this embodiment of the invention, the initial particle set corresponding to each frame of the image to be segmented is sorted in descending order from high to low according to the total likelihood value of each particle. The high-quality particles with higher matching degree are arranged at the beginning of the sequence, and the particles with lower matching degree are placed at the end in turn, so as to obtain multiple particle sequences.

[0137] S14. Select multiple particles sequentially from each particle sequence according to the preset sampling quantity as coarse-grained screening results, and use the Kmeans clustering algorithm to cluster the remaining particles in each particle sequence to obtain multiple fine-grained clustering results.

[0138] The sampling quantity refers to the number of high-weight particles that need to be retained for coarse-grained screening, which is usually set to 20% of the total number of particles.

[0139] Coarse-grained screening results refer to the set of high-matching, high-quality particles directly selected from the beginning of the particle sequence.

[0140] Fine-grained clustering results refer to the supplementary particle set consisting of the retained central particles and newly generated particles after clustering optimization.

[0141] In this embodiment of the invention, high total likelihood particles are selected sequentially from each particle sequence according to a preset sampling number as coarse-grained screening results. At the same time, the Kmeans clustering algorithm is used to cluster the remaining unselected particles in each particle sequence according to their spatial distribution, retaining the central particle of each class and adding new particles in low-weight regions to avoid local optima, thus obtaining multiple fine-grained clustering results.

[0142] It should be noted that the K-means clustering algorithm works as follows: First, determine the number of cluster centers based on the total number of particles. Randomly select a corresponding number of particles as initial cluster centers. Then, calculate the weighted distance between each remaining particle and each cluster center in terms of contour, color, and motion features. Assign the particles to the nearest cluster center to form multiple clusters. Subsequently, update the cluster centers with the mean state of all particles in each cluster. Iterate the assignment and update steps repeatedly until the cluster centers no longer change. Finally, retain the central particle of each cluster as the fine-grained clustering result.

[0143] The expression for the number of cluster centers is as follows:

[0144] ;

[0145] ;

[0146] in, The number of cluster centers. This represents the total number of particles. For the number of samples, This represents the proportion of the number of samples.

[0147] S15. The coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are respectively used as the target particle set.

[0148] The target particle set refers to the final optimized particle set obtained by merging the coarse-grained screening results and the fine-grained clustering results.

[0149] In this embodiment of the invention, the coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are merged to form multiple target particle sets that take into account both high matching degree and distribution diversity.

[0150] S16. Select the candidate segmentation mask corresponding to the maximum total likelihood from each target particle set as the target segmentation mask.

[0151] In this embodiment of the invention, all particles in each target particle set are traversed, and the optimal particle with the largest total likelihood value is found and determined. The candidate segmentation mask corresponding to the optimal particle is directly used as the target segmentation mask.

[0152] It is worth mentioning that by introducing a particle set initialization method based on historical initial distribution parameters during particle filtering segmentation, the rationality and relevance of the initial particle distribution can be effectively improved by utilizing temporal information. At the same time, by conducting a comprehensive likelihood assessment of the particles and sorting them in descending order of total likelihood, high-matching and low-matching particles can be quickly distinguished. On this basis, a two-layer optimization strategy of coarse-grained screening and K-means fine-grained clustering can ensure the accuracy of the segmentation results by directly selecting high-likelihood particles, and enrich the diversity of particle distribution by clustering the remaining particles, avoiding particle depletion and local optima problems. Finally, the candidate segmentation mask with the highest total likelihood is selected as the target segmentation mask, which can improve the efficiency of dynamic image segmentation while effectively enhancing the stability and accuracy of the segmentation results, making the target segmentation mask more closely fit the real target contour.

[0153] Step 202: Determine the foreground probability corresponding to each pixel position in the dynamic image sequence based on each target segmentation mask and the preset probability fusion function.

[0154] A pixel position refers to the coordinates of a single basic unit on the two-dimensional plane of an image; it is the smallest unit that makes up an image.

[0155] The foreground probability corresponding to a pixel location refers to the comprehensive confidence value of a pixel belonging to the target foreground region in multiple consecutive frames, calculated by a probability fusion function.

[0156] In this embodiment of the invention, based on the foreground probability marked by each target segmentation mask and a preset probability fusion function, the cumulative probability of being determined as foreground in the target segmentation mask is calculated for each pixel position. At the same time, a time decay factor is introduced to assign frame weights to the foreground probability of each pixel in the image to be segmented, so as to obtain the foreground probability corresponding to each pixel position.

[0157] It should be noted that the probability fusion function is specifically as follows:

[0158] ;

[0159] ;

[0160] in, For pixels The prospect probability, The total number of images to be segmented. Let the frame weights be the k-th image to be segmented. The time decay factor, Let x be the index of the image to be segmented, and y be the first coordinate index of the pixel. For the k-th pixel in the image to be segmented The probability of the future.

[0161] Step 203: When the foreground probability corresponding to the pixel position is greater than or equal to the preset foreground threshold, the preset first reference value is determined as the mask value corresponding to the pixel position.

[0162] Mask value refers to the numerical identifier assigned to each pixel in an image, used to distinguish between foreground and background regions, and constitutes the core data of the fusion segmentation mask.

[0163] The first reference value refers to a pre-set fixed value (usually 1) used to mark pixels as foreground regions, and is a unified identifier for foreground regions in the fusion segmentation mask.

[0164] The foreground threshold refers to a pre-set critical value used to distinguish whether a pixel belongs to the foreground or the background. It is the core basis for determining the mask value type and can be flexibly adjusted according to the needs of dynamic scenes.

[0165] In this embodiment of the invention, when the foreground probability corresponding to a pixel location is greater than or equal to a preset foreground threshold, the preset first reference value is determined as the mask value corresponding to the pixel location, thereby marking that the pixel location belongs to the foreground region.

[0166] Step 204: When the foreground probability corresponding to the pixel position is less than the foreground threshold, the preset second reference value is determined as the mask value corresponding to the pixel position.

[0167] The second reference value refers to a pre-set fixed value (usually 0) used to mark pixels as background areas. It corresponds to the first reference value used to mark foreground areas, forming a complete mask value system.

[0168] In this embodiment of the invention, when the foreground probability corresponding to a pixel location is less than the foreground threshold, a preset second reference value is determined as the mask value corresponding to the pixel location, thereby marking that the pixel location belongs to the background overlapping area.

[0169] Step 205: Generate the corresponding fusion segmentation mask using each mask value.

[0170] In this embodiment of the invention, a corresponding fusion segmentation mask is generated using the mask values ​​corresponding to each pixel (including the first reference value of the foreground region and the second reference value of the background overlapping region).

[0171] It is worth mentioning that by calculating the foreground probability of each pixel position based on dynamic image sequences and a probability fusion function, temporal information can be fully utilized to comprehensively determine the confidence of multi-frame target segmentation masks, effectively suppressing the impact of single-frame segmentation noise, target jitter, and local missegmentation, and improving the stability of foreground region judgment. By setting a foreground threshold and marking overlapping pixels of the foreground and background with a first reference value and a second reference value respectively, clear and regular binary mask construction can be achieved. The logic is simple and easy to implement, ensuring the consistency of segmentation boundaries. Finally, a fused segmentation mask is generated based on the pixel mask values ​​of each pixel position, which can remove discrete noise and false regions while preserving the complete outline of the target, making the segmentation results more robust and continuous.

[0172] Step 206: Perform gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask to obtain the corresponding stitched image.

[0173] Furthermore, step 206 includes the following sub-steps:

[0174] S21. Divide each image to be segmented in the dynamic image sequence according to the fused segmentation mask to obtain the foreground region and background overlap region corresponding to each image to be segmented.

[0175] The foreground region refers to the set of pixels marked as foreground in the fused segmentation mask, which corresponds to the target subject region in the image and must retain its detailed features completely.

[0176] Background overlap region refers to the set of pixels marked as background overlap in the fusion segmentation mask, which is the area in multi-frame images that can be smoothly stitched together.

[0177] In this embodiment of the invention, based on the foreground and background division criteria defined by the fusion segmentation mask, each image to be segmented in the dynamic image sequence is determined pixel by pixel, and the pixel regions marked as foreground in the fusion segmentation mask are integrated into the foreground regions of the image to be segmented. At the same time, the pixel regions marked as background overlap in the mask are classified as background overlap regions.

[0178] S22. Determine the Laplacian value corresponding to the position of each pixel in each background overlapping region based on the preset Laplacian operator function.

[0179] The Laplacian value is a numerical value calculated using the Laplacian operator function. It is used to quantify the degree of grayscale change of a pixel, reflect the edge features of the pixel's location, and provide data support for the smoothing of background stitching.

[0180] In this embodiment of the invention, based on a preset Laplacian operator function, each pixel in the divided background overlapping area is calculated one by one to determine the Laplacian value corresponding to each pixel position.

[0181] It should be noted that the Laplace operator function is specifically as follows:

[0182] ;

[0183] in, For image u at pixel position The Laplace value, For image u at pixel position grayscale value, For image u at pixel position grayscale value, For image u at pixel position grayscale value, For image u at pixel position grayscale value, For image u at pixel position grayscale value, Let be the pixel spacing, u be the pixel value vector of the merged background image, x be the first coordinate index, and y be the second coordinate index.

[0184] S23. Construct the corresponding sparse coefficient matrix using each Laplacian value.

[0185] A sparse coefficient matrix is ​​a matrix constructed with Laplacian values ​​as the core elements. It retains only the correlation coefficients of the pixel itself and its neighboring pixels, with zero redundant elements. It is sparse and is used to characterize the pixel feature correlations in overlapping background regions.

[0186] In this embodiment of the invention, the Laplacian value of each pixel location is used as the core element, and a sparse coefficient matrix is ​​constructed according to the pixel coordinate order. Only the Laplacian correlation coefficients of the pixel itself and its neighboring pixels are retained, and invalid and redundant elements are eliminated. This enables the matrix to accurately represent the grayscale changes and edge distribution characteristics of the background overlapping area, and provides efficient matrix support for the smooth calculation of subsequent gradient domain fusion.

[0187] S24. Based on the multigrid method, the preset discretized Poisson equations are solved iteratively according to the sparse coefficient matrix to obtain the corresponding pixel value vector.

[0188] A pixel value vector is a vector composed of the calculated values ​​of all background pixels, corresponding to the specific value of each background pixel in the image.

[0189] In this embodiment of the invention, based on the multi-grid method and combined with a sparse coefficient matrix, the preset discretized Poisson equations are solved iteratively in layers. First, a coarse grid layer is used for rapid iteration to reduce errors, and then the grid precision is gradually refined. Through multiple iterations, the corresponding pixel value vector is obtained. This pixel value vector can be directly used to generate a background fusion image, achieving a smooth transition in the background region.

[0190] It should be noted that the discretized Poisson equations are as follows:

[0191] ;

[0192] in, It is a sparse coefficient matrix. To merge the pixel value vectors of the background image, is the constant vector on the right.

[0193] It should be noted that the multigrid method is an efficient numerical solution method. By iterating through layered grids (gradually refining from coarse to fine grids), it can quickly solve the discretized Poisson equations, improving the solution efficiency and accuracy, and is suitable for the smooth blending of background areas.

[0194] S25. Reconstruct the pixel value vector to obtain the corresponding fused background image.

[0195] In this embodiment of the invention, based on the pixel value vector and combined with the constraints of the region division of the fusion segmentation mask, the values ​​corresponding to each pixel are accurately mapped to the image pixel positions, and the background overlapping area is reconstructed to obtain the corresponding fused background image.

[0196] It should be noted that image reconstruction refers to the process of recombining and restoring discrete pixel values ​​into a visual image based on pixel value vectors. The core is to map the obtained pixel values ​​to specific image coordinates to restore the texture and details of the image.

[0197] It is worth mentioning that guided filtering can be introduced during the gradient domain fusion process. This can effectively maintain the edge contours of the foreground target and the background region while smoothing and optimizing the overlapping background area, avoiding the boundary blurring and detail loss problems caused by conventional filtering. Guided filtering can quickly reduce noise and smooth the Laplacian calculation results, sparse coefficient matrix, or fused background image, further suppressing stitching artifacts caused by noise and gradient abrupt changes, making the background transition more uniform and natural. Compared with traditional filtering methods, guided filtering has higher computational efficiency and stronger adaptability. Combined with the multigrid method and Poisson solution process, it can significantly improve the overall fusion speed, enhance the real-time performance of the algorithm while ensuring the visual effect of the stitched image, and make the final stitched image clear and sharp in the foreground and smooth and coherent in the background, further improving the overall robustness and imaging quality.

[0198] S26. Combine the background image and each foreground region to obtain the corresponding stitched image.

[0199] In this embodiment of the invention, the background area and each foreground area are precisely aligned and smoothly connected to eliminate splicing marks between different frame images and generate corresponding spliced ​​images.

[0200] It is worth mentioning that by accurately dividing the foreground and background overlapping regions using a fusion segmentation mask as a constraint, the foreground target can be completely preserved without interference in subsequent fusion processing, while focusing on the background overlapping region to achieve efficient stitching. By extracting background gradient features using the Laplacian operator and constructing a sparse coefficient matrix, the texture and structural changes of the background overlapping region can be accurately characterized, reducing data redundancy and computational complexity. Simultaneously, by combining the multi-grid method to iteratively solve the discretized Poisson equations, rapid convergence and a smoothly transitioning pixel value vector can be obtained, effectively improving the computational efficiency of gradient domain fusion and avoiding the slow convergence and long processing time of traditional methods. After image reconstruction to obtain the fused background image, it is then combined with the foreground region, resulting in a natural, seamless, and color-different stitching effect at the background stitching point. While ensuring clear foreground target contours and complete details, it significantly improves the visual coherence and realism of the overall stitched image, making the stitching effect of dynamic image sequences more stable and reliable.

[0201] In this embodiment of the invention, a dynamic image sequence is acquired, and particle filtering is applied to the dynamic image sequence to obtain multiple target segmentation masks. Confidence fusion is then performed on each target segmentation mask based on the dynamic image sequence to obtain a corresponding fused segmentation mask. Gradient domain fusion is then performed on the dynamic image sequence based on the fused segmentation mask to obtain the corresponding stitched image. This overcomes the technical problem that existing dynamic image sequence stitching techniques generally optimize the stitching line position through graph cuts, dynamic programming, etc., and then combine mean filtering, bilateral filtering, and guided filtering to achieve smooth fusion of overlapping areas. However, in dynamic image sequences with frequent target movement and complex and variable backgrounds, it is difficult to balance stitching accuracy and computational efficiency. Compared with traditional dynamic image sequence stitching methods, this invention first uses particle filtering to segment the dynamic image sequence to obtain target segmentation masks, which can stably identify foreground targets in dynamic scenes and reduce segmentation errors caused by noise in single-frame images and target deformation. Then, confidence fusion is performed on each target segmentation mask based on the dynamic image sequence to obtain the corresponding fused segmentation mask, which further improves the accuracy and temporal consistency of foreground and background segmentation and avoids jitter and discontinuity in the segmentation results. Finally, gradient domain fusion is performed on the dynamic image sequence based on the fused segmentation mask to obtain the corresponding stitched image. This not only preserves the details and contours of the foreground target, but also enables a smooth transition in the background area, eliminates stitching gaps and color differences, and improves stitching accuracy while ensuring processing efficiency.

[0202] Please see Figure 3 , Figure 3 This is a structural block diagram of a dynamic image sequence stitching optimization system provided in Embodiment 3 of the present invention.

[0203] This invention provides a dynamic image sequence stitching optimization system, comprising:

[0204] Acquisition module 301 is used to acquire dynamic image sequences, perform particle filtering segmentation on the dynamic image sequences, and obtain multiple target segmentation masks;

[0205] The confidence fusion module 302 is used to perform confidence fusion on each target segmentation mask based on the dynamic image sequence to obtain the corresponding fused segmentation mask.

[0206] The gradient domain fusion module 303 is used to perform gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask to obtain the corresponding stitched image.

[0207] Furthermore, the acquisition module 301 includes:

[0208] The construction submodule is used to construct the initial particle set corresponding to each image to be segmented in the dynamic image sequence based on the pre-acquired historical initial distribution parameters. Each particle in the initial particle set corresponds to a candidate segmentation mask.

[0209] The evaluation submodule is used to evaluate the likelihood of each particle separately and obtain the total likelihood of each particle.

[0210] The first screening submodule is used to sort each initial particle set in descending order according to each total likelihood, so as to obtain multiple particle sequences.

[0211] The second screening submodule is used to sequentially select multiple particles from each particle sequence according to a preset sampling number as coarse-grained screening results, and use the Kmeans clustering algorithm to cluster the remaining particles in each particle sequence to obtain multiple fine-grained clustering results.

[0212] The coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are respectively used as the target particle set;

[0213] The third filtering submodule is used to select the candidate segmentation mask corresponding to the maximum total likelihood from each target particle set as the target segmentation mask.

[0214] Furthermore, the evaluation submodule includes:

[0215] The color likelihood unit is used to input each particle into a preset color likelihood function to obtain multiple color likelihood values;

[0216] The edge likelihood unit is used to input each particle into a preset edge likelihood function to obtain multiple edge likelihood values;

[0217] The motion likelihood unit is used to input each particle into a preset motion likelihood function to obtain multiple motion likelihood values;

[0218] The normalization unit is used to normalize each color likelihood value, each edge likelihood value, and each motion likelihood value to obtain multiple first likelihood values, multiple second likelihood values, and multiple third likelihood values.

[0219] The weighting unit is used to perform weighted operations on the first likelihood value, second likelihood value and third likelihood value corresponding to each particle based on preset feature weights, so as to obtain the total likelihood of each particle.

[0220] Furthermore, the confidence fusion module 302 includes:

[0221] The foreground probability submodule is used to determine the foreground probability corresponding to each pixel position in the dynamic image sequence based on each target segmentation mask and a preset probability fusion function.

[0222] The binarization submodule is used to determine the preset first reference value as the mask value corresponding to the pixel position when the foreground probability corresponding to the pixel position is greater than or equal to the preset foreground threshold.

[0223] When the foreground probability corresponding to a pixel location is less than the foreground threshold, the preset second reference value is determined as the mask value corresponding to the pixel location.

[0224] The corresponding fusion segmentation mask is generated using each mask value.

[0225] Furthermore, the gradient domain fusion module 303 includes:

[0226] The segmentation submodule is used to segment each image to be segmented in the dynamic image sequence according to the fused segmentation mask, so as to obtain the foreground region and background overlap region corresponding to each image to be segmented;

[0227] The Laplacian submodule is used to determine the Laplacian value corresponding to the position of each pixel in each background overlapping region based on a preset Laplacian operator function;

[0228] The sparse coefficients submodule is used to construct the corresponding sparse coefficient matrix using each Laplacian value;

[0229] The analysis submodule is used to iteratively solve the preset discretized Poisson equations based on the multigrid method and the sparse coefficient matrix to obtain the corresponding pixel value vector.

[0230] The reconstruction submodule is used to reconstruct the image from the pixel value vector to obtain the corresponding fused background image.

[0231] The combination submodule is used to combine the background image and various foreground regions to obtain the corresponding stitched image.

[0232] Furthermore, the probability fusion function is specifically as follows:

[0233] ;

[0234] ;

[0235] in, For pixels The prospect probability, The total number of images to be segmented. Let the frame weights be the k-th image to be segmented. The time decay factor, Let x be the index of the image to be segmented, and y be the first coordinate index of the pixel. For the k-th pixel in the image to be segmented The probability of the future.

[0236] Please see Figure 4 , Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4 of the present invention.

[0237] An electronic device according to an embodiment of the present invention includes: a memory 401 and a processor 402. The memory 401 stores a computer program. When the computer program is executed by the processor 402, the processor 402 performs the dynamic image sequence stitching optimization method as described in any of the above embodiments.

[0238] Memory 401 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 401 has storage space 403 for program code 413 for performing any of the method steps described above. For example, storage space 403 for program code may include individual program codes 413 for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When run by a computing processing device, this code causes the computing processing device to perform the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When this code is run by a computing device, it causes the computing device to perform the various steps in the dynamic image sequence stitching optimization method described above.

[0239] Embodiment 5 of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic image sequence stitching optimization method as described in any of the above embodiments.

[0240] Embodiment 6 of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the dynamic image sequence stitching optimization method as described in any of the above embodiments.

[0241] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0242] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0243] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0244] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0245] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0246] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic image sequence stitching optimization method, characterized in that, include: A dynamic image sequence is acquired, and particle filtering is performed on the dynamic image sequence to obtain multiple target segmentation masks; Based on the dynamic image sequence, confidence fusion is performed on each of the target segmentation masks to obtain the corresponding fused segmentation mask; The dynamic image sequence is fused in the gradient domain according to the fusion segmentation mask to obtain the corresponding stitched image.

2. The dynamic image sequence stitching optimization method according to claim 1, characterized in that, The step of performing particle filtering segmentation on the dynamic image sequence to obtain multiple target segmentation masks includes: Based on the pre-acquired historical initial distribution parameters, an initial particle set corresponding to each image to be segmented in the dynamic image sequence is constructed, wherein each particle in the initial particle set corresponds to a candidate segmentation mask; The likelihood of each particle is evaluated separately to obtain the total likelihood of each particle. Each initial particle set is sorted in descending order according to its total likelihood, resulting in multiple particle sequences. According to the preset sampling quantity, multiple particles are sequentially selected from each particle sequence as coarse-grained screening results, and the Kmeans clustering algorithm is used to cluster the remaining particles in each particle sequence to obtain multiple fine-grained clustering results. The coarse-grained screening results and fine-grained clustering results corresponding to each particle sequence are respectively used as the target particle set; Candidate segmentation masks corresponding to the maximum total likelihood values ​​are selected from each of the target particle sets as target segmentation masks.

3. The dynamic image sequence stitching optimization method according to claim 2, characterized in that, The step of evaluating the likelihood of each particle to obtain the total likelihood of each particle includes: Each particle is input into a preset color likelihood function to obtain multiple color likelihood values; Each particle is input into a preset edge likelihood function to obtain multiple edge likelihood values; Each particle is input into a preset motion likelihood function to obtain multiple motion likelihood values; The color likelihood value, edge likelihood value, and motion likelihood value are normalized to obtain multiple first likelihood values, multiple second likelihood values, and multiple third likelihood values. Based on preset feature weights, the first likelihood value, second likelihood value and third likelihood value corresponding to each particle are weighted and calculated to obtain the total likelihood of each particle.

4. The dynamic image sequence stitching optimization method according to claim 1, characterized in that, The step of fusing confidence scores of each target segmentation mask based on the dynamic image sequence to obtain a corresponding fused segmentation mask includes: Based on each target segmentation mask and a preset probability fusion function, the foreground probability corresponding to each pixel position in the dynamic image sequence is determined; When the foreground probability corresponding to the pixel position is greater than or equal to the preset foreground threshold, the preset first reference value is determined as the mask value corresponding to the pixel position. When the foreground probability corresponding to the pixel position is less than the foreground threshold, the preset second reference value is determined as the mask value corresponding to the pixel position. The corresponding fusion segmentation mask is generated using each of the mask values.

5. The dynamic image sequence stitching optimization method according to claim 1, characterized in that, The step of performing gradient domain fusion on the dynamic image sequence based on the fusion segmentation mask to obtain the corresponding stitched image includes: The dynamic image sequence is divided according to the fused segmentation mask to obtain the foreground region and background overlap region corresponding to each image to be segmented; The Laplacian value corresponding to the position of each pixel in each of the background overlapping regions is determined based on a preset Laplacian operator function. Construct the corresponding sparse coefficient matrix using each of the Laplacian values; Based on the multigrid method, the preset discretized Poisson equations are iteratively solved according to the sparse coefficient matrix to obtain the corresponding pixel value vector. The pixel value vector is then used for image reconstruction to obtain the corresponding fused background image; The fused background image and each of the foreground regions are combined to obtain the corresponding stitched image.

6. The dynamic image sequence stitching optimization method according to claim 4, characterized in that, The probability fusion function is specifically: ； ； in, For pixels The prospect probability, The total number of images to be segmented. Let the frame weights be the k-th image to be segmented. The time decay factor, Let x be the index of the image to be segmented, and y be the first coordinate index of the pixel. For the k-th pixel in the image to be segmented The probability of the future.

7. A dynamic image sequence stitching optimization system, characterized in that, include: The acquisition module is used to acquire dynamic image sequences, perform particle filtering segmentation on the dynamic image sequences, and obtain multiple target segmentation masks; The confidence fusion module is used to perform confidence fusion on each of the target segmentation masks based on the dynamic image sequence to obtain the corresponding fused segmentation mask. The gradient domain fusion module is used to perform gradient domain fusion on the dynamic image sequence according to the fusion segmentation mask to obtain the corresponding stitched image.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the dynamic image sequence stitching optimization method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the dynamic image sequence stitching optimization method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the dynamic image sequence stitching optimization method as described in any one of claims 1-6.

Images