A three-dimensional fusion method and system of multi-path video decoding combined with real scene reconstruction

Abstract

The application discloses a three-dimensional fusion method of multi-path video decoding combined with real scene reconstruction, and relates to the technical field of three-dimensional modeling. The method comprises the following steps: acquiring a plurality of video streams to perform motion area identification processing to obtain an initial motion area distribution map; determining initial positioning information of an object of interest based on the distribution map; performing position change correlation comparison according to the initial positioning information to obtain motion trajectory distribution characteristics; and correcting a video frame based on the trajectory distribution characteristics and outputting a three-dimensional reconstruction result. Through the closed-loop processing of motion area identification, initial positioning, trajectory analysis and video frame correction, the three-dimensional reconstruction precision and efficiency can be synergistically optimized.

Description

Technical Field

[0001] This disclosure generally relates to the field of 3D modeling technology, and in particular to a 3D fusion method and system that combines multi-channel video decoding with real-scene reconstruction. Background Technology

[0002] In fields such as intelligent monitoring, security patrol, and autonomous driving, the fusion technology of real-time processing of multi-channel video and real-scene 3D reconstruction is becoming increasingly critical. This fusion can provide a more comprehensive spatial understanding and dynamic perception capability for complex scenarios, directly affecting the system's accurate response to the environment and decision-making efficiency.

[0003] Currently, when the same object moves rapidly in multiple video streams, its position and boundaries shift in each frame, so the accuracy of the fusion technology cannot be guaranteed during implementation. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in related technologies, it is desirable to provide a three-dimensional fusion method and system that combines multi-channel video decoding with real-scene reconstruction to improve fusion accuracy.

[0005] In a first aspect, a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction is provided. The method includes: acquiring multiple video streams, performing motion region identification processing on each video frame in each video stream, and obtaining an initial motion region distribution map corresponding to each video frame.

[0006] The initial location information of the object of interest in each video frame is determined based on the initial motion region distribution map.

[0007] Based on the initial positioning information of the object of interest in each video frame, the position changes of the object of interest in each video frame are correlated and compared to obtain the motion trajectory distribution characteristics of the object of interest.

[0008] Based on the motion trajectory distribution characteristics of the object of interest, each video frame is corrected, the target video stream is obtained based on the corrected video frames, and the three-dimensional reconstruction result is output based on the target video stream.

[0009] This application provides a 3D fusion method combining multi-channel video decoding and real-scene reconstruction. This method achieves accurate identification and segmentation of motion regions in each video frame by performing motion region identification processing on each video frame in each video stream, providing basic data support for subsequent processing. Then, based on the initial motion region distribution map, the initial positioning information of the object of interest in each video frame is determined, achieving precise position locking of the moving object and avoiding computational redundancy caused by global search. Next, by obtaining the motion trajectory distribution characteristics of the object of interest, the system can model and predict the motion law of the object, enabling it to have temporal continuity analysis capabilities. Furthermore, based on the motion trajectory distribution characteristics of the object of interest, each video frame is corrected, and the target video stream is obtained from the corrected video frames. The 3D reconstruction result is then output based on the target video stream. This technical feature enables dynamic allocation of computing resources and local scene updates, ensuring the real-time performance and accuracy of the 3D reconstruction results. This scheme significantly reduces background noise interference by generating an initial motion region distribution map, providing high-quality input for subsequent processing. Simultaneously, by determining initial positioning information and combining it with cross-frame correlation analysis, stable tracking of moving objects is achieved. Furthermore, by extracting motion trajectory distribution features, a continuous description of object motion is constructed, providing quantifiable control signals for resource scheduling. Finally, video frame correction improves the processing accuracy of key regions. This scheme addresses the problem of 3D model updates caused by dynamic changes in the position of moving objects during the fusion of multi-channel video decoding and real-scene reconstruction. By forming a closed loop between motion perception, positioning, trajectory analysis, and resource scheduling, it achieves synergistic optimization of 3D reconstruction accuracy and efficiency, providing reliable technical support for real-time 3D reconstruction in complex scenes.

[0010] Secondly, a three-dimensional fusion system combining multi-channel video decoding and real-scene reconstruction is provided. The system includes: a recognition processing module, used to acquire multiple video streams, perform motion region recognition processing on each video frame in each video stream, and obtain an initial motion region distribution map corresponding to each video frame.

[0011] The determination module is used to determine the initial location information of the object of interest in each video frame based on the initial motion region distribution map.

[0012] The comparison module is used to perform a correlation comparison of the position changes of the objects of interest in each video frame based on the initial positioning information of the objects of interest in each video frame, and obtain the motion trajectory distribution characteristics of the objects of interest.

[0013] The correction output module is used to correct each video frame based on the motion trajectory distribution characteristics of the object of interest, obtain the target video stream based on the corrected video frames, and output the three-dimensional reconstruction result based on the target video stream.

[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart illustrating the steps of a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, as provided in an embodiment of this application.

[0017] Figure 2 This is a flowchart illustrating a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, provided in an embodiment of this application.

[0018] Figure 3 This is a flowchart illustrating a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, provided in an embodiment of this application.

[0019] Figure 4 This is a flowchart illustrating a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, provided in an embodiment of this application.

[0020] Figure 5 This is a flowchart illustrating a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, provided in an embodiment of this application.

[0021] Figure 6 This is a schematic diagram of the structure of a three-dimensional fusion system that combines multi-channel video decoding with real-scene reconstruction, as provided in an embodiment of this application. Detailed Implementation

[0022] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0024] The following is combined with Figure 1 For an explanation of the 3D fusion method combining multi-channel video decoding and real-scene reconstruction provided in this application, please refer to [link / reference]. Figure 1 , Figure 1The flowchart of a three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction provided for an exemplary embodiment of this application includes the following steps: Step S20, acquiring multiple video streams, performing motion region recognition processing on each video frame in each video stream, and obtaining an initial motion region distribution map corresponding to each video frame.

[0025] Among them, multiple video streams can be synchronously acquired video data from different physical perspectives or logical channels, such as four H.264 encoded video streams output by cameras deployed in four directions at an intersection; motion region recognition processing is used to coarsely locate the region to which an object suspected of being in motion belongs in each video frame of each video stream; the initial motion region distribution map can be a binary or grayscale heatmap, and the pixel values ​​on the map can represent the probability, confidence level, etc. of the corresponding spatial location belonging to the motion region.

[0026] Optionally, this application may perform motion region identification processing on each video frame by any of the following methods: First method: perform optical flow amplitude aggregation analysis based on the motion vector field of each video stream, perform sliding window statistics on the motion vector magnitude at the same spatial location in three consecutive frames, and if the mean exceeds a preset threshold and the standard deviation is lower than the neighborhood fluctuation benchmark, then mark the region as a motion region.

[0027] The second approach is to integrate the disparity consistency constraints between multiple video streams. In a binocular or multi-view configuration, motion compensation is performed on the failed matching areas for re-evaluation. Areas with disparity jumps and consistent motion vector directions are enhanced and labeled as motion regions.

[0028] Step S30: Determine the initial location information of the object of interest in each video frame based on the initial motion region distribution map.

[0029] The object of interest (OI) can be a person, animal, or object in motion within any video frame. Each video frame can contain one OI, multiple OIs, or none at all. The initial location information of the OI can be its two-dimensional spatial coordinates within the video frame, such as the x and y coordinates of its center point, its width and height dimensions, and the corresponding confidence level. This initial location information can be obtained by performing adaptive thresholding and contour fitting on an initial motion region distribution map. This initial location information provides geometric anchors for subsequent cross-frame association, and its accuracy directly affects the stability of trajectory modeling.

[0030] In one alternative implementation, determining the initial positioning information based on the initial motion region distribution map can be as follows: perform global threshold segmentation on the initial motion region distribution map, extract connected regions, fit the minimum bounding rectangle for each connected region, select the rectangle with the largest area and an aspect ratio between 0.3 and 3.0 as candidate bounding boxes, sort them according to their average response intensity in the initial motion region distribution map, and select the top K (K≥1) as the initial positioning results.

[0031] Step S40: Based on the initial positioning information of the object of interest in each video frame, the position changes of the object of interest in each video frame are correlated and compared to obtain the motion trajectory distribution characteristics of the object of interest.

[0032] Among them, the correlation comparison of position changes can refer to establishing the correspondence between the same object of interest in different frames in the time dimension. Its input can be the set of initial positioning information of the object of interest in each video frame of the above output, and the output can be a target ID sequence with temporal continuity. The motion trajectory distribution feature is not a single numerical value or vector, but a structured description of the motion law of the object of interest, including at least trajectory segment length distribution, position offset variance, velocity change trend, turning frequency, spatial coverage density, etc. This feature can be used to support the generation of subsequent correction strategies.

[0033] Optionally, this application may construct an 8-dimensional state vector by extracting the geometric center and aspect ratio of the bounding box from the initial positioning information of each frame, using a linear Kalman filter for state prediction and observation update, and using Mahalanobis distance as the matching criterion, with those whose distance is less than the 95th quantile threshold of the χ² distribution considered as valid associations.

[0034] Step S50: Correct each video frame based on the motion trajectory distribution characteristics of the object of interest, obtain the target video stream based on the corrected video frames, and output the three-dimensional reconstruction result based on the target video stream.

[0035] Modifying video frames can be done without altering the original pixel content. This can be achieved through methods such as resampling, region masking, resolution grading, or reconfiguration of encoding parameters to adjust aspects such as the position of the object of interest, the distance between the object of interest and other objects, and their order. By modifying each video frame, the data received for subsequent 3D reconstruction becomes more suitable for the spatial evolution characteristics of the moving objects.

[0036] The target video stream can be a video data stream optimized for 3D reconstruction after the above corrections. Its frame structure still conforms to the general video coding format, but key areas retain higher spatiotemporal fidelity. The 3D reconstruction result is a point cloud, mesh model, or voxel representation generated based on the target video stream. Its spatial accuracy and update frequency are directly affected by the aforementioned correction effects.

[0037] In one alternative implementation, this application may correct the video frame by analyzing the velocity variance and turning frequency in the motion trajectory distribution characteristics of the object of interest, marking the high variance + high frequency region as a strong dynamic region, enabling a higher QP value (reducing compression distortion) and increasing the motion estimation search range for the local block of the video frame where the strong dynamic region is located, and inserting super-resolution interpolation after decoding.

[0038] Alternatively, video frames can be corrected by inputting motion trajectory distribution features into a lightweight decision network and outputting a region correction strategy label for each video frame. The label can include three categories: "enhanced tracking", "maintain current", and "download buffer". Then, based on the label, the Kalman filter parameters are reconfigured, the default decoding process is maintained, or the inter-frame skipping mechanism is activated.

[0039] This application acquires multiple video streams and performs motion region recognition processing to generate an initial motion region distribution map, providing low-overhead, high-response motion priors for subsequent processing. Based on this, the initial motion region distribution map is used to perform initial localization of objects of interest, avoiding redundant computation caused by global dense detection. Then, based on the initial localization information, correlation comparison of position changes is carried out to form motion trajectory distribution features. Finally, using the motion trajectory distribution features as driving signals, differential correction is performed on the original video frames to generate a target video stream adapted to the needs of 3D reconstruction. Thus, without decoupling the decoding and reconstruction processes, on-demand resource scheduling and closed-loop optimization of reconstruction quality driven by moving objects are achieved, effectively alleviating the contradiction between accuracy and latency under the condition of limited computing power of edge devices.

[0040] In an optional embodiment, such as Figure 2 As shown, Figure 2 This application provides an optional method embodiment for obtaining the initial motion region distribution map corresponding to each video frame. The method embodiment includes the following steps: Step S201, the static background region and motion region in each video frame are decomposed and separated according to the acquisition time sequence to obtain the initial region division result of each video frame.

[0041] The acquisition sequence can be determined based on the time markers in each video frame. Generally, a video stream is a data stream arranged according to the acquisition sequence. A moving region refers to an image region in the current frame that is identified as having pixel-level motion characteristics, while a static background region refers to other regions in the video frame that do not possess motion region characteristics. Deconstructing and separating the static background region and moving region in each video frame involves separating the foreground and background, thereby highlighting the moving region and facilitating subsequent correction processing of the video frame. This application can use a pre-built image analysis module to perform deconstruction processing on each video frame, separating the static background region and moving region in the video frame to obtain the initial region division results for each video frame.

[0042] For example, the OpenCV library is used to synchronously decode the input multi-channel video streams into RGB video frames, and each video frame is resized to a uniform resolution such as 1280x720 to reduce computational load and unify the processing scale. Then, a dynamic background model is constructed for each video frame using the Gaussian Mixture Model (GMM) background subtraction algorithm. The learning rate is set to 0.01 and the number of Gaussian components is 5 to perform pixel-level foreground extraction. The threshold is set to 25 to filter noise, thereby initially identifying potential motion regions in each video frame and avoiding interference from changes in lighting.

[0043] Step S202: Based on the initial region division results of each video frame, pixel continuity detection is performed on the motion regions in each video frame, and the motion regions in the initial region division results are prioritized according to the detection results to obtain a region priority list for the motion regions in each video frame.

[0044] Among them, pixel continuity detection refers to evaluating whether each connected component marked as a motion region in the initial region division result has the integrity and consistency of pixel-level motion trajectory in the acquisition time sequence dimension, rather than just the existence of a single frame; region priority ranking can be a process of assigning a quantitative score to each motion region and arranging them in descending or ascending order. This score can be an observable indicator such as fusion area, motion intensity, confidence, etc., used to characterize the resource allocation weight of the motion region in subsequent tracking and reconstruction tasks.

[0045] Optionally, this application may perform a priority sorting operation on motion regions by the following steps: if the detection result indicates that the motion region is occluded, then the occluded regions in the motion region are marked to obtain the identification information of each occluded region.

[0046] The target information of each motion region is determined based on the identification information of each of the occluded regions. The target information is used to characterize the integrity and continuity of the motion region.

[0047] Based on the target information of each motion region, a priority is assigned to each motion region, and the regions are sorted according to their priorities to obtain the region priority list.

[0048] For example, this application can calculate the inter-frame optical flow field for moving regions in each video frame using the Farneback algorithm, with a density set to 0.5. This is combined with an optical flow direction consistency threshold of 0.8 and motion entropy calculation. If the cosine of the angle between the optical flow vector within the moving region and its neighborhood is less than 0.6, it is marked as an occlusion region. Simultaneously, a pseudo-depth estimation module is used to predict the relative depth difference through a monocular depth network such as MiDaS. If the relative depth difference is greater than a preset threshold (e.g., 1.5 meters), dynamic occlusion is confirmed. This process can compensate for temporary loss by predicting the trajectory using Kalman filtering, improving detection robustness. Next, the detected moving regions are prioritized. For example, the area, motion velocity (mean optical flow amplitude), and confidence level (foreground probability greater than 0.7) of each moving region are extracted. A weighted scoring formula is used to calculate the priority: Priority = 0.4 * Normalized Area + 0.3 * Velocity + 0.3 * Confidence. The calculation results are then sorted in descending order to prioritize large and fast-moving regions, avoid interference from small noises, and optimize subsequent tracking resource allocation.

[0049] Step S203: Determine the target motion region with a priority greater than the priority threshold from the region priority list, perform detail enhancement processing on the target motion region to obtain region feature data, and generate the initial motion region distribution map corresponding to each video frame based on the region feature data.

[0050] The priority threshold is a configurable scalar parameter used to control the number of motion regions participating in subsequent processing. Its value ranges from [0.3, 0.8], and can be dynamically adjusted based on factors such as system load and scene complexity. The target motion region refers to the set of high-priority motion regions retained after threshold filtering. Its number can be less than or equal to the initial total number of motion regions. Detail enhancement processing is an image processing operation that improves the spatial representation robustness of a region while maintaining its topological structure. It does not introduce new semantics but only enhances the geometric integrity and boundary sharpness of existing motion regions. Examples include morphological dilation of the target motion region and mapping the target motion region to overlapping areas in the corresponding viewpoints of multiple video streams. Region feature data can be a combination of a detail-enhanced binary mask image and its spatial statistical features. The initial motion region distribution map can be a final output heatmap visualization that can be used for cross-frame association and localization. Its pixel values ​​can reflect the probability density of a corresponding location belonging to a high-priority motion region.

[0051] For example, this application can generate a preliminary motion region distribution map based on the sorting results, then expand the kernel size of high-priority regions to 5x5 pixels, merge overlapping regions, and draw a binary mask image, where white pixels represent motion distribution. This is then fused with multiple views where the IoU is greater than 0.5 to form a heatmap showing regions with density peaks greater than 10, serving as the initial motion region distribution map for each video frame. This initial motion region distribution map can be directly used for subsequent target tracking or anomaly alerts. The entire process forms a closed-loop logic chain: preliminary decomposition provides clean input, occlusion detection ensures region accuracy, priority sorting optimizes efficiency, and finally, the initial motion region distribution map enables visualization analysis and business applications such as traffic statistics.

[0052] This application establishes a background evolution model resistant to illumination interference by collecting time-driven static / motion region decomposition and separation data; by using pixel continuity detection and region priority ranking, motion recognition is upgraded from "whether it exists" to "whether it is reliable and important"; after threshold filtering and detail enhancement, an initial motion region distribution map with both spatial accuracy and semantic confidence is output, solving the technical problems of insufficient robustness, weak interpretability, and difficulty in driving subsequent localization in the motion region recognition step of this application in complex real-world scenarios, enabling the entire 3D fusion process to accurately anchor dynamic targets from the decoding source.

[0053] In an optional embodiment, such as Figure 3 As shown, Figure 3 This application provides an optional method embodiment for determining the initial location information of objects of interest in each video frame based on an initial motion region distribution map. The method embodiment includes the following steps: Step S301, using cross-frame association analysis technology to lock the bounding boxes of motion regions in each video frame based on the initial motion region distribution map.

[0054] Cross-frame association analysis technology refers to the technical means of establishing a continuous mapping relationship of objects of interest in a continuous video frame sequence based on the spatial location, appearance features, and motion trends of the motion region. The initial motion region distribution map serves as the input constraint for cross-frame association, used to limit the range of candidate regions and eliminate static background interference. The boundaries of the motion regions are obtained by filtering from the region priority list. Boundary box locking of the motion region can refer to defining the identified motion region by using a two-dimensional rectangular bounding box, which can be based on the center coordinates and width and height dimensions as basic parameters.

[0055] Optionally, this application may lock the bounding box of the moving region by the following steps: binarizing the moving region in each video frame to obtain the motion pixel distribution of each moving region.

[0056] The initial region of interest is formed by marking connected components based on the motion pixel distribution of each motion region.

[0057] Cross-frame correlation analysis is used to lock the bounding box of the initial region of interest.

[0058] For example, in a 1080x1920 resolution video frame, the motion pixel distribution of each motion region in each video frame can be obtained by binarizing the motion regions (e.g., setting a threshold of 0.5). Then, connected components are marked according to the motion pixel distribution of each motion region to form an initial region of interest, with its area threshold set to 500 pixels to filter noise, thereby obtaining coarse bounding boxes for multiple motion regions, such as center coordinates (x=450.2, y=320.7) and width and height (w=120.5, h=180.3). Finally, cross-frame correlation analysis technology is used to lock the bounding boxes of the initial regions of interest.

[0059] Step S302: Obtain the position data and boundary data of the object of interest in each video frame through trajectory smoothing processing technology, and obtain the initial positioning information of the object of interest in each video frame.

[0060] Among them, trajectory smoothing processing technology refers to the post-processing technology of performing temporal modeling and noise suppression on the original bounding box sequence of cross-frame associated output to improve the continuity and stability of position and boundary parameters; position data refers to the center point coordinates of the object of interest in the image coordinate system; boundary data refers to the width and height of the bounding box corresponding to the object; the initial positioning information is a target space description with cross-frame consistency composed of position data and boundary data, which serves as the direct input for motion trajectory analysis.

[0061] For example, this application may employ the DeepSORT algorithm framework, initializing the Kalman filter state vector as 8-dimensional (u,v,a,h,vu,vu,va,vh), assuming a uniform motion model, and setting the process noise covariance Q as diag(0.01,0.01,0.1,0.1,0.01,0.01,0.01,0.01). For adjacent video frames, the IoU matrix between the predicted bounding box and the current detection box is calculated, for example, with an IoU threshold of 0.6 as the initial screening. The cost matrix is ​​constructed by combining Mahalanobis distance (threshold χ² distribution 95% confidence interval 9.4877) and cosine appearance feature distance (extracting ReID embedding vector, threshold 0.2). The optimal matching is solved using the Hungarian algorithm to achieve cross-frame association and lock accurate bounding boxes. The analysis process shows that in occluded scenes, Mahalanobis distance preferentially suppresses associations with high uncertainty, reducing the ID switching rate by 15.3%. Finally, trajectory smoothing is used to obtain the position and boundary data of the object of interest in each frame. Kalman filters are used to update the fused observations. For example, the measurement noise R is set to diag(10,10,5,5). The prediction state is corrected by weighting the gain K. The standard deviation of the smoothed trajectory noise is reduced from 12.4 pixels to 3.8 pixels. At the same time, the virtual position of the lost trajectory is retained for 30 frames of predicted value. Gaussian smoothing interpolation is used to recover the interrupted trajectory to ensure continuity. This determines the initial positioning information of the object in the picture, such as the center of frame 1 (450.2,320.7) and the smoothed position of frame 2 (462.8,335.1). The whole process forms a closed-loop logic: the initial distribution provides candidates, cross-frame locking is used for precise positioning, and trajectory smoothing optimizes stability, improving the overall tracking MOTA index to 78.5%.

[0062] In an optional embodiment, such as Figure 4 As shown, Figure 4 An optional method embodiment for obtaining the motion trajectory distribution characteristics of an object of interest is provided in this application. The method embodiment includes the following steps: Step S401, obtaining the position information of the object of interest in temporally consecutive video frames based on the initial positioning information of the object of interest in each video frame, and obtaining a position information sequence.

[0063] The location information of the object of interest can refer to the core parameters extracted from the initial positioning information to characterize the spatial location of the object of interest, such as the coordinates of the center point and the optional normalized velocity vector.

[0064] In terms of temporal continuity, video frames refer to a subset of adjacent video frames arranged in the order of acquisition and whose frame number difference does not exceed a preset maximum interval threshold (e.g., 5 frames) to ensure that the temporal continuity constraint is met.

[0065] For example, this application uses the YOLOv8 object detection algorithm to detect objects of interest in video frames, obtains the bounding box coordinates (x, y, w, h) of the objects of interest in each video frame, sets the confidence threshold to 0.6, and obtains the location information of the objects of interest to filter out low-confidence detection results.

[0066] Step S402: The location information sequence is verified through a spatiotemporal consistency verification mechanism and a Kalman filter.

[0067] The spatiotemporal consistency verification mechanism refers to a dual-criteria system that integrates the rationality of prediction in the time dimension and the matching of observation in the spatial dimension. Its technical focus is: at the current frame t, predict the position of the current frame based on the state of the previous frame t-1, and determine whether the deviation between the prediction result and the actual observed position meets the statistical significance requirement. This mechanism does not rely on a specific algorithm, but must simultaneously meet two basic conditions: temporal continuity (e.g., limited rate of position change) and spatial consistency (e.g., IoU or Mahalanobis distance between adjacent frames is within a threshold range). The Kalman filter refers to a standard linear / extended Kalman filter structure used to model the motion state of a target and perform recursive estimation. Its state vector includes at least position and velocity components, and the process noise covariance and measurement noise covariance can be dynamically adjusted according to the scene. The Kalman filter works in conjunction with the spatiotemporal consistency verification mechanism. The former provides the optimal estimate in a mathematical sense, while the latter provides the validity verification at the logical level. Together, they constitute a closed-loop verification unit.

[0068] Optionally, the present application may perform verification through the following steps: determining the first position information of the current video frame from the position information sequence, and predicting the second position information of the next video frame based on the first position information using a Kalman filter.

[0069] The third position information of the next video frame is obtained from the position information sequence, and the second position information is compared with the third position information.

[0070] If the comparison result shows that the deviation between the second and third position information is less than the deviation threshold, then the verification result is determined to be that the current frame and the next video frame satisfy the spatiotemporal consistency.

[0071] For example, this application may determine the bounding box center point (x_t, y_t) of the current video frame (e.g., frame t) from the position information sequence; then, using a Kalman filter with a constant velocity motion model assuming the state vector is [x, y, v_x, v_y], the process noise covariance Q is set to diag(0.01, 0.01, 0.1, 0.1), and the measurement noise covariance R is set to diag(25, 25), predicting the second position information of the next video frame of the current video frame; next, obtaining the measured third position information of the next video frame from the position information sequence; then, calculating the Mahalanobis distance between the second and third position information, using this Mahalanobis distance as the deviation between the two; finally, comparing this Mahalanobis distance with a deviation threshold (e.g., 3.0 times the standard deviation); if the Mahalanobis distance is less than 3.0 times the standard deviation, the verification result can be considered that the current frame and the next video frame satisfy spatiotemporal consistency; furthermore, this result can trigger subsequent update operations on the unknown information sequence. Conversely, if the Mahalanobis distance is greater than or equal to 3.0 times the standard deviation, it can trigger an operation that treats the object of interest as a new object of interest or as a lost object.

[0072] Step S403: Update the location information sequence according to the verification result to obtain the updated location information sequence.

[0073] The verification result refers to the judgment conclusion jointly output by the spatiotemporal consistency verification mechanism and the Kalman filter, which can include "match successful" or "match failed" results. The update refers to the correction, replacement, or supplementation of data at the corresponding position in the original position information sequence when a match is successful. For example, for a marked position information sequence to be corrected, auxiliary data during viewpoint switching is obtained, and abnormal position information is smoothed using a compensation method to obtain the updated position information sequence. Exemplarily, this application can perform a state update when the Mahalanobis distance is determined to be less than 3.0 times the standard deviation.

[0074] Step S404: Perform correlation and comparison on the updated location information sequence to obtain the motion trajectory distribution of the object of interest.

[0075] This application involves extracting the motion trajectory information of the object of interest (OPI) across consecutive video frames from an updated location information sequence. Through frame-by-frame correlation analysis, the motion trajectory distribution of OPI is determined. Based on this distribution, a Kalman filter algorithm is used to optimize the motion trajectory distribution, obtaining smooth trajectory data that conforms to actual motion patterns. Using the optimized trajectory data, segmented analysis is performed on the motion trajectories at different time periods to determine whether each segment meets spatiotemporal consistency. If deviations exist, local adjustments are made based on historical data to obtain the final motion trajectory distribution of OPI.

[0076] For example, this application may employ the Hungarian algorithm to solve the multi-object matching problem. The cost matrix is ​​defined as the weighted sum of the IoU distance and Mahalanobis distance between matching pairs of adjacent video frames, with weights of 0.7 and 0.3, respectively. When the cost is less than 0.5, an association is established, thereby forming a preliminary trajectory segment. To cope with the abrupt changes caused by viewpoint switching, a viewpoint switching compensation technique is introduced. First, the homography matrix H between adjacent video frames is calculated through ORB feature point matching. If there are more than 50 pairs of feature matching points and the reprojection error is less than 2.0 pixels, a viewpoint switching is considered to have occurred. The bounding box coordinates of the object of interest are then compensated by affine transformation using H, normalizing the coordinates to a unified viewpoint. Finally, by integrating the above processing, the motion trajectory distribution of the object of interest is obtained. Specifically, the normalized distribution histogram of the number of consecutive associated frames in each segment of the trajectory is calculated, and statistical features such as average continuous length of 25 frames, longest continuous length of 78 frames, and trajectory fragment ratio of 15% are extracted. Combined with the velocity vector smoothness (variance less than 10.5 pixels² / frame²), it is used as the final continuity distribution feature for subsequent anomaly detection or behavior analysis. The whole process forms a closed-loop feedback to ensure the robustness and continuity of the trajectory in the spatiotemporal domain.

[0077] Step S405: Extract key characteristic data from the motion trajectory distribution, generate a continuous description of the motion of the object of interest, and determine the continuous description as the motion trajectory distribution feature of the object of interest.

[0078] Key characteristic data refers to a set of structured numerical indicators that can quantitatively characterize the intrinsic motion attributes of a trajectory, including but not limited to average continuous length, longest continuous length, trajectory fragmentation rate, velocity vector smoothness variance, and acceleration abrupt change frequency. Continuity description refers to a textual or structured expression composed of the above key characteristic data, possessing clear physical meaning and engineering interpretability, such as "average continuous length 25 frames, longest continuous length 78 frames, fragmentation rate 15%, velocity smoothness variance <10.5 pixels² / frame²".

[0079] This application transforms initial positioning information into a sequence of location information. It then employs a spatiotemporal consistency verification mechanism and a Kalman filter to jointly perform a prediction-observation-correction closed-loop verification. Based on the verification results, the sequence content is dynamically updated. Furthermore, a complete motion trajectory distribution is constructed through correlation comparison of the updated sequence. Key characteristic data with engineering interpretability is extracted from this distribution to generate a continuity description, ultimately forming a clearly structured, quantifiable, and programmable motion trajectory distribution feature. This feature not only accurately characterizes the true motion patterns of the object of interest across multiple video streams but also provides clear control criteria and triggering conditions for "correcting each video frame based on the motion trajectory distribution feature of the object of interest" in this application. For example, when the continuity description shows "fragmentation rate > 20%" or "average continuity length < 15 frames," the system can automatically increase the processing priority of the corresponding area, thereby achieving a technical closed loop from the perception layer to the decision layer.

[0080] In an optional embodiment, such as Figure 5 As shown, Figure 5 This application provides an optional method embodiment for correcting each video frame based on the motion trajectory distribution characteristics of the object of interest. The method embodiment includes the following steps: Step S501, based on the motion trajectory distribution characteristics of the object of interest and the scene depth estimation of the video frame, the object of interest from different perspectives in each video stream is corrected and integrated using a multi-source data fusion method to determine the relative position data of the object of interest in three-dimensional space.

[0081] Among them, the motion trajectory distribution characteristics of the object of interest refer to the continuous description composed of key characteristic data extracted from the motion trajectory distribution as defined in this application, which characterizes the positional continuity, velocity change trend, and trajectory fragmentation degree of the object of interest in the temporal dimension; the scene depth estimation of the video frame refers to the estimation result of the scene depth value corresponding to each pixel or region in the current video frame, which can be the relative depth map output by the monocular depth model, the absolute depth value obtained by triangulation of the binocular disparity map, or the depth mapping generated by the voxel projection of the LiDAR point cloud; the multi-source data fusion method refers to the fusion of data from different video stream perspectives. The process of geometrically aligning and weighting the motion trajectory features and corresponding depth estimates does not introduce new data modalities, but only relies on the video stream input and trajectory feature output defined in the above embodiments. The correction and integration includes, for example, mapping the two-dimensional positioning points determined based on trajectory features and the corresponding depth values ​​from each viewpoint to preliminary three-dimensional coordinates, unifying the coordinate system according to the spatial attitude parameters of each viewpoint, and then weighting the three-dimensional coordinates of the multiple viewpoints according to preset weights to obtain the relative position data of the object of interest in a unified three-dimensional coordinate system. This relative position data is used for subsequent accuracy judgment and does not directly participate in the three-dimensional reconstruction output.

[0082] For example, in the process of determining the relative three-dimensional position of an object in multiple video streams, the continuous distribution characteristics of the object's motion trajectory are first analyzed. A temporal analysis algorithm is then used to process the video frame sequence. Assuming the time interval between each video frame is 0.033 seconds, a Kalman filter algorithm is used to predict and update the object's position on the two-dimensional plane, obtaining continuous displacement data along the x and y axes. For instance, in the first frame, the object's position is (100, 200) pixels, and in the second frame, it is predicted to be (105, 205) pixels. After error correction, the actual position is (104, 204) pixels. The smoothness of the trajectory is analyzed to reduce noise interference. Next, combined with scene depth estimation, a monocular depth estimation model based on a deep neural network, such as the MiDaS model, is used. The input video frame resolution is 1920x1080, and the output is the depth value of each pixel. Assuming the depth of the object's center point is 5.2 meters, the object's coordinate data in three-dimensional space is initially constructed by combining the depth map with the two-dimensional trajectory, such as (104, 204, 5.2). Subsequently, for multi-source data fusion, a weighted average algorithm was used to integrate data from different perspectives of multiple video streams. Assuming three video perspectives, the object coordinates were (104, 204, 5.2), (106, 203, 5.3), and (105, 205, 5.1), respectively. By assigning confidence weights of 0.4, 0.3, and 0.3 to each perspective, the fused coordinates were calculated to be (104.7, 204.0, 5.2). Finally, object information was corrected and integrated. A geometric transformation matrix was used to spatially align the multi-view data. Assuming a 5-degree rotational deviation between perspective 1 and perspective 2, after correction using the rotation matrix, all data were unified to the same coordinate system. Combined with depth error analysis, the final three-dimensional relative position of the object was determined to be (104.5, 204.2, 5.2). The reasonableness of this was verified by the relative distance to a reference object in the scene (e.g., ground height of 0 meters), ensuring the error was within 0.1 meters. The above process is automatically processed by algorithms, forming a complete logical chain from trajectory analysis to 3D positioning. The combination of depth estimation and multi-source fusion effectively improves positioning accuracy.

[0083] In step S502, if the relative position data of the object of interest in three-dimensional space does not match the preset accuracy threshold, the region in the video frame that includes the object of interest is marked as a high-frequency region, and the other regions in the video frame are marked as low-frequency regions.

[0084] Among them, the preset accuracy threshold refers to the numerical limit used to determine whether the three-dimensional relative position data meets the reconstruction accuracy requirements. It can be the upper limit of point cloud registration error, ICP residual threshold, or absolute value of coordinate deviation, such as 0.05 meters. The region in the video frame that includes the object of interest refers to the image region covered by expanding a preset pixel range (such as a 5×5 dilation kernel) outward based on the initial positioning information (including bounding box coordinates) obtained in this application. The "high frequency region" and "low frequency region" are logical division labels that do not change the original pixel values ​​and are only used to indicate the scheduling basis of subsequent processing strategies.

[0085] For example, in a three-dimensional spatial positioning system, this application may first collect relative position data through multi-sensor fusion, such as using an RGB-D camera to acquire point clouds and calculate the Euclidean distance error. If the average positioning error exceeds a preset accuracy threshold of 0.05 meters, an adjustment mechanism is triggered. Specifically, the point cloud registration error between the current frame and the reference frame is calculated, and the rotation matrix R and translation vector t are iteratively solved using the Iterative Closest Point (ICP) algorithm to minimize the error function ∑||Rp_i + t - q_i||². When the error is greater than the threshold, it is confirmed that the relative position data of the object of interest in three-dimensional space does not match the preset accuracy threshold.

[0086] Step S503: Focusing is performed on the high-frequency region and compression is performed on the low-frequency region through a dynamic load balancing mechanism to obtain the target spatial localization result of the object of interest in the video frame.

[0087] Among these measures, focusing on high-frequency regions involves, for example, using an octree to subdivide the space of detected high-frequency regions (such as regions with speeds exceeding 0.5 m / s), increasing the voxel resolution of the high-frequency region to 0.01 m, increasing the sampling point density by 30%, and applying Kalman filtering to predict the trajectory, thereby improving the tracking accuracy to 0.02 m.

[0088] Low-frequency regions are compressed, for example, by using sparse convolution to reduce the number of voxels by 50%, thus reducing computational overhead by 20%. Finally, resources are tilted towards high-frequency regions, allocating an additional 30% of computational resources. Error reduction is monitored in real time through a feedback loop. If the accuracy improves to within a threshold, more accurate spatial positioning results are output, such as centimeter-level 3D coordinates. The entire process forms a closed-loop thinking chain, ensuring dynamic adaptation to complex scenarios and improving the overall system efficiency and accuracy.

[0089] This application employs high-frequency region focusing processing to densely sample and finely calculate the position data of the object of interest, obtaining preliminary refined position data. Low-frequency region compression is then used to sparsely sample and simplify the position data of non-moving regions, yielding compressed position data. Based on the preliminary refined and compressed position data, the overall relative position data in three-dimensional space is re-integrated to determine the target spatial localization result of the object of interest within the video frame.

[0090] Step S504: Based on the target spatial positioning results and combined with real-time feedback adjustment and resource occupancy monitoring technology, dynamically correct the object of interest and its surrounding environment in the video frame to obtain the corrected video frame.

[0091] Among them, real-time feedback adjustment refers to the operation of closed-loop correction of the current processing parameters based on the deviation between the output result of the previous processing cycle and the current input, such as adjusting the process noise covariance of the Kalman filter based on the feedback of the 3D positioning error; resource usage monitoring technology refers to the technical means of periodically sampling system-level indicators such as CPU / GPU utilization, memory bandwidth, video memory usage, and task latency; dynamic correction refers to locally updating and reweighting the texture, depth, motion vector, and other attributes of the object of interest and its neighboring regions in the video frame without interrupting the video stream decoding and reconstruction process, rather than reprocessing the entire frame; the corrected video frame refers to the final video frame data used as input to the 3D reconstruction module in this application after the above operations, which retains higher spatiotemporal fidelity in the region of interest, while maintaining semantic integrity and structural coherence in the other regions.

[0092] To achieve more accurate spatial positioning and real-time feedback adjustment, this application first collects data using a high-precision lidar and an inertial measurement unit (IMU). The lidar scans 1 million point cloud data points per second with an accuracy of 0.02 meters, while the IMU provides attitude data at a frequency of 200 Hz. A Kalman filter algorithm is used to fuse the two types of data, reducing the positioning error to within 0.05 meters. Analysis shows that the error mainly originates from interference from dynamic objects; therefore, the system prioritizes marking objects with speeds exceeding 0.5 meters per second as dynamic targets. Next, combined with real-time feedback adjustment, the system uses edge computing devices to process scene data every 0.1 seconds. By comparing the differences in point clouds between consecutive frames, the system calculates the displacement vector of dynamic objects. For example, if an object moves 0.3 meters along the X-axis within 0.2 seconds, the system predicts its position in the next frame and generates a correction command. Resource usage monitoring technology detects GPU utilization in real time. If the utilization exceeds 80%, the point cloud density in non-critical areas is automatically reduced from 1000 points per cubic meter to 500 points per cubic meter. Analysis shows that this adjustment can save 30% of computing resources while ensuring that the update frequency of core areas is not less than 30Hz. Subsequently, when driving the local update unit of the 3D scene, the system uses an octree algorithm to divide the scene into blocks, updating only the area with a radius of 2 meters around the location of dynamic objects. For example, when a vehicle moves, only the point cloud data of the surrounding 125 cubic meters is reconstructed, reducing the computational burden of global updates. Analysis shows that local updates can shorten the rendering time from 0.5 seconds to 0.1 seconds. Finally, the dynamically corrected 3D structural data generates a real-time model through a triangular mesh reconstruction algorithm, with the mesh resolution controlled at 0.01 meters. The system updates once per second, ensuring an error of less than 0.03 meters. Analysis shows that this accuracy meets the requirements of virtual reality applications. To form a logical chain, the system is also linked with traffic flow monitoring services, using updated 3D data to predict traffic congestion. If the vehicle density on a certain road segment exceeds 20 vehicles per 100 meters, an early warning signal is triggered to optimize resource allocation. The entire process is seamlessly connected through automated algorithms.

[0093] In an optional embodiment, the three-dimensional fusion method of multi-channel video decoding combined with real-scene reconstruction provided in this application further includes: continuously monitoring the latency accumulation of the three-dimensional reconstruction results; if the latency accumulation exceeds a preset range, resource optimization allocation is triggered to update the three-dimensional reconstruction results.

[0094] This application involves calculating the accumulated processing latency in real time based on an initial 3D structure dataset. By comparing the processing time of each batch of data with timestamps, the current state of latency accumulation is determined. If the current state of latency accumulation exceeds a preset range, an anomaly detection mechanism is triggered to identify the specific data batches exceeding the latency limit and determine the processing steps requiring optimization. For the processing steps requiring optimization, a precision requirement assessment model is used to analyze key and non-key regions in the 3D structure data, resulting in a region priority division. Based on the region priority division, the resource optimization scheme is adjusted to reduce the processing depth of non-key regions, employing a simplified mesh generation technique to determine the optimized data processing strategy. Using the optimized data processing strategy, the 3D structure dataset is processed hierarchically to generate intermediate results for dynamic reconstruction, obtaining an adjusted 3D model framework. For the adjusted 3D model framework, final rendering and integration operations are performed to output the final result of dynamic 3D reconstruction, completing the real-time updated data presentation.

[0095] Specifically, this application can receive point cloud data streams from multi-view cameras or LiDAR in real time, processing 60 frames of 3D structure updates per second. First, it stores the raw frame data within the most recent 200ms in a circular buffer queue and calculates the latency from acquisition to entry into the processing pipeline for each frame. Specifically, it uses a high-precision clock `gettimeofday` to record timestamps t1 and t2, with latency Δt = t2 - t1. If the average latency of 10 consecutive frames exceeds a preset threshold of 80ms, a latency exceedance detection is triggered. Upon detection, it immediately enters the accuracy requirement assessment stage, employing a region importance analysis algorithm based on information entropy. The current point cloud is divided into voxel grids (voxel size 0.01m), and the local curvature entropy H = -Σp_i log p_i is calculated for each voxel, where p_i is the probability distribution of the normal vector within the voxel. If the entropy value is greater than 1.5, it is marked as a critical region (typically accounting for 25%-35% of the total point cloud), and the rest are marked as non-critical regions. Subsequently, resource optimization and allocation are performed. A greedy algorithm is used to prioritize allocating 80% of computing resources to critical regions. Specifically, in the TSDF (Truncated Signed Distance Function) volumetric fusion, critical regions maintain a full depth of 16 layers for integration, while non-critical regions are simplified to 8 layers with a reduced sampling rate of 50%. Simultaneously, the surface point density threshold for non-critical regions is increased from 0.005m to 0.015m to reduce the amount of triangular mesh generation. Through these dynamic adjustments, the overall processing latency can be reduced by 35%-45%. The final fused 3D mesh, after extraction using the Marching Cubes algorithm, is directly output as a dynamic 3D reconstruction result, and latency statistics are fed back to the next monitoring cycle, forming a closed-loop adaptive optimization.

[0096] This application achieves objective quantification of the entire lifecycle latency of the reconstruction link by continuously collecting timestamps of 3D reconstruction results and calculating the average latency of the sliding window. It uses a preset range as a hard criterion to ensure that the intervention mechanism is only activated when actual performance degradation occurs. Furthermore, relying on information entropy-driven key area identification and differentiated resource allocation strategies, it enables limited computing power to be precisely focused on the spatial subdomains that most significantly affect reconstruction quality. Finally, through coordinated adjustment of TSDF integration depth, sampling rate, and surface point density, it reduces overall processing latency by 35%–45% while ensuring centimeter-level accuracy (error <0.03 m) and an update frequency of ≥30 Hz in key areas, forming a closed-loop quality assurance mechanism of "monitoring to promote response, evaluation to ensure accuracy, allocation to improve efficiency, and simplification to stabilize output."

[0097] It should be noted that although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart may be performed in a different order.

[0098] Further reference Figure 6 The diagram illustrates an exemplary structural block diagram of a 3D fusion system 600 that combines multi-channel video decoding with real-scene reconstruction according to this application. The 3D fusion system 600 includes an identification processing module 601, a determination module 602, a comparison acquisition module 603, and a correction output module 604.

[0099] The recognition and processing module 601 is used to acquire multiple video streams, perform motion region recognition processing on each video frame in each video stream, and obtain the initial motion region distribution map corresponding to each video frame.

[0100] The determination module 602 is used to determine the initial location information of the object of interest in each video frame based on the initial motion region distribution map.

[0101] The comparison module 603 is used to compare the position changes of objects of interest in each video frame based on the initial positioning information of the objects of interest in each video frame, and obtain the motion trajectory distribution characteristics of the objects of interest.

[0102] The correction output module 604 is used to correct each video frame based on the motion trajectory distribution characteristics of the object of interest, obtain the target video stream based on the corrected video frames, and output the three-dimensional reconstruction result based on the target video stream.

[0103] It should be understood that the units or modules described in a three-dimensional fusion system 600 for multi-channel video decoding combined with real-scene reconstruction are related to the reference... Figure 1The steps in the described method correspond accordingly. Therefore, the operations and features described above for the method are also applicable to a 3D fusion system 600 for multi-channel video decoding combined with real-scene reconstruction and the units contained therein, and will not be repeated here. A 3D fusion system 600 for multi-channel video decoding combined with real-scene reconstruction can be pre-implemented in the browser or other security applications of an electronic device, or it can be loaded into the browser or other security applications of an electronic device through download or other means. The corresponding units in the 3D fusion system 600 for multi-channel video decoding combined with real-scene reconstruction can cooperate with units in the electronic device to implement the solutions of the embodiments of this application.

[0104] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A three-dimensional fusion method combining multi-channel video decoding and real-scene reconstruction, characterized in that, The method includes: Multiple video streams are acquired, and motion region recognition processing is performed on each video frame in each video stream to obtain an initial motion region distribution map corresponding to each video frame. Based on the initial motion region distribution map, determine the initial location information of the object of interest in each video frame; Based on the initial positioning information of the object of interest in each video frame, the position changes of the object of interest in each video frame are correlated and compared to obtain the motion trajectory distribution characteristics of the object of interest; Based on the motion trajectory distribution characteristics of the object of interest, each video frame is corrected, the target video stream is obtained based on the corrected video frames, and the three-dimensional reconstruction result is output based on the target video stream.

2. The three-dimensional fusion method according to claim 1, characterized in that, The step of acquiring multiple video streams and performing motion region recognition processing on each video frame in each video stream to obtain an initial motion region distribution map corresponding to each video frame includes: According to the acquisition sequence, the static background region and motion region in each video frame are decomposed and separated to obtain the initial region division result of each video frame; Based on the initial region segmentation results of each video frame, pixel continuity detection is performed on the motion regions in each video frame, and the motion regions in the initial region segmentation results are prioritized according to the detection results to obtain a region priority list for the motion regions in each video frame. Target motion regions with priority greater than the priority threshold are determined from the region priority list. Detail enhancement processing is performed on the target motion regions to obtain region feature data. Based on the region feature data, an initial motion region distribution map corresponding to each video frame is generated.

3. The three-dimensional fusion method according to claim 2, characterized in that, The step of prioritizing the motion regions in the initial region segmentation result based on the detection results to obtain a region priority list for the motion regions in each video frame includes: If the detection result indicates that the motion area is occluded, then the occluded areas in the motion area are marked to obtain the identification information of each occluded area; The target information of each motion region is determined based on the identification information of each of the occluded regions, and the target information is used to characterize the integrity and continuity of the motion region; Based on the target information of each motion region, a priority is assigned to each motion region, and the regions are sorted according to their priorities to obtain the region priority list.

4. The three-dimensional fusion method according to claim 1, characterized in that, The determination of the initial location information of the object of interest in each video frame based on the initial motion region distribution map includes: Using cross-frame correlation analysis technology, bounding boxes are locked for motion regions in each video frame based on the initial motion region distribution map; The position and boundary data of the object of interest in each video frame are obtained by using trajectory smoothing processing technology, thereby obtaining the initial positioning information of the object of interest in each video frame.

5. The three-dimensional fusion method according to claim 4, characterized in that, The step of using cross-frame correlation analysis technology to lock the bounding boxes of motion regions in each video frame based on the initial motion region distribution map includes: The motion regions in each of the video frames are binarized to obtain the motion pixel distribution of each motion region; An initial region of interest is formed by marking connected components based on the motion pixel distribution of each motion region; The cross-frame correlation analysis technique is used to lock the bounding box of the initial region of interest.

6. The three-dimensional fusion method according to claim 1, characterized in that, Based on the initial positioning information of the object of interest in each video frame, the position changes of the object of interest in each video frame are correlated and compared to obtain the motion trajectory distribution characteristics of the object of interest, including: Based on the initial positioning information of the object of interest in each video frame, the position information of the object of interest in the temporally consecutive video frames is obtained to obtain a position information sequence; The location information sequence is verified using a spatiotemporal consistency verification mechanism and a Kalman filter. The location information sequence is updated based on the verification result to obtain the updated location information sequence; The updated location information series is correlated and compared to obtain the motion trajectory distribution of the object of interest; Key characteristic data are extracted from the motion trajectory distribution to generate a continuous description of the motion of the object of interest, and the continuous description is determined as the motion trajectory distribution feature of the object of interest.

7. The three-dimensional fusion method according to claim 6, characterized in that, The verification of the location information sequence through a spatiotemporal consistency check mechanism and a Kalman filter includes: The first position information of the current video frame is determined from the position information sequence, and the second position information of the next video frame is predicted based on the first position information using the Kalman filter. Obtain the measured third position information of the next video frame from the position information sequence, and compare the second position information with the third position information; If the comparison result shows that the deviation between the second location information and the third location information is less than the deviation threshold, then the verification result is determined to be that the current frame and the next video frame satisfy the spatiotemporal consistency.

8. The three-dimensional fusion method according to claim 1, characterized in that, The step of correcting each video frame based on the motion trajectory distribution features of the object of interest includes: Based on the motion trajectory distribution characteristics of the object of interest and the scene depth estimation of the video frame, the object of interest from different perspectives in each video stream is corrected and integrated using a multi-source data fusion method to determine the relative position data of the object of interest in three-dimensional space. If the relative position data of the object of interest in three-dimensional space does not match the preset accuracy threshold, then the region in the video frame that includes the object of interest is marked as a high-frequency region, and the other regions in the video frame are marked as low-frequency regions. By using a dynamic load balancing mechanism to focus the high-frequency region and compress the low-frequency region, the target spatial localization result of the object of interest in the video frame is obtained. Based on the target spatial positioning results, combined with real-time feedback adjustment and resource usage monitoring technology, the object of interest and its surrounding environment in the video frame are dynamically corrected to obtain the corrected video frame.

9. The three-dimensional fusion method according to claim 8, characterized in that, The method further includes: The latency of the 3D reconstruction results is continuously monitored. If the accumulated latency exceeds a preset range, resource optimization allocation is triggered to update the 3D reconstruction results.

10. A three-dimensional fusion system combining multi-channel video decoding and real-scene reconstruction, characterized in that, The system includes: The recognition and processing module is used to acquire multiple video streams, perform motion region recognition processing on each video frame in each video stream, and obtain an initial motion region distribution map corresponding to each video frame. The determination module is used to determine the initial positioning information of the object of interest in each video frame based on the initial motion region distribution map; The comparison and acquisition module is used to perform a correlation comparison of the position changes of the objects of interest in each video frame based on the initial positioning information of the objects of interest in each video frame, and obtain the motion trajectory distribution characteristics of the objects of interest. The correction output module is used to correct each video frame based on the motion trajectory distribution characteristics of the object of interest, obtain the target video stream based on the corrected video frames, and output the three-dimensional reconstruction result based on the target video stream.

Images