A method, system, electronic device, and storage medium for synchronizing visual speed matching.

Abstract

This invention relates to the field of stereo vision technology and discloses a synchronous velocity visual matching method, system, electronic device, and storage medium. The method includes: synchronously acquiring stereo video sequences with a predefined frame rate; performing multi-target hybrid tracking and motion-induced detection, outputting target motion information containing position and velocity vectors; extracting hierarchical motion features of the targets from multiple consecutive frames of the stereo video sequence and performing unified spatiotemporal encoding; calculating scale cosine similarity, orientation similarity, and trajectory consistency metrics under the geometric constraints of stereo vision, and using these as observational evidence input into a probabilistic inference model for fusion, outputting a posterior probability characterizing the reliability of the matching; adjusting the weight allocation of velocity similarity and orientation similarity according to the motion characteristics of the targets in the scene to establish a stable correspondence matching relationship between targets in the left and right views. This invention can utilize high temporal resolution information, effectively fuse motion features and geometric constraints, and possesses adaptive capabilities.

Description

Technical Field

[0001] This invention relates to the field of stereoscopic vision technology, and specifically to a synchronous speed visual matching method, system, electronic device, and storage medium. Background Technology

[0002] Stereo vision is one of the core technologies in computer vision. It simulates human binocular vision by finding corresponding points in images captured by two cameras (usually left and right cameras) from two different viewpoints. It then uses the principle of parallax to calculate depth information, thereby obtaining the three-dimensional geometric structure of a scene. Stereo matching, the process of establishing correspondences between pixels or feature points in the left and right images, is a crucial step in stereo vision. Its performance directly determines the accuracy of subsequent tasks such as 3D reconstruction, object tracking, and behavior understanding.

[0003] Existing stereo matching methods can be mainly classified into the following categories:

[0004] Methods based on local features, such as Sum of Absolute Differences (SAD) and Normalized Cross-Correlation (NCC), estimate disparity by calculating the pixel intensity similarity within a fixed or adaptive window. These methods are computationally efficient, but are sensitive to texture loss and duplicate texture regions, and their matching accuracy is significantly affected by the window size.

[0005] Global optimization-based methods, such as graph cut, belief propagation, and semi-global matching (SGM), solve for the globally or quasi-globally optimal disparity map by constructing and minimizing an energy function that includes data items and smoothing terms. These methods outperform local methods in weakly textured regions, but they have high computational complexity and poor real-time performance.

[0006] Deep learning-based methods, such as GC-Net, PSMNet, and RAFT-Stereo, utilize deep convolutional neural networks to learn and predict disparity end-to-end. These methods have achieved breakthroughs in accuracy, but their performance heavily relies on training on large-scale labeled data, and their model generalization ability and interpretability remain challenging.

[0007] However, the above methods generally suffer from the following technical bottlenecks when dealing with complex scenarios such as multiple targets with similar appearances and high-speed movement:

[0008] Strong dependence on appearance features: When the observed targets have similar or identical appearance features such as color and texture (e.g., uniformly dressed people, animals of the same kind, or industrial parts with symmetrical structures), traditional appearance-based matching features are difficult to provide effective distinguishing information, leading to matching confusion and errors.

[0009] Insufficient temporal consistency: At normal frame rates (e.g., 30fps to 60fps), the sampling interval for motion information is relatively long, making it difficult to accurately capture the instantaneous motion state of the target. This leads to delays and distortions in the estimation of the target's motion trajectory's speed and direction, and makes it difficult to guarantee strict synchronization between the images acquired by the left and right cameras, further exacerbating the uncertainty in temporal matching.

[0010] Insufficient utilization of motion information: Most existing matching methods that utilize motion features only use simple velocity or displacement vector similarity for measurement, lacking a detailed characterization of the temporal continuity of motion information, and failing to deeply integrate and jointly model motion features with the geometric constraints of the stereo vision system.

[0011] Poor algorithm adaptability: The parameters of traditional matching algorithms are usually fixed, and they cannot dynamically adjust the matching strategy according to the movement pattern of the target in the scene (such as periodic movement, random movement, sudden high-speed movement), resulting in a decrease in robustness and accuracy in complex dynamic scenes.

[0012] Therefore, this application provides a synchronous speed visual matching method to solve the above-mentioned technical problems. Summary of the Invention

[0013] The purpose of this invention is to provide a synchronous speed visual matching method, system, electronic device and storage medium to solve the technical problem that the prior art cannot solve the accurate and robust matching of multiple targets with similar appearance in high-speed motion scenes.

[0014] To address the aforementioned technical problems, this invention provides a method for synchronous speed visual matching, comprising:

[0015] Synchronously acquire stereo video sequences at a predefined frame rate and dynamically adjust image acquisition parameters based on a target motion prediction model;

[0016] Multi-target hybrid tracking and motion-induced detection are performed on stereo video sequences. By combining target detection and template matching that run at different time intervals, a multi-hypothesis tracking framework is constructed to maintain multiple matching hypotheses. Inter-frame target association is completed based on the first matching algorithm, and a motion state estimation algorithm is applied to optimize the motion trajectory. The output includes target motion information containing position and velocity vectors.

[0017] Hierarchical motion features of the target are extracted from multiple consecutive frames of a stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and are uniformly coded in spatiotemporal.

[0018] Under the geometric constraints of stereo vision, the scale cosine similarity based on the basic velocity vector feature set, the orientation similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets are calculated and used as observation evidence to input the probabilistic inference model for fusion, and the output is the posterior probability representing the reliability of the matching.

[0019] The weights of velocity similarity and orientation similarity are adjusted according to the motion characteristics of the targets in the scene. A joint optimization energy function containing a weighted similarity term, a geometric consistency term, and a time smoothness constraint term is constructed and solved based on the second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views.

[0020] In some specific embodiments, the synchronous acquisition of stereoscopic video sequences at a predefined frame rate, and the dynamic adjustment of image acquisition parameters based on a target motion prediction model, further includes:

[0021] Acquire predefined high frame rate stereoscopic video streams;

[0022] Using motion prediction models based on Kalman filtering or recurrent neural networks, the position and velocity state of a target at future time points can be predicted;

[0023] Based on the predicted target motion state, the acquisition parameters, including exposure time, frame rate and white balance, are adjusted in real time.

[0024] The system continuously optimizes the acquisition parameter settings through a feedback control mechanism, and automatically adjusts the parameters when the target's motion state changes.

[0025] In some specific embodiments, multi-target hybrid tracking and motion-induced detection are performed on the stereo video sequence. A multi-hypothesis tracking framework is constructed by combining target detection and template matching run at different time intervals to maintain multiple matching hypotheses. Inter-frame target association is completed based on a first matching algorithm, and a motion state estimation algorithm is applied to optimize the motion trajectory, outputting target motion information containing position and velocity vectors. Further, this includes:

[0026] Target detection based on a deep learning model is performed at the first detection interval to obtain the bounding rectangle coordinates, width and height dimensions, category label, and confidence information of the target.

[0027] The target template library is updated based on the detection results, and a sliding window mechanism is used to maintain the target appearance templates of the most recent frames.

[0028] Template matching based on the absolute difference algorithm is performed at a second tracking interval that is less than the first detection interval to locate the target position between consecutive frames;

[0029] A multi-hypothesis tracking framework is constructed, which maintains multiple candidate matching hypotheses for each target and gradually eliminates erroneous matches through multi-frame evidence accumulation and trajectory consistency testing.

[0030] Construct the Euclidean distance matrix of the target position between consecutive frames, apply the first matching algorithm to perform optimal association matching, and use the Kalman filter algorithm to smooth and predict the target motion state.

[0031] In some specific embodiments, hierarchical motion features of the target are extracted from multiple consecutive frames of a stereoscopic video sequence. These hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained through frequency domain analysis. A unified spatiotemporal coding process is then performed, further including:

[0032] Pixel-level motion information of each target is extracted from multiple consecutive frames of the stereoscopic video sequence, including horizontal displacement components and vertical displacement components, to form the basic velocity vector feature set;

[0033] By calculating the ratio of the dot product to the magnitude of the velocity vectors between adjacent frames, the acceleration and steering angle features representing changes in motion direction are derived, forming the motion primitive feature set.

[0034] A fast Fourier transform is performed on the target motion trajectory to extract the dominant frequency components and power spectrum features in the frequency domain, and the motion periodic feature set is constructed.

[0035] The basic velocity vector feature set, motion primitive feature set, and motion periodic feature set are normalized in the spatiotemporal dimension, and a comprehensive motion feature descriptor is formed by concatenating the features.

[0036] In some specific embodiments, under the geometric constraints of stereo vision, the calculation of scale cosine similarity based on the basic velocity vector feature set, orientation similarity based on the motion primitive feature set, and trajectory consistency measure based on epipolar geometry between the left and right view targets further includes:

[0037] Calculate the scale-normalized cosine similarity between the basic velocity vector feature sets of corresponding targets in the left and right cameras, and use it as the velocity similarity;

[0038] Calculate the directional consistency between the motion primitive feature sets of corresponding targets in the left and right cameras, and calculate the directional similarity based on the absolute difference of the feature values;

[0039] Based on camera calibration parameters and epipolar geometric constraints, the reprojection error of the target motion trajectory in the left and right views is calculated and used as a trajectory consistency measure.

[0040] Within the Bayesian inference framework, the velocity similarity, orientation similarity, and trajectory consistency measures are considered as input observational evidence for conditional probabilities.

[0041] By combining the prior matching probability, the posterior matching probability distribution is calculated using the probability product rule.

[0042] In some specific embodiments, the observational evidence is input into a probabilistic inference model for fusion, and the output is a posterior probability representing the reliability of the match, further including:

[0043] A Bayesian probabilistic graphical model is established, and the velocity similarity, orientation similarity, and trajectory consistency metrics are defined as observation nodes;

[0044] Based on the conditional independence assumption, the likelihood probability function of each observed variable under the given matching condition is calculated respectively;

[0045] Multiply the prior matching probability by each likelihood probability to obtain the unnormalized joint probability distribution;

[0046] The probability is normalized by summing the marginal probabilities to obtain the standardized posterior matching probability.

[0047] Output the final matching probability value and matching decision result for each target pair.

[0048] In some specific embodiments, the weighting of velocity similarity and orientation similarity is adjusted according to the motion characteristics of the targets in the scene, a joint optimization energy function including a weighted similarity term, a geometric consistency term, and a temporal smoothness constraint term is constructed, and solved based on a second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views, further including:

[0049] Based on the scene's field of view and the target's motion speed range, a linear interpolation strategy is used to dynamically allocate the weights of the velocity similarity and orientation similarity.

[0050] Construct the joint optimization energy function, which includes data terms, geometric constraint terms, and time smoothing terms, wherein the data terms consist of weighted velocity similarity and orientation similarity.

[0051] The geometric constraint term consists of a trajectory consistency measure based on epipolar geometry, and the temporal smoothing term consists of the consistency of matching results between adjacent frames.

[0052] The second matching algorithm is used to solve for the global optimal matching of the similarity matrix derived from the energy function;

[0053] Based on the final solution, a one-to-one correspondence is established between the targets in the left and right views, and a stable stereo matching result is output.

[0054] Based on the same concept, the present invention also provides a synchronous speed visual matching system, comprising:

[0055] The stereo video sequence acquisition module is configured to synchronously acquire stereo video sequences at a predefined frame rate and dynamically adjust image acquisition parameters based on a target motion prediction model.

[0056] The motion trajectory optimization module is configured to perform multi-target hybrid tracking and motion-induced detection on the stereo video sequence. It combines target detection and template matching that run at different time intervals to build a multi-hypothesis tracking framework to maintain multiple matching hypotheses. Based on the first matching algorithm, it completes inter-frame target association and applies a motion state estimation algorithm to optimize the motion trajectory, outputting target motion information containing position and velocity vectors.

[0057] The spatiotemporal coding definition module is configured to extract hierarchical motion features of the target from multiple consecutive frames of a stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and perform unified spatiotemporal coding.

[0058] The posterior probability output module is configured to calculate the scale cosine similarity based on the basic velocity vector feature set, the orientation similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets under the geometric constraints of stereo vision. These are then used as observation evidence inputs to the probabilistic inference model for fusion, and the output is a posterior probability representing the reliability of the matching.

[0059] The matching relationship establishment module is configured to adjust the weight allocation of velocity similarity and orientation similarity according to the motion characteristics of the targets in the scene, construct a joint optimization energy function including weighted similarity term, geometric consistency term and time smoothness constraint term, and solve it based on the second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views.

[0060] Based on the same concept, the present invention also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of a synchronous speed visual matching method.

[0061] Based on the same concept, the present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a synchronous speed visual matching method.

[0062] Compared with existing technologies, its advantages are as follows:

[0063] This invention discloses a synchronous velocity visual matching method, system, electronic device, and storage medium, which effectively improves the matching and differentiation capability of similar-looking targets: by extracting and utilizing hierarchical motion features including micro-velocities, meso-motion primitives, and macro-behavioral patterns, a unique "motion fingerprint" is constructed for each target. This enables accurate matching based on the differences in motion features when the appearance features of targets are difficult to distinguish, thus solving the matching confusion problem of multiple similar-looking targets.

[0064] Improved synchronization and estimation accuracy of motion information: By employing a high frame rate (no less than 200fps) stereo camera for synchronous acquisition, combined with an active vision control mechanism, the motion sampling interval was shortened, ensuring high synchronization of left and right view acquisition. This enables the acquisition of more accurate and timely short-term velocity information of the target, effectively overcoming the motion delay and distortion caused by low frame rate systems.

[0065] A deep fusion and joint optimization of motion features and solid geometry was achieved: a multi-dimensional similarity calculation framework combining scale cosine similarity, orientation similarity, and epipolar consistency metrics was designed and fused within a Bayesian probabilistic framework, organically unifying temporally continuous motion information with spatially consistent geometric constraints. This enhanced the robustness of the matching.

[0066] The algorithm's adaptability to different motion scenarios has been enhanced: by introducing a dynamic weight adjustment strategy and a joint optimization energy function that includes a time smoothing term, the matching model can adaptively adjust the focus of the matching criteria according to the dynamic characteristics of the scene, such as the field of view and the target's motion speed. Attached Figure Description

[0067] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0068] Figure 1 This is a flowchart illustrating some specific embodiments of the synchronous speed visual matching method of the present invention;

[0069] Figure 2 This is a schematic diagram of the structure of a synchronous speed visual matching system according to some specific embodiments of the present invention;

[0070] Figure 3 This is a schematic diagram of the structure of an electronic device according to some specific embodiments of the present invention;

[0071] In the diagram, 710 is the processor; 720 is the memory; 730 is the input device; and 740 is the output device. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0074] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0075] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0076] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0077] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0078] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0079] Reference Figure 1 A method for synchronous speed visual matching, comprising:

[0080] S101 synchronously acquires stereo video sequences at a predefined frame rate and dynamically adjusts image acquisition parameters based on a target motion prediction model;

[0081] S102, perform multi-target hybrid tracking and motion-induced detection on the stereo video sequence, combine target detection and template matching that run at different time intervals, construct a multi-hypothesis tracking framework to maintain multiple matching hypotheses, complete inter-frame target association based on the first matching algorithm, and apply a motion state estimation algorithm to optimize the motion trajectory, outputting target motion information containing position and velocity vectors;

[0082] S103, extract hierarchical motion features of the target from multiple consecutive frames of the stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and perform unified spatiotemporal coding.

[0083] S104, under the geometric constraints of stereo vision, calculates the scale cosine similarity based on the basic velocity vector feature set, the direction similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets, and uses them as observation evidence to input the probabilistic inference model for fusion, and outputs the posterior probability representing the reliability of the matching.

[0084] S105, adjust the weight allocation of velocity similarity and orientation similarity according to the motion characteristics of the target in the scene, construct a joint optimization energy function including weighted similarity term, geometric consistency term and time smoothness constraint term, and solve it based on the second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views.

[0085] Specifically, in this embodiment of the invention, a stereoscopic video sequence with a predefined frame rate is acquired through a hardware-synchronized dual-camera system, and the acquisition parameters are dynamically adjusted based on a target motion prediction model to optimize motion information capture. Multi-target hybrid tracking and motion-induced detection are performed on the acquired stereoscopic video sequence. By combining a target detection algorithm that runs periodically at different time intervals and a fast tracking algorithm based on template matching, a multi-hypothesis tracking framework that maintains multiple possible matching hypotheses is constructed. The first matching algorithm completes the association and correspondence of targets between consecutive frames, and a motion state estimation algorithm is applied to smooth and optimize the target motion trajectory, ultimately outputting target motion information containing precise position and velocity vectors. Hierarchical motion features of each target are extracted from consecutive frames of the stereoscopic video sequence. These hierarchical motion features include a basic velocity vector feature set composed of velocity vectors calculated from pixel-level displacements, a motion primitive feature set containing acceleration and steering angle obtained by calculating the directional change relationship between adjacent velocity vectors, and a... A motion periodic feature set representing periodic characteristics is obtained through frequency domain transformation analysis of the motion trajectory. These three levels of features are then uniformly encoded in a spatiotemporal dimension to form a comprehensive descriptor. Under the epipolar geometric constraints of the stereo vision system, the scale-normalized cosine similarity based on the basic velocity vector feature set, the direction absolute difference similarity based on the motion primitive feature set, and the consistency measure based on the degree to which the motion trajectory satisfies the epipolar geometric constraints are calculated between the left and right view targets. These calculated similarities are used as observation evidence input into a Bayesian inference-based probability model for fusion calculation, and the posterior probability value representing the matching reliability is output. The weight allocation of velocity similarity and direction similarity in the target optimization is dynamically adjusted according to the motion characteristics of the target in the current scene. A joint optimization energy function containing a weighted similarity term, a geometric consistency constraint term, and an inter-frame temporal smoothness constraint term is constructed. The energy function is solved based on the second matching algorithm to establish a stable and reliable correspondence matching relationship between the left and right view targets.

[0086] For example, a stereo vision system is composed of two synchronized high-speed USB cameras, with a frame rate of 200 frames per second and an exposure time of 2 milliseconds. The camera acquisition parameters are adjusted in real-time based on the output of a Kalman filter motion prediction model. In multi-target hybrid tracking and motion-induced detection, the target detection execution time interval is set to 10 milliseconds, and the template matching execution time interval is set to 5 milliseconds. A multi-hypothesis tracking framework is constructed to maintain multiple candidate matching hypotheses for each target. The Hungarian matching algorithm is used as the first matching algorithm to complete inter-frame target association, and the Kalman filter algorithm is applied to optimize the target motion state, outputting motion information including position coordinates and velocity vectors. In the feature extraction stage, a sequence of 64 consecutive frames is selected, and the pixel-level velocity vector of each target in each frame is calculated to form a basic feature set. The cosine value of the angle between the velocity vectors of adjacent frames is calculated to obtain the motion primitive feature set. A fast Fourier transform is performed on the motion trajectory to extract the dominant frequency components, forming a motion periodic feature set. These three types of features are then combined... After standardization, the components are concatenated to form a spatiotemporal code. In the similarity calculation and fusion stage, the scale cosine similarity of the feature sets of the left and right target velocity vectors is calculated to obtain the velocity similarity value, the average absolute difference of the feature sets of motion primitives is calculated to obtain the orientation similarity value, and the reprojection error of the motion trajectory is calculated based on the camera calibration parameters to obtain the trajectory consistency metric. These similarity values ​​are input into the Bayesian inference model. Assuming the prior matching probability is 0.5, the posterior matching probability is obtained by calculating the product of the likelihood probabilities. In the optimization solution stage, based on the scene field of view and the target motion speed, the velocity similarity weight is set to 0.6 and the orientation similarity weight is set to 0.4. A joint optimization energy function containing a weighted similarity term, a geometric consistency term, and a temporal smoothing term is constructed. The geometric consistency weight coefficient is set to 0.3 and the temporal smoothing term weight coefficient is set to 0.2. Finally, the Hungarian matching algorithm is used as the second matching algorithm to solve the constructed similarity matrix optimally, establishing a one-to-one correspondence between the targets in the left and right views.

[0087] In some of these applications, stereoscopic video sequences with a predefined frame rate are acquired synchronously, and image acquisition parameters are dynamically adjusted based on a target motion prediction model, including acquiring a predefined high frame rate stereoscopic video stream; using a motion prediction model based on Kalman filtering or a recurrent neural network to predict the target's position and velocity state at future moments; adjusting acquisition parameters, including exposure time, frame rate, and white balance, in real time according to the predicted target motion state; and continuously optimizing acquisition parameter settings through a feedback control mechanism, automatically adjusting parameters when the target motion state changes.

[0088] Understandably, a hardware-synchronized dual-camera system acquires a predefined high-frame-rate stereoscopic video stream to ensure strict synchronization between the left and right views; a motion prediction model based on Kalman filtering or recurrent neural networks is used to predict and estimate the position coordinates and velocity vector state of targets in the scene at future moments; based on the predicted target motion state information, the acquisition parameters of the dual-camera system are dynamically adjusted in real time, including exposure time, image acquisition frame rate, and white balance settings; a feedback control mechanism is constructed to continuously optimize the configuration of acquisition parameters, automatically triggering the parameter adjustment process when a significant change in the target motion state is detected, thereby ensuring that high-quality image data can be obtained under different motion states.

[0089] For example, a stereo vision system is composed of two high-speed USB cameras (model DFK 37BUX273). The initial acquisition frame rate is set to 200 frames per second, the exposure time to 2 milliseconds, and the white balance to 5500K. A Kalman filter is used as the motion prediction model to predict the target's position and velocity state in the next 2 milliseconds based on the target's motion trajectory in the first 5 frames. The prediction results show that the target's displacement in the horizontal direction is 3.2 pixels and the displacement in the vertical direction is 1.5 pixels. Based on this prediction result, the exposure time is adjusted to 1.8 milliseconds, the acquisition frame rate is increased to 220 frames per second, and the white balance remains unchanged at 5500K. Through the established feedback control mechanism, the mean square error between the actual motion state and the predicted state of the target is continuously monitored. When the error exceeds the preset threshold of 0.1, the parameters are immediately readjusted to ensure that a clear, moderately bright, and color-accurate stereo image sequence can be obtained under different motion states.

[0090] In some applications, multi-target hybrid tracking and motion-induced detection are performed on stereo video sequences. A multi-hypothesis tracking framework is constructed by combining target detection and template matching run at different time intervals to maintain multiple matching hypotheses. Inter-frame target association is completed based on a first matching algorithm, and a motion state estimation algorithm is applied to optimize the motion trajectory. The output includes target motion information containing position and velocity vectors. This includes performing deep learning-based target detection at a first detection interval to obtain the target's bounding rectangle coordinates, width and height dimensions, category label, and confidence information; updating the target template library based on the detection results and maintaining target appearance templates for the most recent frames using a sliding window mechanism; performing template matching based on an absolute difference algorithm at a second tracking interval less than the first detection interval to locate the target position between consecutive frames; constructing a multi-hypothesis tracking framework, maintaining multiple candidate matching hypotheses for each target, and gradually eliminating erroneous matches through multi-frame evidence accumulation and trajectory consistency checks; constructing the Euclidean distance matrix of the target position between consecutive frames; applying the first matching algorithm for optimal association matching; and using a Kalman filter algorithm to smooth and predict the target motion state.

[0091] Understandably, the process involves: performing target detection based on a deep learning model at a first detection interval to obtain the bounding rectangle coordinates, width and height dimensions, category label, and detection confidence information for each target in the scene; dynamically updating the target template library based on the detection results and using a sliding window mechanism to maintain the target appearance templates of the most recent frames to ensure template timeliness; performing template matching based on an absolute difference algorithm at a second tracking interval less than the first detection interval to quickly and accurately locate the target position between consecutive frames; constructing a multi-hypothesis tracking framework to maintain multiple candidate matching hypotheses for each target, and gradually eliminating erroneous matches through multi-frame evidence accumulation and trajectory consistency checks to improve tracking robustness; constructing the Euclidean distance matrix of the target position between consecutive frames, applying the first matching algorithm for optimal association matching, and using the Kalman filter algorithm to smooth and predict the target motion state, ultimately outputting target motion information containing precise position information and velocity vectors.

[0092] For example, the first detection interval for object detection is set to 20 milliseconds, and the second tracking interval for template matching is set to 5 milliseconds; the YOLOv5 model is used as a deep learning object detector, and the detection output includes the bounding rectangle coordinates of the target [x=320 pixels, y=240 pixels], width and height dimensions [width=60 pixels, height=80 pixels], the category label "hand", and a detection confidence score of 0.92; a sliding window mechanism is used to maintain the target appearance template of the most recent 10 frames, and the template size is set to 64×64 pixels; template matching is performed between consecutive frames based on the absolute difference and the algorithm, and the difference between the candidate region in the current frame and the template is calculated. When the difference is less than the threshold of 15, it is confirmed as the same target; multiple hypotheses are constructed. The tracking framework maintains three candidate matching hypotheses for each target and eliminates erroneous matches with a cumulative difference exceeding 50 through trajectory consistency checks over five consecutive frames. A 5×5 Euclidean distance matrix is ​​constructed for the five target positions between consecutive frames. The Hungarian algorithm is applied as the first matching algorithm to solve for the optimal association, with a matching cost threshold of 25 pixels. The Kalman filter algorithm is used to smooth the target motion state. The state vector includes position [x=325.3 pixels, y=242.1 pixels] and velocity [vx=12.5 pixels / second, vy=8.2 pixels / second]. The process noise covariance is set to 0.1, and the observation noise covariance is set to 0.5. Finally, the optimized target motion information is output for subsequent processing.

[0093] In some applications, hierarchical motion features of a target are extracted from multiple consecutive frames of a stereoscopic video sequence. These hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained through frequency domain analysis. A unified spatiotemporal encoding process is then performed. This includes extracting pixel-level motion information of each target from multiple consecutive frames of the stereoscopic video sequence, including horizontal and vertical displacement components, to form the basic velocity vector feature set; deriving acceleration and steering angle features representing changes in motion direction by calculating the ratio of the dot product to the magnitude of velocity vectors between adjacent frames, thus forming the motion primitive feature set; performing a Fast Fourier Transform on the target's motion trajectory to extract the dominant frequency components and power spectrum features in the frequency domain, thus constructing the motion periodicity feature set; and normalizing the basic velocity vector feature set, motion primitive feature set, and motion periodicity feature set in the spatiotemporal dimension, then using a feature concatenation method to form a comprehensive motion feature descriptor.

[0094] Understandably, pixel-level motion information of each target is extracted from multiple consecutive frames of a stereoscopic video sequence, including horizontal and vertical displacement components. These displacement components form a basic velocity vector feature set representing the target's fundamental motion state. By calculating the ratio of the dot product to the magnitude of the velocity vectors between adjacent frames, acceleration and steering angle features representing changes in motion direction are derived. These features together constitute a motion primitive feature set reflecting subtle changes in the target's motion. A fast Fourier transform is performed on the target's motion trajectory to extract dominant frequency components and power spectrum features from the frequency domain, constructing a motion periodic feature set that can characterize the periodicity of the target's motion. Finally, the obtained basic velocity vector feature set, motion primitive feature set, and motion periodic feature set are normalized in the spatiotemporal dimension, and features from different levels are combined using a feature concatenation method to form a unified comprehensive motion feature descriptor.

[0095] For example, hierarchical motion features of the target are extracted from 64 consecutive frames of a stereoscopic video sequence. The horizontal and vertical displacement components of the target in each frame are calculated, forming a basic velocity vector feature set containing 64 velocity vectors. The velocity vector of a specific frame is [horizontal displacement 3.2 pixels, vertical displacement 1.8 pixels]. The ratio of the dot product to the magnitude of the velocity vectors between adjacent frames is calculated. For example, the dot product of the velocity vectors of frame n and frame (n+1) is 156.8, and the product of their magnitudes is 164.2, resulting in a ratio of 0.95. The corresponding turning angle is 18.2 degrees, forming a feature set containing 64 velocity vectors. The feature set of motion primitive features; the motion trajectory of the target in 64 frames is subjected to fast Fourier transform to extract the dominant frequency component 2.5Hz and its corresponding power spectral density 0.86 in the frequency domain, and a motion periodicity feature set is constructed; the three feature sets are normalized, the basic velocity vector feature set is normalized to the range of [0,1], the motion primitive feature set is normalized to the range of [-1,1], and the motion periodicity feature set is normalized to the range of [0,1]. The 127 feature values ​​are combined into a unified comprehensive motion feature descriptor by feature concatenation, which is used for subsequent similarity calculation and matching processing.

[0096] In some applications, under the geometric constraints of stereo vision, the scale cosine similarity, orientation similarity, and trajectory consistency measure based on epipolar geometry are calculated between targets in the left and right views. This includes calculating the scale-normalized cosine similarity between the basic velocity vector feature sets of corresponding targets in the left and right cameras as the velocity similarity; calculating the orientation consistency between the orientation feature sets of corresponding targets in the left and right cameras, and calculating the orientation similarity based on the absolute difference of the eigenvalues; calculating the reprojection error of the motion trajectory of the targets in the left and right views based on camera calibration parameters and epipolar geometry constraints, as the trajectory consistency measure; under the Bayesian inference framework, the velocity similarity, orientation similarity, and trajectory consistency measure are regarded as input observation evidence for conditional probabilities; and the posterior matching probability distribution is calculated by combining the prior matching probability with the probability product rule.

[0097] Understandably, the scale-normalized cosine similarity between the basic velocity vector feature sets of corresponding targets in the left and right cameras is used as a velocity similarity measure; the directional consistency between the motion primitive feature sets of corresponding targets in the left and right cameras is calculated, and the directional similarity measure is obtained based on the absolute difference of the feature values; based on the camera calibration parameters and epipolar geometric constraints, the reprojection error of the target motion trajectory in the left and right views is calculated as a trajectory consistency measure; under the Bayesian inference framework, the calculated velocity similarity, directional similarity, and trajectory consistency measures are regarded as input observation evidence for conditional probabilities; combined with the pre-set prior matching probability, the posterior matching probability distribution is calculated through the probability product rule.

[0098] For example, the basic velocity vector feature sets of two corresponding targets in the left and right cameras are calculated. The velocity vector of the target in the left camera is [3.2 pixels, 1.5 pixels], and the velocity vector of the target in the right camera is [3.1 pixels, 1.6 pixels]. The velocity similarity calculated using scale-normalized cosine similarity is 0.92. The motion primitive feature sets of the two targets are calculated. The motion primitive feature value of the target in the left camera is 0.85, and the motion primitive feature value of the target in the right camera is 0.83. The directional similarity calculated based on the absolute difference of the feature values ​​is 0.98. Based on the fundamental matrix and epipolar geometric constraints in the camera calibration parameters, the left... The reprojection error of the target's trajectory in the right view is 1.2 pixels, which translates to a trajectory consistency metric of 0.88. Under the Bayesian inference framework, the prior matching probability is set to 0.5. The velocity similarity (0.92), orientation similarity (0.98), and trajectory consistency metric (0.88) are used as observational evidence, and the likelihood probabilities are calculated to be 0.90, 0.95, and 0.85, respectively. The unnormalized posterior matching probability is calculated using the probability product rule to be 0.5 × 0.90 × 0.95 × 0.85 = 0.363. After probability normalization, the final posterior matching probability is 0.78.

[0099] In some applications, the observational evidence is input into a probabilistic inference model for fusion, and the output is a posterior probability representing the reliability of the match. This includes establishing a Bayesian probabilistic graphical model, defining the velocity similarity, direction similarity, and trajectory consistency measures as observation nodes; calculating the likelihood probability function of each observation variable under a given matching condition based on the conditional independence assumption; multiplying the prior matching probability by each likelihood probability to obtain an unnormalized joint probability distribution; performing probability normalization by summing marginal probabilities to obtain a standardized posterior matching probability; and outputting the final matching probability value and matching decision result for each target pair.

[0100] Understandably, a Bayesian probabilistic graphical model is established, defining three observation nodes: velocity similarity, orientation similarity, and trajectory consistency metrics. Based on the conditional independence assumption, the likelihood probability function of each observed variable is calculated under a given matching condition. The pre-set prior matching probability is multiplied by the likelihood probability of each observed variable to obtain an unnormalized joint probability distribution. The probability is normalized by calculating the sum of the marginal probabilities of all possible matching hypotheses to obtain the standardized posterior matching probability. Finally, the final matching probability value of each target pair and the matching decision result based on the probability threshold are output.

[0101] For example, a Bayesian probabilistic graphical model with three observation nodes is established, where the velocity similarity observation node has a value of 0.92, the orientation similarity observation node has a value of 0.95, and the trajectory consistency measure observation node has a value of 0.88. Based on the conditional independence assumption, the likelihood probability of velocity similarity is calculated to be 0.90, the likelihood probability of orientation similarity is 0.93, and the likelihood probability of trajectory consistency measure is 0.85 under a given matching condition. The prior matching probability is set to 0.6, and the unnormalized joint probability value is obtained by multiplying the prior probability by the three likelihood probabilities: 0.6 × 0.90 × 0.93 × 0.85 = 0.427. The sum of the marginal probabilities under the matching and non-matching assumptions is calculated to be 0.515. After normalization, the standardized posterior matching probability is obtained as 0.427 / 0.515 = 0.83. The matching decision threshold is set to 0.75. Since 0.83 is greater than 0.75, the final matching probability value of the target pair is output as 83%, and the matching decision result is a successful match.

[0102] In some applications, the weights of velocity similarity and orientation similarity are adjusted according to the motion characteristics of targets in the scene. A joint optimization energy function containing a weighted similarity term, a geometric consistency term, and a temporal smoothness constraint term is constructed and solved based on a second matching algorithm to establish a stable correspondence between targets in the left and right views. This includes dynamically allocating the weights of velocity similarity and orientation similarity using a linear interpolation strategy based on the scene's field of view and the target's motion velocity range; constructing the joint optimization energy function containing a data term, a geometric constraint term, and a temporal smoothness term, where the data term consists of weighted velocity similarity and orientation similarity; the geometric constraint term consists of a trajectory consistency measure based on epipolar geometry; and the temporal smoothness term consists of the consistency of matching results between adjacent frames. The second matching algorithm is used to solve for the global optimal matching of the similarity matrix derived from the energy function. Based on the final solution, a one-to-one correspondence between targets in the left and right views is established, and a stable stereo matching result is output.

[0103] Understandably, based on the scene's field of view and the target's velocity range, a linear interpolation strategy is used to dynamically allocate the weight ratios of velocity similarity and orientation similarity in the energy function. A joint optimization energy function is constructed, comprising three key components: the data term is a linear combination of weighted velocity and orientation similarity; the geometric constraint term is a trajectory consistency measure based on epipolar geometry; and the temporal smoothing term is the consistency of matching results between adjacent frames. A second matching algorithm is used to solve for the global optimal matching of the comprehensive similarity matrix derived from the energy function. Based on the algorithm's solution, a stable one-to-one correspondence is established between targets in the left and right views, and the final stereo matching result is output.

[0104] For example, if the current scene field of view is detected to be 60 degrees and the average target speed is 150 pixels / second, a linear interpolation strategy is used to calculate the weight allocation: when the field of view is 60 degrees, the basic weight for direction similarity is 0.5; when the speed is 150 pixels / second, the basic weight for speed similarity is 0.7. The final weights are obtained by weighted averaging: speed similarity weight 0.6 and direction similarity weight 0.4. A joint optimization energy function E = 0.6 × S is then constructed. v +0.4×S a +0.3×S e +0.2×S t S v With a velocity similarity value of 0.92, S a Let the directional similarity value be 0.95, S e The trajectory consistency metric is set to 0.88, S t With a consistency value of 0.90 for adjacent frame matching, the calculated energy function value is 0.6×0.92+0.4×0.95+0.3×0.88+0.2×0.90=1.238. The Hungarian algorithm is used as the second matching algorithm to solve the 5×5 similarity matrix composed of 5 targets. Each element in the matrix is ​​calculated from the energy function value of the corresponding target pair. After iterative optimization, the optimal matching result is output: a one-to-one correspondence is established between the left view target [1,2,3,4,5] and the right view target [3,1,5,2,4], and the matching confidence is higher than 85%. Finally, a stable stereo matching result is output.

[0105] The following describes another embodiment of the synchronous speed visual matching method of the present invention:

[0106] This embodiment includes:

[0107] High-speed stereo video acquisition and active vision control: High-speed stereo cameras (composed of two high-speed USB 3.0 cameras) with a frame rate of no less than 200fps are used to synchronously acquire scene images and obtain high frame rate (HFR) stereo video from the left and right cameras; an active vision control mechanism is introduced: Based on the target motion prediction model (such as Kalman filter or LSTM network), the camera exposure time and frame rate are adjusted in real time to optimize the observation angle and ensure low latency, high synchronization and high quality acquisition of motion information.

[0108] Multi-target hybrid tracking and motion-induced detection: A hybrid tracking algorithm is constructed by combining target detection and template matching, and a motion-induced detection mechanism is introduced: ( Interval between adjacent HFR frames To perform object detection at intervals (e.g., using MediaPipe for hand detection, or OpenCV for marker detection), obtain the target image patch coordinates, width, height, confidence level, and category information, and update the target template; For intervals, the Sum of Absolute Differences (SAD) algorithm is used for template matching based on the latest template to quickly locate the target position; a multi-hypothesis tracking framework is constructed: multiple possible matching hypotheses are maintained simultaneously, and disambiguation is gradually achieved through time accumulation, with delayed decision-making to handle temporary target occlusion or loss; the previous frame is constructed. Target and current frame The Euclidean distance matrix of each target is used to achieve inter-frame correlation of targets through the Hungarian matching algorithm, and motion estimation is optimized by Kalman filtering.

[0109] Hierarchical Motion Feature Extraction and Spatiotemporal Coding: Extracting multi-level motion features for each target in N consecutive HFR images:

[0110] Microscopic layer: pixel-level velocity vector (including horizontal component) ; and vertical component This constitutes the high synchronous short-term velocity (STV) feature set. ;in, The velocity vector of the nth frame is represented by... and composition;

[0111] Mesoscopic layer: Motion element characteristics, including acceleration, steering angle, etc., calculated by measuring the cosine of the angle between adjacent velocity vectors. ,in, These represent the magnitudes of the vectors, forming a directional feature set. ;in, This indicates the change in motion direction between frame n and frame (n+1).

[0112] Macro level: Behavioral pattern features. Motion periodic features are extracted through frequency domain analysis (such as FFT) to construct "motion fingerprint" coding, which enhances the ability to distinguish similar moving targets.

[0113] Similarity calculation and probability fusion under solid geometric constraints:

[0114] Deeply integrate stereo geometric constraints into similarity calculation: calculate the scale cosine similarity of the STV feature sets of the left and right cameras, using the following formula: ;(in , Let be the velocity vectors of the targets in the left and right cameras, respectively, in the k-th frame. (where the vector length is 0); calculate the directional similarity of the target orientation feature sets of the left and right cameras using the formula: ;in, This represents the directional feature value of the left camera target in the kth frame (from the directional feature set A). This represents the directional feature value of the target on the right camera in the k-th frame;

[0115] Introducing an epipolar consistency metric: Calculating the spatial consistency score of the target's motion trajectory in the left and right views based on epipolar constraints. This enhances the geometric robustness of the matching.

[0116] Constructing a probabilistic fusion framework: Using the aforementioned similarity as observational evidence, the posterior matching probability is calculated through Bayesian inference, as shown in the formula. .in, This represents the prior matching probability. Represents speed similarity under matching conditions. The likelihood probability, Represents directional similarity under matching conditions. The likelihood probability, This represents the polar consistency measure under matching conditions. The likelihood probability.

[0117] Adaptive matching solution and joint optimization:

[0118] Matching solution based on dynamic weight adjustment and spatiotemporal consistency: An adaptive weight strategy is designed to dynamically adjust the velocity similarity weights according to the scene's motion complexity. Similarity weights with orientation ( Increase when the field of view is large Increases when the speed of movement is higher. ;

[0119] Construct a joint optimization energy function: ,in To balance the parameters, the TemporalSmoothness term ensures temporal continuity. The weight parameters represent the time smoothness constraint term; the Hungarian matching algorithm is used to solve the optimized similarity matrix to establish the correspondence between the left and right camera targets, thus completing the stereo vision matching.

[0120] In summary, this embodiment also includes a matching system:

[0121] High-speed stereo camera module: Composed of two high-speed USB 3.0 cameras (such as Imaging Source Corp. DFK37BUX273), supporting 10-bit color image acquisition with a frame rate of no less than 200fps, transmitting pixel images to the computing platform via USB 3.0 interface for synchronous acquisition of scene HFR stereo video;

[0122] Multi-target tracking module: running on a computing platform equipped with AI, it achieves fast tracking of multiple targets with similar appearances based on the above-mentioned hybrid tracking algorithm, and outputs the real-time position and velocity information of the targets;

[0123] The stereo matching module includes a feature extraction unit, a similarity calculation unit, and a matching solution unit. The feature extraction unit extracts the STV feature set and the orientation feature set. The similarity calculation unit calculates the scale cosine similarity and orientation similarity and fuses them into a hybrid similarity matrix. The matching solution unit establishes the target correspondence relationship through the Hungarian matching algorithm.

[0124] Optional expansion modules: Data storage module (for storing HFR stereo video and tracking matching results), 3D reconstruction module (for calculating the target's three-dimensional spatial coordinates and motion trajectory based on the matching results).

[0125] The following describes this embodiment in conjunction with an application scenario:

[0126] Metronome Marker Stereo Matching Experiment:

[0127] The high-speed stereo camera module is equipped with a 12mm lens and a frame rate set to 200fps. =5ms), exposure time 2ms; two metronomes were fixed at a distance of 800mm from the camera, running at frequencies of 3.0Hz and 2.6Hz respectively, with OpenCV-generated markers pasted at different pointer positions (similar in appearance but different in amplitude); the computing platform was an Intel Core i9-9900K CPU, 64GB RAM, and an NVIDIA GeForce RTX 2080 Ti GPU.

[0128] The high-speed stereo camera is activated to simultaneously acquire HFR stereo video of the metronome's motion; the multi-target tracking module uses OpenCV to detect marker points at detection intervals. =10ms, template matching interval =5ms, inter-frame tracking of marker points is achieved using Euclidean distance matrix and Hungarian algorithm, and motion estimation is optimized by Kalman filtering; the feature extraction unit selects N=64 consecutive frames of images to extract the STV feature set and orientation feature set of each marker point; the similarity calculation unit is set ( ), ( The system calculates the mixed similarity matrix; the matching solution unit obtains the correspondence between the left and right camera marker points through the Hungarian algorithm and outputs the matching results.

[0129] The experimental results show that the mixed similarity of the same marker points in the left and right cameras is ≥0.8, which can accurately distinguish marker points with different motion amplitudes. Compared with the matching results of the 30fps camera (traditional frame rate), the 30fps camera caused some marker points (such as marker point 0 and marker point 3) to be mismatched due to motion information delay (about 30ms). However, the matching accuracy of the method in this embodiment reached 100%, which verified the effectiveness of the high synchronization short-term velocity feature.

[0130] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0131] like Figure 2 As shown, the present invention also provides a synchronous speed visual matching system, comprising:

[0132] The stereo video sequence acquisition module 201 is configured to synchronously acquire stereo video sequences at a predefined frame rate and dynamically adjust image acquisition parameters based on a target motion prediction model.

[0133] The motion trajectory optimization module 202 is configured to perform multi-target hybrid tracking and motion-induced detection on the stereo video sequence. It combines target detection and template matching that run at different time intervals to build a multi-hypothesis tracking framework to maintain multiple matching hypotheses. It completes inter-frame target association based on the first matching algorithm and applies a motion state estimation algorithm to optimize the motion trajectory, outputting target motion information containing position and velocity vectors.

[0134] The spatiotemporal coding definition module 203 is configured to extract hierarchical motion features of the target from multiple consecutive frames of a stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and perform unified spatiotemporal coding.

[0135] The posterior probability output module 204 is configured to calculate, under the geometric constraints of stereo vision, the scale cosine similarity based on the basic velocity vector feature set, the orientation similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets, and use them as observation evidence to input the probabilistic inference model for fusion, and output the posterior probability representing the reliability of the matching.

[0136] The matching relationship establishment module 205 is configured to adjust the weight allocation of velocity similarity and orientation similarity according to the motion characteristics of the target in the scene, construct a joint optimization energy function including weighted similarity term, geometric consistency term and time smoothness constraint term, and solve it based on the second matching algorithm to establish a stable correspondence matching relationship between the left and right view targets.

[0137] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of the claims of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0138] like Figure 3 As shown, the present invention also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of a synchronous speed visual matching method.

[0139] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 3 The structure shown in this embodiment of the invention includes an electronic device comprising one or more processors 710 and a memory 720; the processors 710 in this electronic device may be one or more. Figure 3 Taking a processor 710 as an example; a memory 720 is used to store one or more programs; the one or more programs are executed by the one or more processors 710, so that the one or more processors 710 implement a synchronous speed visual matching method as described in any one of the embodiments of the present invention.

[0140] The electronic device may also include an input device 730 and an output device 740.

[0141] The processor 710, memory 720, input device 730, and output device 740 in this electronic device can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0142] The memory 720 in this electronic device serves as a computer-readable storage medium, capable of storing one or more programs. These programs can be software programs, computer-executable programs, or modules, such as the program instructions / modules corresponding to the synchronous speed visual matching method provided in this embodiment of the invention. The processor 710 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 720, thereby implementing the synchronous speed visual matching method described in the above embodiment.

[0143] The memory 720 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 720 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 720 may further include memory remotely located relative to the processor 710, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] Input device 730 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 740 may include display devices such as a display screen.

[0145] The present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a synchronous speed visual matching method.

[0146] Specifically, the computer storage medium in this embodiment of the invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for synchronous speed visual matching, characterized in that, include: Synchronously acquire stereo video sequences at a predefined frame rate and dynamically adjust image acquisition parameters based on a target motion prediction model; Multi-target hybrid tracking and motion-induced detection are performed on stereo video sequences. By combining target detection and template matching that run at different time intervals, a multi-hypothesis tracking framework is constructed to maintain multiple matching hypotheses. Inter-frame target association is completed based on the first matching algorithm, and a motion state estimation algorithm is applied to optimize the motion trajectory. The output includes target motion information containing position and velocity vectors. Hierarchical motion features of the target are extracted from multiple consecutive frames of a stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and are uniformly coded in spatiotemporal. Under the geometric constraints of stereo vision, the scale cosine similarity based on the basic velocity vector feature set, the orientation similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets are calculated and used as observation evidence to input the probabilistic inference model for fusion, and the output is the posterior probability representing the reliability of the matching. The weights of velocity similarity and orientation similarity are adjusted according to the motion characteristics of the targets in the scene. A joint optimization energy function containing a weighted similarity term, a geometric consistency term, and a time smoothness constraint term is constructed and solved based on the second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views.

2. The synchronous speed visual matching method according to claim 1, characterized in that, Synchronously acquire stereoscopic video sequences at a predefined frame rate, and dynamically adjust image acquisition parameters based on a target motion prediction model, further including: Acquire predefined high frame rate stereoscopic video streams; Using motion prediction models based on Kalman filtering or recurrent neural networks, the position and velocity state of a target at future time points can be predicted; Based on the predicted target motion state, the acquisition parameters, including exposure time, frame rate and white balance, are adjusted in real time. The system continuously optimizes the acquisition parameter settings through a feedback control mechanism, and automatically adjusts the parameters when the target's motion state changes.

3. The synchronous speed visual matching method according to claim 1, characterized in that, Multi-target hybrid tracking and motion-induced detection are performed on stereo video sequences. A multi-hypothesis tracking framework is constructed by combining target detection and template matching run at different time intervals to maintain multiple matching hypotheses. Inter-frame target association is completed based on a first matching algorithm, and a motion state estimation algorithm is applied to optimize the motion trajectory. The output includes target motion information containing position and velocity vectors. Further steps include: Target detection based on a deep learning model is performed at the first detection interval to obtain the bounding rectangle coordinates, width and height dimensions, category label, and confidence information of the target. The target template library is updated based on the detection results, and a sliding window mechanism is used to maintain the target appearance templates of the most recent frames. Template matching based on the absolute difference algorithm is performed at a second tracking interval that is less than the first detection interval to locate the target position between consecutive frames; A multi-hypothesis tracking framework is constructed, which maintains multiple candidate matching hypotheses for each target and gradually eliminates erroneous matches through multi-frame evidence accumulation and trajectory consistency testing. Construct the Euclidean distance matrix of the target position between consecutive frames, apply the first matching algorithm to perform optimal association matching, and use the Kalman filter algorithm to smooth and predict the target motion state.

4. The synchronous speed visual matching method according to claim 1, characterized in that, Hierarchical motion features of the target are extracted from multiple consecutive frames of a stereoscopic video sequence. These hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained through frequency domain analysis. A unified spatiotemporal coding process is then performed, further including: Pixel-level motion information of each target is extracted from multiple consecutive frames of the stereoscopic video sequence, including horizontal displacement components and vertical displacement components, to form the basic velocity vector feature set; By calculating the ratio of the dot product to the magnitude of the velocity vectors between adjacent frames, the acceleration and steering angle features representing changes in motion direction are derived, forming the motion primitive feature set. A fast Fourier transform is performed on the target motion trajectory to extract the dominant frequency components and power spectrum features in the frequency domain, and the motion periodic feature set is constructed. The basic velocity vector feature set, motion primitive feature set, and motion periodic feature set are normalized in the spatiotemporal dimension, and a comprehensive motion feature descriptor is formed by concatenating the features.

5. The synchronous speed visual matching method according to claim 1, characterized in that, Under the geometric constraints of stereo vision, the scale cosine similarity between targets in the left and right views is calculated based on the feature set of the basic velocity vector, the orientation similarity is calculated based on the feature set of motion primitives, and the trajectory consistency measure is calculated based on epipolar geometry. Further, this includes: Calculate the scale-normalized cosine similarity between the basic velocity vector feature sets of corresponding targets in the left and right cameras, and use it as the velocity similarity; Calculate the directional consistency between the motion primitive feature sets of corresponding targets in the left and right cameras, and calculate the directional similarity based on the absolute difference of the feature values; Based on camera calibration parameters and epipolar geometric constraints, the reprojection error of the target motion trajectory in the left and right views is calculated and used as a trajectory consistency measure. Within the Bayesian inference framework, the velocity similarity, orientation similarity, and trajectory consistency measures are considered as input observational evidence for conditional probabilities. By combining the prior matching probability, the posterior matching probability distribution is calculated using the probability product rule.

6. The synchronous speed visual matching method according to claim 5, characterized in that, The observed evidence is input into the probabilistic inference model and fused to output a posterior probability representing the reliability of the match, which further includes: A Bayesian probabilistic graphical model is established, and the velocity similarity, orientation similarity, and trajectory consistency metrics are defined as observation nodes; Based on the conditional independence assumption, the likelihood probability function of each observed variable under the given matching condition is calculated respectively; Multiply the prior matching probability by each likelihood probability to obtain the unnormalized joint probability distribution; The probability is normalized by summing the marginal probabilities to obtain the standardized posterior matching probability. Output the final matching probability value and matching decision result for each target pair.

7. The synchronous speed visual matching method according to claim 1, characterized in that, The weights of velocity and orientation similarity are adjusted based on the motion characteristics of targets in the scene. A joint optimization energy function, including a weighted similarity term, a geometric consistency term, and a temporal smoothness constraint term, is constructed and solved based on the second matching algorithm to establish a stable correspondence matching relationship between targets in the left and right views. Further steps include: Based on the scene's field of view and the target's motion speed range, a linear interpolation strategy is used to dynamically allocate the weights of the velocity similarity and orientation similarity. Construct the joint optimization energy function, which includes data terms, geometric constraint terms, and time smoothing terms, wherein the data terms consist of weighted velocity similarity and orientation similarity. The geometric constraint term consists of a trajectory consistency measure based on epipolar geometry, and the temporal smoothing term consists of the consistency of matching results between adjacent frames. The second matching algorithm is used to solve for the global optimal matching of the similarity matrix derived from the energy function; Based on the final solution, a one-to-one correspondence is established between the targets in the left and right views, and a stable stereo matching result is output.

8. A synchronous speed visual matching system, characterized in that, include: The stereo video sequence acquisition module is configured to synchronously acquire stereo video sequences at a predefined frame rate and dynamically adjust image acquisition parameters based on a target motion prediction model. The motion trajectory optimization module is configured to perform multi-target hybrid tracking and motion-induced detection on the stereo video sequence. It combines target detection and template matching that run at different time intervals to build a multi-hypothesis tracking framework to maintain multiple matching hypotheses. Based on the first matching algorithm, it completes inter-frame target association and applies a motion state estimation algorithm to optimize the motion trajectory, outputting target motion information containing position and velocity vectors. The spatiotemporal coding definition module is configured to extract hierarchical motion features of the target from multiple consecutive frames of a stereoscopic video sequence. The hierarchical motion features include a basic velocity vector feature set composed of pixel-level velocity vectors, a motion primitive feature set obtained by calculating the relationship between adjacent velocity vectors, and a motion periodicity feature set obtained by frequency domain analysis, and perform unified spatiotemporal coding. The posterior probability output module is configured to calculate the scale cosine similarity based on the basic velocity vector feature set, the orientation similarity based on the motion primitive feature set, and the trajectory consistency measure based on epipolar geometry between the left and right view targets under the geometric constraints of stereo vision. These are then used as observation evidence inputs to the probabilistic inference model for fusion, and the output is a posterior probability representing the reliability of the matching. The matching relationship establishment module is configured to adjust the weight allocation of velocity similarity and orientation similarity according to the motion characteristics of the targets in the scene, construct a joint optimization energy function including weighted similarity term, geometric consistency term and time smoothness constraint term, and solve it based on the second matching algorithm to establish a stable correspondence matching relationship between the targets in the left and right views.

9. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method according to any one of claims 1 to 7.

Images