A laser spot robust recognition and tracking method for complex light environment

Abstract

This invention discloses a robust method, device, and storage medium for laser spot recognition and tracking in complex lighting environments, belonging to the field of computer vision and target detection. Addressing the problems of high false alarm rates and poor robustness in existing methods under interference such as strong ambient light, dynamic projector flicker, and specular reflection, this invention proposes a multi-layered processing framework of "detection-verification-tracking." First, adaptive threshold segmentation is performed on the current frame image to extract candidate regions with brightness higher than the background. Second, multi-dimensional feature verification is introduced. Based on the inherent physical properties of the laser spot (such as spectral matching degree, shape and size, and two-dimensional Gaussian intensity distribution) and temporal motion patterns, candidate regions are jointly verified and filtered to effectively remove various homogeneous and heterogeneous interferences. Finally, through a multi-target trajectory management module, the filtered candidate spots are correlated with historical trajectories (e.g., using the Hungarian algorithm), and combined with Kalman filter prediction and state machine-based trajectory lifecycle management (temporary, confirmed, lost, terminated), stable tracking and output of the laser spot are achieved. This invention does not rely on a large amount of training data, has high computational efficiency, and can significantly improve the accuracy and robustness of laser spot recognition under complex dynamic lighting conditions, providing a reliable front-end perception guarantee for vision-based high-precision laser spatial positioning and interaction systems.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, target detection and tracking technology, and in particular to a method and system for highly robust identification and tracking of light spots generated by devices such as laser pointers and laser pens in complex lighting environments. It can be applied to multiple fields such as human-computer interaction, spatial positioning, virtual reality, augmented reality and industrial inspection. Background Technology

[0002] In vision-based laser spatial interaction systems, such as using a laser pointer to point, draw, or manipulate on a large display screen or in three-dimensional space, the primary and core step is to accurately and stably identify the position of the laser spot from the video stream captured by the camera and continuously and reliably track its motion trajectory. A laser spot spatial absolute positioning method and system based on multi-camera collaboration has solved the problem of high-precision spatial positioning, but the effectiveness of its positioning algorithm depends entirely on whether the front-end vision system can correctly extract the pixel coordinates of the laser spot from each frame of the image.

[0003] However, in practical applications, the imaging environment is often very complex, posing a serious challenge to the reliable identification of laser spots: Strong ambient light interference: Strong sunlight projected from indoor windows and indoor lighting (especially LED lights and fluorescent lights) may create bright specular or diffuse reflection areas on smooth surfaces (such as whiteboards, desktops, and screen bezels), which appear as bright spots similar to light spots in the image.

[0004] Dynamic light source interference: When the projector is working, the image content it projects may contain bright spots, flickering animations or high-contrast pixel blocks. These dynamic contents are very easy to be misjudged as laser spots.

[0005] Other light source pollution: Other point light sources in the environment, such as indicator lights, mobile phone screen flashes, and highly reflective decorations, can also cause interference.

[0006] Changes in laser spot imaging characteristics: When a laser spot illuminates a surface of different materials at different distances and angles, its image size, brightness, and shape (which may be diffused due to defocusing or surface roughness) will change dynamically.

[0007] Existing spot recognition methods, such as simple fixed thresholding or adaptive thresholding (e.g., Otsu's algorithm), produce a large number of false detections in complex backgrounds and cannot distinguish between real laser spots and environmental noise. While some template matching or feature matching methods offer some robustness, they are sensitive to changes in spot scale and rotation, and are prone to losing targets when they suddenly appear, disappear, or are severely occluded. Deep learning-based target detection methods (e.g., YOLO, SSD), although high-performance, require large amounts of labeled data for training, their generalization ability depends on the comprehensiveness of the training data, and their deployment cost is high in embedded systems with limited computing resources or in scenarios with extremely high real-time requirements.

[0008] Therefore, there is an urgent need for a method that does not rely on a large amount of training data, has high computational efficiency, and can make full use of the inherent physical characteristics and motion laws of laser spots to achieve highly robust and accurate recognition and tracking in complex and ever-changing light environments, so as to ensure the stable and reliable operation of the entire laser space interaction system. Summary of the Invention

[0009] This invention aims to overcome the shortcomings of existing technologies and provide a robust laser spot recognition and tracking method for complex lighting environments. The core idea of ​​this method is to construct a complete "detection-verification-tracking" process. It not only utilizes simple brightness information but, more importantly, comprehensively leverages the inherent physical properties of the laser spot in dimensions such as spectrum, morphology, and intensity distribution, combined with its temporal motion continuity, to perform multi-level, multi-dimensional joint verification and filtering of candidate regions. This achieves an extremely high correct detection rate and an extremely low false alarm rate even in environments with strong interference.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a robust method for identifying and tracking laser spots, applied to an optical positioning system including an image acquisition device, the method comprising the following iteratively executed steps: Candidate region detection steps: Process the current frame image and extract potential light spot regions with brightness higher than the background as candidate regions; Feature verification and filtering steps: Based on the inherent physical properties and / or motion laws of the laser spot, the candidate regions are verified, and interference regions that do not conform to the characteristics of the laser spot are filtered out to obtain the verified candidate spot set; Multi-target trajectory management steps: Associate the verified candidate light spots in the current frame with the established historical light spot trajectories, update the status of the matching trajectories, and create new trajectories for unmatched candidate light spots; at the same time, manage the life cycle of the trajectory and output stable light spot trajectories and their position information.

[0011] In one embodiment, the inherent physical property includes at least one of the following: Spectral properties: The degree of matching between the spectral composition of the candidate region and the preset laser wavelength; Morphological attributes: The degree of fit between the shape and size of the candidate region and the preset laser spot imaging model; Intensity distribution attribute: The similarity between the intensity distribution of pixels within the candidate region and the point spread function model.

[0012] In one specific embodiment, the verification of the intensity distribution attribute includes: calculating the goodness of fit between the pixel intensity distribution of the candidate region and a two-dimensional Gaussian distribution model, and filtering based on a goodness-of-fit threshold. Due to its good coherence, laser spots typically exhibit an approximate Gaussian distribution on camera sensors, while most environmental noise (such as blocky highlights and linear reflections) does not conform to this distribution.

[0013] In one embodiment, the verification of the motion pattern includes: using a motion estimator (such as a Kalman filter or particle filter) to predict the occurrence position and uncertainty range of an existing trajectory in the current frame, and retaining only candidate regions falling within the uncertainty range in the verified candidate spot set. This utilizes the physical principle that the motion of laser spots is usually smooth and continuous, which can effectively filter out randomly occurring flicker noise.

[0014] In one embodiment, the data association employs a globally optimal association algorithm (such as the Hungarian algorithm) to match all trajectories with the goal of minimizing the overall association cost between all trajectories and all candidate spots (such as based on Mahalanobis distance or Euclidean distance) in order to handle situations such as multiple spot intersections and proximity.

[0015] In one embodiment, the lifecycle management of the trajectory includes: Set a temporary state for the newly created trajectory to filter out transient noise; If a trajectory in a temporary state is successfully associated in N consecutive frames (N≥2, preferably 3-5 frames), its state is promoted to a confirmed state, and its position is output as a real laser spot. If a track in the confirmed state fails to associate for M consecutive frames (M≥1, usually 2-3 frames), its state is set to the lost state, and the track is terminated after a certain number of frames are lost, in order to handle the situation where the light spot disappears after being briefly blocked.

[0016] Secondly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the laser spot robustness identification and tracking method as described above.

[0017] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the laser spot robustness identification and tracking method as described above. Beneficial effects

[0018] Compared with the prior art, the present invention has the following significant advantages: Extremely high environmental robustness: By integrating multi-dimensional features such as spectrum, morphology, intensity distribution model and motion continuity for joint verification, it can effectively resist various complex interferences such as strong ambient light, projector flicker, and specular reflection, and can still work stably in scenarios where existing simple threshold methods fail.

[0019] High accuracy and low false alarm rate: Physical property-based verification (such as Gaussian fitting) can fundamentally distinguish between laser point sources and interference from surface sources, line sources, etc. Combined with spatiotemporal continuity filtering, it significantly reduces the probability of misidentifying noise as light spots.

[0020] Good real-time performance and low computational overhead: Compared with deep learning-based solutions, the method of this invention is based on explicit physical rules and mathematical models, with controllable computational complexity. It does not require large-scale data training and expensive GPU computing power, making it more suitable for deployment in embedded devices or systems with high real-time requirements.

[0021] It has a strong ability to handle dynamic changes in targets: it adopts a "detection-verification-tracking" framework, and each frame is detected independently without relying on the initial template. It can effectively handle complex situations such as the sudden appearance and disappearance of light spots, as well as the overlapping and occlusion of multiple targets.

[0022] Seamless integration with upstream positioning systems: The method of this invention provides extremely clean and reliable laser spot pixel coordinate input for high-precision spatial positioning systems such as "a laser spot spatial absolute positioning method and system based on multi-camera collaboration". It is the "front-end sentinel" that ensures the stable and accurate operation of the entire interactive system and solves the key reliability problem of moving from the laboratory environment to complex real-world application scenarios. Attached Figure Description

[0023] Figure 1 The overall flowchart of the laser spot robustness identification and tracking method provided in the embodiments of the present invention is shown.

[0024] Figure 2 This is a schematic diagram of the multi-target trajectory management and data association process. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Example

[0026] This embodiment uses a binocular vision positioning system deployed in a smart conference room as an example. The system aims to track the light spot formed on the projection screen by a laser pointer held by the speaker. The conference room environment is complex, with strong projector light, natural light from windows, and mirror reflections from the metal frame.

[0027] like Figure 1 As shown, the method in this embodiment is implemented in software on the system's main processor (such as an embedded industrial control computer) to process synchronized video streams from two global cameras. The method flow is as follows: Image preprocessing and candidate region detection: The system acquires the grayscale image of the current frame. First, the image is Gaussian filtered to suppress sensor noise.

[0028] An adaptive thresholding algorithm (e.g., thresholding based on local image mean) is used to binarize the filtered image. Due to ambient light inconsistencies (such as dark areas in a projected image), adaptive thresholding performs better than fixed thresholding. This step yields initial candidate regions (white connected regions) containing the actual laser spot and various interferences.

[0029] Multi-dimensional feature validation and filtering: For each candidate connected component in the graph, perform the following verifications sequentially (this can be done in parallel or according to a set priority): Spectral verification: A narrowband filter matching the laser pointer wavelength (e.g., 650nm red light) was installed in front of the camera lens to pre-filter out most non-red light interference. At the image processing level, the component proportions of candidate regions in the RGB color space were further analyzed, and candidate regions with a red component proportion exceeding 90% were given higher confidence.

[0030] Morphological verification: Calculate the area (number of pixels) and roundness (4π * area / perimeter^2) of each candidate region. Set the effective region area range to [4, 100] pixels, and the roundness threshold > 0.7. Remove regions that are too large / too small, or whose shape is too elongated (such as reflective stripes).

[0031] Intensity Distribution Verification: For candidate regions that pass morphological verification, their pixel coordinates and intensity values ​​are extracted. A nonlinear least squares method is used to fit the intensity distribution within the region to a two-dimensional Gaussian surface. The coefficient of determination R² is calculated as the goodness of fit. A threshold of R² > 0.85 is set. The intensity distribution of the actual laser spot (region A) closely matches the Gaussian model (R² = 0.95), while the distribution of a projected bright spot (region B) is chaotic (R² = 0.3), therefore region B is filtered out.

[0032] Spatiotemporal continuity verification (prediction verification): For established historical trajectories, a Kalman filter is used to predict their expected position (x_pred, y_pred) and covariance matrix P in the current frame. The Mahalanobis distance from the center (x_c, y_c) of each candidate region to the predicted position is calculated. If the Mahalanobis distance is less than a set threshold (e.g., corresponding to a 95% confidence interval), the candidate region is considered a continuation of the trajectory. This step effectively filters out randomly occurring transient noise.

[0033] Only candidate regions that pass both morphological and intensity distribution verification, and at least one of the verifications—spectral or spatiotemporal continuity—are retained as "verified candidate spots" and proceed to the next stage. After rigorous verification, only the actual laser spots remain.

[0034] Multi-target trajectory management: Data association: such as Figure 2 As shown, the candidate light spots verified in the current frame (solid circles in the figure) are associated with existing trajectories (dashed lines represent their predicted positions and uncertainty ellipses). This embodiment uses the Hungarian algorithm, with the Mahalanobis distance between the candidate light spot and the predicted position of the trajectory as the association cost, to perform a globally optimal match. Successfully matched candidate light spots are used to update the state (including position and velocity) and Kalman filter parameters of the corresponding trajectories.

[0035] Trajectory lifecycle management: Initialization: For candidate light spots that do not match any historical trajectory, a new trajectory will be created and its status will be set to "temporary".

[0036] Confirmation: If a "temporary" trajectory is successfully associated for 3 consecutive frames, its status changes to "Confirmation". Only trajectories in the "Confirmation" state will have their center position output, which will be provided to "A method and system for absolute spatial positioning of laser spot based on multi-camera collaboration" for 3D coordinate calculation.

[0037] Prediction and Loss: For a trajectory in the "confirmed" state, if a match fails in the current frame, the Kalman filter will still predict it, and the trajectory state will remain unchanged. If a match fails for two consecutive frames, the state changes to "lost".

[0038] Termination: If a trajectory in the "lost" state cannot be associated with a candidate spot in the next frame, the trajectory will be terminated and its ID resource will be released.

[0039] Through iterative execution of the above steps, the system can stably and accurately track the laser spot in complex environments. In a simple thresholding scheme without this method, tracking would frequently be interrupted due to the periodic flickering of the projector (resulting in significant fluctuations in the success rate curve); however, after applying this invention, the tracking success rate stabilizes at over 98%. Example

[0040] In another embodiment, for tracking scenarios involving invisible light (such as infrared lasers), spectral verification of color channels can be omitted, relying entirely on narrowband filters for optical filtering, and morphological verification and intensity distribution verification can be further strengthened. Simultaneously, a more complex model based on trajectory motion smoothness can be introduced to further distinguish the true laser point from multiple similar infrared noise points.

[0041] This invention provides a front-end visual processing method with strong anti-interference ability, high reliability and good real-time performance for laser space interaction systems through the above-mentioned multi-dimensional feature fusion verification and rigorous trajectory management mechanism. It is a key guarantee for realizing the technology from ideal environment to complex real-world application.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A robust method for identifying and tracking laser spots, characterized in that, Applied to an optical positioning system that includes an image acquisition device, the method comprises the following steps performed iteratively: Candidate region detection: Process the current frame image and extract potential spot regions with brightness higher than the background as candidate regions; Feature verification and filtering: Based on the inherent physical properties and / or motion laws of the laser spot, the candidate regions are verified, and interference regions that do not conform to the characteristics of the laser spot are filtered out to obtain a set of verified candidate spots; Multi-target trajectory management: The verified candidate light spots in the current frame are associated with the established historical light spot trajectories, the status of the matching trajectories is updated, and new trajectories are created for unmatched candidate light spots; at the same time, the life cycle of the trajectory is managed, and stable light spot trajectories and their position information are output.

2. The method as described in claim 1, characterized in that, The inherent physical properties include at least one of the following: Spectral properties: The degree of matching between the spectral composition of the candidate region and the preset laser wavelength; Morphological attributes: The degree of fit between the shape and size of the candidate region and the preset laser spot imaging model; Intensity distribution attribute: The similarity between the intensity distribution of pixels within the candidate region and the point spread function model.

3. The method as described in claim 2, characterized in that, The verification of the intensity distribution attribute includes: calculating the goodness of fit between the pixel intensity distribution of the candidate region and the two-dimensional Gaussian distribution model, and filtering according to the goodness of fit threshold.

4. The method as described in claim 1, characterized in that, The verification of the motion pattern includes: using a motion estimator to predict the occurrence position and uncertainty range of an existing trajectory in the current frame, and retaining only the candidate regions that fall within the uncertainty range in the verified candidate spot set.

5. The method as described in claim 4, characterized in that, The motion estimator is a Kalman filter or a particle filter.

6. The method as described in claim 1, characterized in that, The data association employs a global optimal association algorithm, aiming to minimize the overall association cost between all trajectories and all candidate light spots.

7. The method as described in claim 1, characterized in that, The lifecycle management of the trajectory includes: Set a temporary state for the newly created trajectory; If a trajectory in a temporary state is successfully associated in N consecutive frames (N≥2), its state is promoted to a confirmed state, and its position is output as a real laser spot. If a trajectory in the confirmed state fails to associate for M consecutive frames (M≥1), its state is set to the lost state, and the trajectory is terminated after a certain number of frames are lost.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

Images