Goods yard vehicle monitoring method and system based on monocular laser point cloud, and storage medium

Abstract

The invention discloses a monocular laser point cloud-based goods yard vehicle monitoring method and system, and aims to solve the problems of low vehicle monitoring precision and insufficient data integrity in a complex goods yard environment. The method comprises the following three steps: S1, building a multi-condition constraint portable experimental field, planning a sensor layout through a multi-target optimization model, carrying out view angle consistency calibration, carrying out weighted fusion on point clouds, dynamically adjusting weights, and generating a real-time updated panoramic distance image; s2, Gaussian filtering denoising and point cloud quality improvement are carried out, a neighborhood is determined in combination with spatial features, dynamic kernel width filtering denoising is carried out, abnormal points are eliminated through a dynamic threshold value, and weights are updated according to a denoised signal-to-noise ratio in a multi-sensor scene; and S3, complementing the invisible area of the point cloud through 3D Gaussian splash processing, generating a preliminary panoramic video frame through multi-scale fusion and projection, and optimizing to obtain a panoramic video of which the time sequence is consistent with the visual angle. The system adapts to the dynamic environment of the goods yard and provides high-precision data support for vehicle detection and trajectory analysis.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent monitoring, and more specifically, relates to a method, system and storage medium for monitoring freight yard vehicles based on monocular laser point clouds. Background Technology

[0002] Current freight yard vehicle monitoring systems and methods face the following technical problems and challenges: Laser point cloud data is often affected by various factors, resulting in sparse data and high noise levels. These problems may stem from measurement errors in the laser sensor, interference from the external environment (such as weather and lighting) on ​​the laser signal, or reflections from other objects in the field. Noise directly impacts the accuracy of the data, thereby affecting the quality of subsequent point cloud processing, target detection, and localization tasks. Therefore, how to remove interference noise while preserving key features through efficient denoising algorithms is a core technical challenge in system development.

[0003] In freight yard environments, target detection and localization using monocular laser point clouds present significant challenges due to differences in vehicle type, appearance, and movement. The sparsity and occlusion issues of point cloud data further exacerbate the difficulty in achieving accurate detection. Particularly when multiple vehicles appear simultaneously in the same scene, accurately extracting complete information about the target object from a limited field of view for high-precision vehicle identification and localization, while avoiding false positives and false negatives caused by occlusion, angle changes, and other factors, becomes a major technical challenge for this system.

[0004] Vehicles in a freight yard are typically dynamic and may obstruct or approach each other, increasing the complexity of multi-target tracking. Effectively tracking multiple vehicles, especially in situations involving intersections, overlaps, or occlusions, and avoiding target loss or misassociation, presents a challenge. Furthermore, the dynamic states of vehicles change over time; real-time state estimation while ensuring tracking accuracy is a significant technical challenge in the system design.

[0005] Since freight yard monitoring systems typically operate in real-time, improving the processing efficiency of point cloud data while maintaining system accuracy presents another significant technical challenge. Point cloud data is usually large in volume and requires processing using complex algorithms. Reducing computation time, avoiding system sluggishness due to computational delays, and ensuring the efficient completion of monitoring tasks are key issues in achieving system real-time performance.

[0006] Based on the above challenges, there is an urgent need for a freight yard vehicle monitoring method and system that can provide high quality, high precision, and high efficiency based on monocular laser point clouds. Summary of the Invention

[0007] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method, system, and storage medium for monitoring freight yard vehicles based on monocular laser point clouds. Through multi-sensor optimized layout and calibration fusion, dynamic Gaussian filtering for noise reduction, and 3D Gaussian splashing to complete the point cloud and generate panoramic video, this invention solves the problems of low vehicle monitoring accuracy and insufficient data integrity in complex freight yard environments. It achieves the technical effects of no blind spots in the freight yard monitoring area, high-precision and stable point cloud quality, consistent temporal perspective of panoramic video, and reliable visual data support for vehicle monitoring.

[0008] To achieve the above objectives, according to a first aspect of the present invention, a method for monitoring freight yard vehicles based on monocular laser point clouds is provided, comprising the following steps: S1. Construction of a portable test field under multiple constraints: Based on the monitoring needs of the cargo yard, the sensor layout is planned using a multi-objective optimization model; a mapping between the sensor and the world coordinates is established, and after optimization and calibration, the weighted fusion point cloud is combined with the real-time signal-to-noise ratio for dynamic weight adjustment; the fused point cloud is converted into a two-dimensional image, and after registration, distance filtering and filtering optimization, a panoramic distance image is generated, which is updated by real-time data stream to ensure accuracy; S2. Gaussian filtering for denoising and point cloud data quality improvement: First, the original point cloud of monocular laser in the freight yard is acquired, and the neighborhood of the point cloud is determined by combining spatial features; Gaussian filtering with dynamic kernel width is used for denoising; then, outliers are removed by dynamic thresholding; in multi-sensor scenarios, the weights are updated according to the signal-to-noise ratio after denoising, and a high-precision point cloud is output. S3.3D Gaussian splash processing and panoramic video generation: First, a limited-view video sequence of the freight yard is acquired, and each frame image is calibrated and mapped to a local point cloud; then, the invisible area point cloud is completed by using the local tensor field and historical frame state, and after multi-scale fusion, it is projected to generate a preliminary panoramic video frame; finally, through pixel-level depth confidence fusion optimization, a freight yard panoramic video with consistent time sequence and viewpoint is generated to support vehicle monitoring.

[0009] Furthermore, step S1, which involves planning the sensor layout using a multi-objective optimization model based on the monitoring needs of the freight yard, includes the following steps: Taking into account the complexity of the freight yard environment and the trajectory of the target object, and using coverage, point cloud sampling density, system computing resource utilization and sensor installation safety as constraints, a multi-objective optimization model is constructed to determine the optimal installation location of the sensor. Let the sensor set be The sensor i sensor is installed at the following location: The coverage radius is Construct a multi-objective optimization model to determine the optimal sensor installation location function; ; in, Indicates the effective coverage area. Indicates the overlapping area. Indicates installation and maintenance costs. is the weighting coefficient, and max indicates taking the maximum value.

[0010] Furthermore, the establishment of the sensor-world coordinate mapping in step S1, after optimization and calibration, and the weighted fusion of point clouds combined with real-time signal-to-noise ratio dynamic weight adjustment, includes the following steps: Viewpoint Consistency Modeling and Point Cloud Fusion: A mapping relationship between multiple sensors and the world coordinate system is established to achieve data unification. Weighted least squares method is used to fuse point clouds from multiple sensors. The mapping relationship is as follows: ; in The point cloud collected by sensor i For world coordinate point cloud, The rotation matrix corresponding to sensor i, Let i be the displacement vector corresponding to sensor i; The rotation matrix corresponding to sensor i is adjusted by least squares optimization. Displacement vector corresponding to sensor i To improve calibration accuracy, the relationship is optimized as follows: ; in Let be the mean point of the k-th class of points collected by sensor i. This is the standard point of the k-th type of point in the world coordinate system; The fusion relationship of the weighted least squares method is as follows: ; in For the merged point cloud, Let be the fusion weights for sensor i; Dynamic weighting is achieved by incorporating real-time error feedback. Updated to further improve the integration effect.

[0011] Furthermore, the dynamic weighting is achieved by incorporating real-time error feedback. The update steps specifically include: Dynamic weight update: When fusing multi-sensor point clouds using the weighted least squares method, the fusion weights of each sensor are dynamically updated using the following formula, combined with real-time error feedback: ; Let i be the fusion weights for the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. For update rate, To measure the signal-to-noise ratio in real time, The sum of the signal-to-noise ratios measured by the sensors in real time; For each point cloud acquired by the sensor, a perspective projection function is used. After mapping, we obtain two-dimensional pixel coordinates, and the projection relationship is as follows: ; in This is a perspective projection function that maps a 3D point cloud to 2D pixels; Let be the two-dimensional pixel coordinates corresponding to the point cloud of the i-th sensor. Let be the rotation matrix of the i-th sensor. For world coordinate point cloud, Let i be the displacement vector of the i-th sensor. The projection results from different sensors are aligned using an image registration algorithm, and then weighted and fused according to the minimum distance principle to generate a panoramic distance image. The fusion relationship is as follows: ; in Let be the depth value of the pixel corresponding to sensor i. Select the nearest boundary that covers the target object with the minimum distance; Depth Optimization and Dynamic Update: Gaussian kernel filtering is used to perform depth interpolation and smoothing on the panoramic distance image. The optimization relationship is as follows: ; in The optimized panoramic distance image depth value, where K is the Gaussian kernel. For pixels Neighborhood, for The system uses neighboring pixels and combines real-time sensor data streams to dynamically update the panoramic image, ensuring the integrity and accuracy of the point cloud and panoramic image when the freight yard environment changes, vehicles move, or there is temporary obstruction.

[0012] Further, step S2 acquires the original point cloud of the monocular laser in the cargo yard, determines the point cloud neighborhood by combining spatial features, and uses a Gaussian filter with dynamic kernel width for noise reduction; including the following steps: The original point cloud set acquired by monocular laser scanning of the cargo yard is represented as follows: Each point Its actual location may be affected by noise interference. For each point The set of neighboring points is determined by combining spatial distance, local curvature, density changes, and historical frame information. The Gaussian filter formula for neighborhood weighted smoothing is applied to the points. For noise reduction, the filtering formula is: ; in, For point The set of neighborhood points, The width of the Gaussian kernel is used to control the smoothing intensity. For point Points after filtering and denoising for Points within the neighborhood.

[0013] Furthermore, the step of removing outliers through dynamic thresholds; in multi-sensor scenarios, updating weights based on the denoised signal-to-noise ratio includes: Abnormal noise point removal: The quality of each frame of point cloud is evaluated before and after denoising, and the noise level of each point is calculated. Local noise measurement The measurement formula is: ; when When this point is identified as an abnormal noise point, it needs to be further removed or replaced; threshold Dynamically adjust based on sensor accuracy and environmental disturbances: ; in, As the initial threshold, This is the adjustment coefficient; Multi-sensor dynamic weight update: In multi-sensor acquisition scenarios, the signal-to-noise ratio of each sensor after denoising is used as the basis for weight update. As the basis for weight adjustment, the sensor fusion weights are updated through a dynamic weight update mechanism, and the update formula is as follows: ; in, The fusion weights of the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. To ensure a high update rate, the fused point cloud maintains high accuracy even when the sensor is interfered with or experiences error fluctuations.

[0014] Furthermore, the method for 3D Gaussian splash processing and panoramic video generation in step S3 includes the following steps: Local point cloud acquisition: Acquire a limited-view video sequence of the freight yard, denoted as: ;in Let be the nth frame image in the video sequence; map each frame image to a local point cloud set at the corresponding time point through sensor calibration, denoted as ; ; in, Let be the local point cloud set at time t. For the first in this set point Limited by visibility and noise disturbance; Point cloud completion for invisible regions: defining local tensor fields The local tensor field Given a local point cloud set, the tensor can describe the local directionality, density, and curvature constraints of each point; based on the local tensor field... Recursive encoding state of historical frames Through prediction function Generate missing points in invisible regions of a local point cloud, denoted as . The prediction relationship is: ; Multi-scale point cloud fusion and panoramic projection: A multi-scale fusion strategy is adopted to perform nonlinear weighted accumulation of predicted point clouds at different scales; combined with camera extrinsic parameters. in Let be the camera rotation matrix at time t. Let be the camera displacement vector at time t, and then project it using the perspective projection function. The fused predicted point cloud is mapped onto a two-dimensional image plane to generate preliminary panoramic video frames. The mapping relationship is as follows: ; in, Let t be the initial panoramic video frame at time t. The fused predicted point cloud is at time t. Panoramic video optimization and panoramic viewpoint cloud generation: Pixel-level deep confidence fusion technology is introduced to perform multi-frame accumulation and local filtering optimization on the initial panoramic video frames, generating a high-precision panoramic viewpoint cloud with consistent temporal sequence and viewpoint. The panoramic viewpoint cloud possesses spatial continuity, geometric consistency, and structural integrity, providing reliable data support for subsequent tasks such as dynamic target detection, trajectory analysis, and monitoring of freight yard vehicles.

[0015] As another aspect of the present invention, the present invention provides a freight yard vehicle monitoring system based on monocular laser point clouds, comprising: Multi-sensor calibration and fusion and panoramic image generation unit; establishes sensor-world coordinate mapping, and after optimization and calibration, weighted fusion of point clouds and dynamically adjusts weights in combination with real-time signal-to-noise ratio; converts the fused point cloud into a two-dimensional image, and generates a panoramic distance image through registration, distance filtering and filtering optimization, and relies on real-time data stream updates to ensure accuracy; The Gaussian filtering denoising and point cloud data quality improvement unit first acquires the original point cloud of the monocular laser in the freight yard, and determines the neighborhood of the point cloud by combining spatial features; then it uses dynamic kernel width Gaussian filtering for denoising; subsequently, it removes outliers by dynamic thresholding; in multi-sensor scenarios, it updates the weights according to the signal-to-noise ratio after denoising and outputs a high-precision point cloud. The 3D Gaussian splash processing and panoramic video generation unit first acquires a video sequence with limited viewpoints in the freight yard, and then maps each frame image to a local point cloud. Next, it uses the local tensor field and historical frame states to complete the invisible area point cloud, and after multi-scale fusion, it projects to generate preliminary panoramic video frames. Finally, through pixel-level depth confidence fusion optimization, it generates a freight yard panoramic video with consistent time sequence and viewpoint to support vehicle monitoring.

[0016] As another aspect of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of any of the preceding claims of a method for monitoring freight yard vehicles based on monocular laser point clouds.

[0017] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. The present invention provides a freight yard vehicle monitoring method based on monocular laser point cloud. Through the construction of a portable experimental field with multiple constraints, the method constructs a multi-objective optimization model for point selection with constraints such as coverage and sampling density, establishes a sensor-world coordinate mapping and dynamically updates weights, and generates a real-time updated panoramic distance image. This achieves the technical effects of no blind spots in the freight yard vehicle monitoring area, high accuracy of multi-sensor data fusion, and dynamic adaptation of panoramic images to environmental changes.

[0018] 2. The present invention provides a method for monitoring freight vehicles based on monocular laser point clouds. Through Gaussian filtering for noise reduction and point cloud data quality improvement techniques, the method employs dynamic kernel width Gaussian filtering combined with spatial features and historical frame information to determine the neighborhood, removes abnormal noise points based on dynamic thresholds, and updates weights with the signal-to-noise ratio after denoising in multi-sensor scenarios. This achieves the technical effects of effectively suppressing point cloud noise, completely preserving local features, and ensuring stable accuracy of multi-sensor fused point clouds.

[0019] 3. The present invention provides a freight yard vehicle monitoring method based on monocular laser point clouds, which utilizes 3D Gaussian splashing processing and panoramic video generation technology: by using local tensor fields combined with historical frame states to complete the invisible areas of the point cloud, and generating panoramic videos through multi-scale fusion and pixel-level depth confidence optimization, the method achieves the technical effect of complete point cloud spatial information, consistent panoramic video temporal sequence and perspective, and high-precision visualization data support for vehicle monitoring. Attached Figure Description

[0020] Figure 1 This is a flowchart of a freight yard vehicle monitoring method based on monocular laser point clouds, as an embodiment of the present invention. Figure 2 This is a structural diagram of a freight yard vehicle monitoring system based on monocular laser point clouds, according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] Example 1, Please refer to Figure 1 Please refer to Figure 1 This embodiment relates to a method for monitoring freight yard vehicles based on monocular laser point clouds, including... S1. Construction of a portable experimental field under multiple constraints: Based on the monitoring needs of the cargo yard, the sensor layout is planned using a multi-objective optimization model; a mapping between the sensor and the world coordinates is established, and after optimization and calibration, the weighted fusion point cloud is combined with the real-time signal-to-noise ratio for dynamic weight adjustment; the fused point cloud is converted into a two-dimensional image, and after registration, distance filtering and filtering optimization, a panoramic distance image is generated, which is updated by real-time data stream to ensure accuracy.

[0023] To facilitate the rapid construction of experimental fields in dynamic cargo yard environments and meet the requirements of different sensor perspectives and point cloud data fusion, this embodiment proposes a portable temporary experimental field construction scheme with multiple constraints, comprising three main parts: point selection strategy, perspective consistency modeling and fusion, and panoramic distance image generation. This scheme not only focuses on the spatial layout of sensors but also combines data fusion models and panoramic mapping technology to achieve high-precision and high-efficiency point cloud generation.

[0024] Regarding sensor selection strategies, considering the complexity of the freight yard environment and the trajectory of target objects, sensor selection must simultaneously meet multiple constraints, including coverage, sampling density, computational resources, and security. Let the sensor set be... Each sensor is installed at the following location: The coverage radius is The selection of points can be represented as a multi-objective optimization problem: ; in, Indicates the effective coverage area. Indicates the overlapping area. Indicates installation and maintenance costs. Here, represents the weighting coefficient, and `max` indicates taking the maximum value. In practice, heuristic search, genetic algorithms, or mixed-integer programming can be used to solve for the optimal installation location, while dynamically adjusting based on obstacle distribution and real-time vehicle trajectory.

[0025] In terms of viewpoint consistency modeling and fusion, an angle consistency equation is established to ensure the uniformity of data from different sensors. Let the coordinates of sensor i be... The attitude angle is Point cloud collection is World coordinates are The mapping relationship is as follows: ; in The point cloud collected by sensor i For world coordinate point cloud, The rotation matrix corresponding to sensor i, Let i be the displacement vector corresponding to sensor i; ; in Let be the mean point of the k-th class of points collected by sensor i. This is the standard point of the k-th type of point in the world coordinate system; The fusion relationship of the weighted least squares method is as follows: ; in For the merged point cloud, Let be the fusion weights for sensor i; Meanwhile, dynamic weight updates can be performed by combining real-time error feedback, further improving the fusion effect.

[0026] When fusing multi-sensor point clouds using the weighted least squares method, the fusion weights of each sensor are dynamically updated using the following formula, taking into account real-time error feedback: ; in Let i be the fusion weights for the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. For update rate, To measure the signal-to-noise ratio in real time, The sum of the signal-to-noise ratios measured by the sensors in real time ensures that the fusion results remain accurate even when the experimental field changes dynamically or when sensor errors fluctuate.

[0027] In terms of panoramic distance image generation, the fused point cloud is used to generate high-precision panoramic images. The point cloud from each sensor is then subjected to perspective projection. ; in This is a perspective projection function, a current technique, that maps a 3D point cloud to 2D pixels. Let be the two-dimensional pixel coordinates corresponding to the point cloud of the i-th sensor. Let be the rotation matrix of the i-th sensor. For world coordinate point cloud, Let be the displacement vector of the i-th sensor; Next, an image registration algorithm is used to align the projection results from different sensors, and then weighted fusion is performed to generate a panoramic distance image. ; in Let be the depth value of the pixel corresponding to sensor i. Select the nearest boundary that covers the target object with the minimum distance.

[0028] To further improve panoramic continuity and accuracy, depth interpolation and filtering can be added: ; Where K is the Gaussian kernel. For pixel neighborhood, For pixels Neighborhood, for The system utilizes neighboring pixels to achieve smooth transitions and compensate for missing points. Combined with real-time sensor data streams, the panoramic imagery can be dynamically updated, ensuring the integrity and accuracy of the point cloud and panoramic imagery even when the freight yard environment changes, vehicles move, or there is temporary occlusion.

[0029] S2. Gaussian filtering for denoising and point cloud data quality improvement: First, the original point cloud of monocular laser in the freight yard is acquired, and the neighborhood of the point cloud is determined by combining spatial features; Gaussian filtering with dynamic kernel width is used for denoising; then, outliers are removed by dynamic thresholding; in multi-sensor scenarios, the weights are updated according to the signal-to-noise ratio after denoising, and a high-precision point cloud is output. In dynamic monitoring environments at freight yards, monocular laser point clouds are inevitably subject to various noise interferences during acquisition, including ambient light fluctuations, sensor system errors, discrete scattering points caused by irregular reflective surfaces, and dynamic occlusion caused by vehicle or cargo movement. These noises significantly affect the accuracy of the point cloud and subsequent fusion results, thereby reducing the reliability of the monitoring system. To address this, this solution proposes a multi-level precise denoising method based on Gaussian filtering, combined with dynamic point cloud quality assessment and adaptive weight update mechanisms, to refine the point cloud processing throughout the entire process from acquisition, filtering, anomaly removal to data fusion.

[0030] The original point cloud set acquired by monocular laser scanning of the cargo yard is represented as follows: Each point Its actual location may be affected by noise interference. For each point The set of neighboring points is determined by combining spatial distance, local curvature, density changes, and historical frame information. The Gaussian filter formula for neighborhood weighted smoothing is applied to the points. For noise reduction, the filtering formula is: ; in, For point The set of neighborhood points, The width of the Gaussian kernel is used to control the smoothing intensity. For point Points after filtering and denoising for Points within the neighborhood. Neighborhood selection considers not only spatial distance but also local curvature, density variations, and historical frame information to enhance the adaptability of the filter. In practical applications, It can be dynamically adjusted according to the data collection environment to maintain stable accuracy of point clouds under different lighting, wind speed, or moving occlusion conditions.

[0031] To ensure point cloud quality in real time, each frame of point cloud undergoes quality assessment before and after denoising, and the local noise metric of the points is calculated: ; when At this point, the point is identified as an abnormal noise point and needs to be further removed or replaced. Threshold Dynamically adjust based on sensor accuracy and environmental disturbances: ; in, As the initial threshold, This is the adjustment coefficient; In multi-sensor acquisition scenarios, the signal-to-noise ratio after denoising of each sensor is used. As the basis for weight adjustment, the sensor fusion weights are updated through a dynamic weight update mechanism, and the update formula is as follows: ; in, The fusion weights of the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. To ensure a high update rate, the fused point cloud maintains high accuracy even when the sensor is interfered with or experiences error fluctuations.

[0032] The entire process forms a closed-loop iterative mechanism, which optimizes filtering parameters, dynamically adjusts point cloud quality thresholds, and updates sensor weights in real time based on environmental disturbances, acquisition conditions, and dynamic changes of target objects. This enables the continuous output of high-precision and high-completeness point cloud data in complex freight yard environments, providing a reliable data foundation for subsequent panoramic fusion, target recognition, and dynamic monitoring.

[0033] S3.3D Gaussian splash processing and panoramic video generation: First, a limited-view video sequence of the freight yard is acquired, and each frame image is calibrated and mapped to a local point cloud; then, the invisible area point cloud is completed by using the local tensor field and historical frame state, and after multi-scale fusion, it is projected to generate a preliminary panoramic video frame; finally, through pixel-level depth confidence fusion optimization, a freight yard panoramic video with consistent time sequence and viewpoint is generated to support vehicle monitoring.

[0034] To achieve full-view cloud and corresponding panoramic video generation, this solution proposes a spatiotemporal continuous reasoning and multi-scale fusion method based on finite-view video.

[0035] The limited-viewpoint video sequence is as follows: ;in For the nth frame image in the video sequence Each frame of the image is used to obtain a local point cloud set through calibration mapping: ; in, Let be the local point cloud set at time t. For the first in this set point Limited by visibility and noise disturbance; Point cloud completion for invisible regions: defining local tensor fields The local tensor field Given a local point cloud set, the tensor can describe the local directionality, density, and curvature constraints of each point; based on the local tensor field... Recursive encoding state of historical frames Through prediction function Generate missing points in invisible regions of a local point cloud, denoted as . The prediction relationship is: ; Generate missing points of This is the recursive encoding state for historical frames, used to maintain temporal continuity and structural consistency.

[0036] Multi-scale point cloud fusion and panoramic projection: A multi-scale fusion strategy is adopted to perform nonlinear weighted accumulation of predicted point clouds at different scales; combined with camera extrinsic parameters. in Let be the camera rotation matrix at time t. Let be the camera displacement vector at time t, and then project it using the perspective projection function. The fused predicted point cloud is mapped onto a two-dimensional image plane to generate preliminary panoramic video frames. The mapping relationship is as follows: ; in, Let t be the initial panoramic video frame at time t. The fused predicted point cloud is at time t. Panoramic video optimization and panoramic viewpoint cloud generation: Pixel-level deep confidence fusion technology is introduced to perform multi-frame accumulation and local filtering optimization on the initial panoramic video frames, generating a high-precision panoramic viewpoint cloud with consistent temporal sequence and viewpoint. The panoramic viewpoint cloud possesses spatial continuity, geometric consistency, and structural integrity, providing reliable data support for subsequent tasks such as dynamic target detection, trajectory analysis, and monitoring of freight yard vehicles.

[0037] This method forms a closed-loop generation process from limited-viewpoint videos to complete panoramic cloud and panoramic videos by tensor field prediction, multi-scale fusion, recursive temporal coding and pixel-level deep fusion, realizing high-precision panoramic reconstruction and continuous visualization of target objects in dynamic freight yards.

[0038] Example 2 This embodiment 2 provides a freight yard vehicle monitoring system based on monocular laser point clouds, including: A freight yard vehicle monitoring system based on monocular laser point clouds, characterized in that it includes: Multi-sensor calibration and fusion and panoramic image generation unit; establishes sensor-world coordinate mapping, and after optimization and calibration, weighted fusion of point clouds and dynamically adjusts weights in combination with real-time signal-to-noise ratio; converts the fused point cloud into a two-dimensional image, and generates a panoramic distance image through registration, distance filtering and filtering optimization, and relies on real-time data stream updates to ensure accuracy; The Gaussian filtering denoising and point cloud data quality improvement unit first acquires the original point cloud of the monocular laser in the freight yard, and determines the neighborhood of the point cloud by combining spatial features; then it uses dynamic kernel width Gaussian filtering for denoising; subsequently, it removes outliers by dynamic thresholding; in multi-sensor scenarios, it updates the weights according to the signal-to-noise ratio after denoising and outputs a high-precision point cloud. The 3D Gaussian splash processing and panoramic video generation unit first acquires a video sequence with limited viewpoints in the freight yard, and then maps each frame image to a local point cloud. Next, it uses the local tensor field and historical frame states to complete the invisible area point cloud, and after multi-scale fusion, it projects to generate preliminary panoramic video frames. Finally, through pixel-level depth confidence fusion optimization, it generates a freight yard panoramic video with consistent time sequence and viewpoint to support vehicle monitoring.

[0039] Example 3 Please refer to Figure 2 This embodiment 3 also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement any step of a freight yard vehicle monitoring method based on monocular laser point clouds.

[0040] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0041] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0042] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring freight yard vehicles based on monocular laser point clouds, characterized in that, Includes the following steps: S1. Construction of a portable test field under multiple constraints: Based on the monitoring needs of the cargo yard, the sensor layout is planned using a multi-objective optimization model; a mapping between the sensor and the world coordinates is established, and after optimization and calibration, the weighted fusion point cloud is combined with the real-time signal-to-noise ratio for dynamic weight adjustment; the fused point cloud is converted into a two-dimensional image, and after registration, distance filtering and filtering optimization, a panoramic distance image is generated, which is updated by real-time data stream to ensure accuracy; S2. Gaussian filtering for denoising and point cloud data quality improvement: First, the original point cloud of monocular laser in the freight yard is acquired, and the neighborhood of the point cloud is determined by combining spatial features; Gaussian filtering with dynamic kernel width is used for denoising; then, outliers are removed by dynamic thresholding; in multi-sensor scenarios, the weights are updated according to the signal-to-noise ratio after denoising, and a high-precision point cloud is output. S3.3D Gaussian splash processing and panoramic video generation: First, a limited-view video sequence of the freight yard is acquired, and each frame image is calibrated and mapped to a local point cloud; then, the invisible area point cloud is completed by using the local tensor field and historical frame state, and after multi-scale fusion, it is projected to generate a preliminary panoramic video frame; finally, through pixel-level depth confidence fusion optimization, a freight yard panoramic video with consistent time sequence and viewpoint is generated to support vehicle monitoring.

2. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 1, characterized in that, The step S1, which plans the sensor layout according to the monitoring needs of the freight yard using a multi-objective optimization model, includes the following steps: Taking into account the complexity of the freight yard environment and the trajectory of the target object, and using coverage, point cloud sampling density, system computing resource utilization and sensor installation safety as constraints, a multi-objective optimization model is constructed to determine the optimal installation location of the sensor. Let the sensor set be The sensor i sensor is installed at the following location: The coverage radius is Construct a multi-objective optimization model to determine the optimal sensor installation location function; ； in, Indicates the effective coverage area. Indicates the overlapping area. Indicates installation and maintenance costs. is the weighting coefficient, and max indicates taking the maximum value.

3. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 1, characterized in that, In step S1, establishing the sensor-world coordinate mapping and, after optimization and calibration, the weighted fusion of point clouds combined with real-time signal-to-noise ratio dynamic weight adjustment includes the following steps: Viewpoint Consistency Modeling and Point Cloud Fusion: A mapping relationship between multiple sensors and the world coordinate system is established to achieve data unification. Weighted least squares method is used to fuse point clouds from multiple sensors. The mapping relationship is as follows: ； in The point cloud collected by sensor i For world coordinate point cloud, The rotation matrix corresponding to sensor i, Let i be the displacement vector corresponding to sensor i; The rotation matrix corresponding to sensor i is adjusted by least squares optimization. Displacement vector corresponding to sensor i To improve calibration accuracy, the relationship is optimized as follows: ； in Let be the mean point of the k-th class of points collected by sensor i. This is the standard point of the k-th type of point in the world coordinate system; The fusion relationship of the weighted least squares method is as follows: ； in For the merged point cloud, Let be the fusion weights for sensor i; Dynamic weighting is achieved by incorporating real-time error feedback. Updated to further improve the integration effect.

4. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 3, characterized in that, The dynamic weighting is achieved by combining real-time error feedback. The update steps specifically include: Dynamic weight update: When fusing multi-sensor point clouds using the weighted least squares method, the fusion weights of each sensor are dynamically updated using the following formula, combined with real-time error feedback: ； Let i be the fusion weights for the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. For update rate, To measure the signal-to-noise ratio in real time, The sum of the signal-to-noise ratios measured by the sensors in real time; For each point cloud acquired by the sensor, a perspective projection function is used. After mapping, we obtain two-dimensional pixel coordinates, and the projection relationship is as follows: ； in This is a perspective projection function that maps a 3D point cloud to 2D pixels; Let be the two-dimensional pixel coordinates corresponding to the point cloud of the i-th sensor. Let be the rotation matrix of the i-th sensor. For world coordinate point cloud, Let i be the displacement vector of the i-th sensor. The projection results from different sensors are aligned using an image registration algorithm, and then weighted and fused according to the minimum distance principle to generate a panoramic distance image. The fusion relationship is as follows: ； in Let be the depth value of the pixel corresponding to sensor i. Select the nearest boundary that covers the target object with the minimum distance; Depth Optimization and Dynamic Update: Gaussian kernel filtering is used to perform depth interpolation and smoothing on the panoramic distance image. The optimization relationship is as follows: ； in The optimized panoramic distance image depth value, where K is the Gaussian kernel. For pixels Neighborhood, for The system uses neighboring pixels and combines real-time sensor data streams to dynamically update the panoramic image, ensuring the integrity and accuracy of the point cloud and panoramic image when the freight yard environment changes, vehicles move, or there is temporary obstruction.

5. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 1, characterized in that, In step S2, the original point cloud of the cargo yard is obtained by monocular laser, and the neighborhood of the point cloud is determined by combining spatial features. Denoising using a Gaussian filter with dynamic kernel width; including the following steps: The original point cloud set acquired by monocular laser scanning of the cargo yard is represented as follows: Each point Its actual location may be affected by noise interference. For each point The set of neighboring points is determined by combining spatial distance, local curvature, density changes, and historical frame information. The Gaussian filter formula for neighborhood weighted smoothing is applied to the points. For noise reduction, the filtering formula is: ； in, For point The set of neighborhood points, The width of the Gaussian kernel is used to control the smoothing intensity. For point Points after filtering and denoising for Points within the neighborhood.

6. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 1, characterized in that, The method of removing outliers using dynamic thresholds; In multi-sensor scenarios, updating weights based on the denoised signal-to-noise ratio includes: Abnormal noise point removal: The quality of each frame of point cloud is evaluated before and after denoising, and the noise level of each point is calculated. Local noise measurement The measurement formula is: ； when When this point is identified as an abnormal noise point, it needs to be further removed or replaced; threshold Dynamically adjust based on sensor accuracy and environmental disturbances: ； in, As the initial threshold, This is the adjustment coefficient; Multi-sensor dynamic weight update: In multi-sensor acquisition scenarios, the signal-to-noise ratio of each sensor after denoising is used as the basis for weight update. As the basis for weight adjustment, the sensor fusion weights are updated through a dynamic weight update mechanism, and the update formula is as follows: ； in, The fusion weights of the i-th sensor at the next time step. Let i be the fusion weight of the i-th sensor at the current moment. To ensure a high update rate, the fused point cloud maintains high accuracy even when the sensor is interfered with or experiences error fluctuations.

7. The freight yard vehicle monitoring method based on monocular laser point clouds according to claim 1, characterized in that, The method for 3D Gaussian splashing processing and panoramic video generation in step S3 includes the following steps: Local point cloud acquisition: Acquire a limited-view video sequence of the freight yard, denoted as: ;in Let be the nth frame image in the video sequence; map each frame image to a local point cloud set at the corresponding time point through sensor calibration, denoted as ; ； in, Let be the local point cloud set at time t. For the first in this set point Limited by visibility and noise disturbance; Point cloud completion for invisible regions: defining local tensor fields The local tensor field Given a local point cloud set, the tensor can describe the local directionality, density, and curvature constraints of each point; based on the local tensor field... Recursive encoding state of historical frames Through prediction function Generate missing points in invisible regions of a local point cloud, denoted as . The prediction relationship is: ； Multi-scale point cloud fusion and panoramic projection: A multi-scale fusion strategy is adopted to perform nonlinear weighted accumulation of predicted point clouds at different scales; combined with camera extrinsic parameters. in Let be the camera rotation matrix at time t. Let be the camera displacement vector at time t, and then project it using the perspective projection function. The fused predicted point cloud is mapped onto a two-dimensional image plane to generate preliminary panoramic video frames. The mapping relationship is as follows: ； in, Let t be the initial panoramic video frame at time t. The fused predicted point cloud is at time t. Panoramic video optimization and panoramic viewpoint cloud generation: Pixel-level deep confidence fusion technology is introduced to perform multi-frame accumulation and local filtering optimization on the initial panoramic video frames, generating a high-precision panoramic viewpoint cloud with consistent temporal sequence and viewpoint. The panoramic viewpoint cloud possesses spatial continuity, geometric consistency, and structural integrity, providing reliable data support for subsequent tasks such as dynamic target detection, trajectory analysis, and monitoring of freight yard vehicles.

8. A freight yard vehicle monitoring system based on monocular laser point clouds, characterized in that, include: Multi-sensor calibration and fusion and panoramic image generation unit; establishes sensor-world coordinate mapping, and after optimization and calibration, weighted fusion of point clouds and dynamically adjusts weights in combination with real-time signal-to-noise ratio; converts the fused point cloud into a two-dimensional image, and generates a panoramic distance image through registration, distance filtering and filtering optimization, and relies on real-time data stream updates to ensure accuracy; The Gaussian filtering denoising and point cloud data quality improvement unit first acquires the original point cloud of the monocular laser in the freight yard, and determines the neighborhood of the point cloud by combining spatial features; then it uses dynamic kernel width Gaussian filtering for denoising; subsequently, it removes outliers by dynamic thresholding; in multi-sensor scenarios, it updates the weights according to the signal-to-noise ratio after denoising and outputs a high-precision point cloud. The 3D Gaussian splash processing and panoramic video generation unit first acquires a video sequence with limited viewpoints in the freight yard, and then maps each frame image to a local point cloud. Next, it uses the local tensor field and historical frame states to complete the invisible area point cloud, and after multi-scale fusion, it projects to generate preliminary panoramic video frames. Finally, through pixel-level depth confidence fusion optimization, it generates a freight yard panoramic video with consistent time sequence and viewpoint to support vehicle monitoring.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor according to any one of claims 1-7, a method for monitoring freight yard vehicles based on monocular laser point clouds.

Images