A vehicle centering detection method and device based on deep vision, and an electronic device

Abstract

The application discloses a vehicle centering detection method and device based on deep vision, and electronic equipment, and the method comprises the following steps: setting a dynamic cut-off threshold interval on a depth axis based on the prior height physical parameter of a target vehicle, performing spatial reference filtering on three-dimensional depth data of a working area of the target vehicle, performing spatial plane segmentation on the filtered data, fitting a three-dimensional plane equation of a surface of the target vehicle area and extracting effective inner points, orthogonally projecting the effective inner points to a two-dimensional physical grid along a normal vector, extracting an outer contour spatial point set through a computational geometry algorithm, dividing the contour point set into a horizontal boundary set and a vertical boundary set, fitting a straight line to the horizontal boundary and calculating a symmetric center line to output a horizontal centering deviation and a yaw angle, and extracting statistical geometric feature points from the vertical boundary to output a vertical positioning deviation. The application directly performs geometric calculation in a three-dimensional physical space, gets rid of the dependence on a two-dimensional pixel gradient, and realizes high-precision and high-robustness loading positioning of various vehicles at a millimeter level.

Description

Technical Field

[0001] This invention relates to the fields of machine vision, photoelectric detection and automated loading technology, and to a vehicle centering detection method and device, and electronic equipment based on depth vision. Background Technology

[0002] In high-end industrial automated logistics, vehicle transportation, and rail freight scenarios, loading equipment requires extremely high accuracy in vehicle alignment detection and environmental robustness. Existing technologies mainly include the following solutions:

[0003] (1) Centering detection scheme based on 2D vision:

[0004] CN105488454B discloses a method for detecting and ranging vehicles ahead based on monocular vision. It utilizes a monocular camera to acquire images of the vehicle ahead and achieves vehicle detection through edge detection and vehicle hypothesis generation. This scheme primarily focuses on detecting vehicles ahead and estimating distance, without addressing the centering deviation between the vehicle and the target object, nor does it include yaw angle estimation functionality.

[0005] CN105987684A discloses an agricultural vehicle navigation line detection system and method based on monocular vision, which determines the navigation line through green feature extraction and centroid detection. This scheme relies on the green features of crops for navigation, making it unsuitable for grayscale textured scenes such as railway slabs, and the color segmentation method cannot suppress interference from brightness variations such as shadows.

[0006] CN116080644A discloses a vehicle driving control method, device, and electronic device. This scheme acquires road images using a single forward-looking camera, obtains a set of lane line edge coordinate points through Gaussian filtering, Canny edge detection, and ROI region extraction, and calculates a set of lane center reference points to achieve lane-centered driving. However, this scheme uses a general edge detection method, which generates a large number of false edges in scenes with severe shadow interference, and it cannot solve the detection problem when the target has a yaw angle.

[0007] The aforementioned two-dimensional vision scheme has the following inherent limitations at the physical level: traditional monocular vision is highly dependent on ambient lighting and the two-dimensional texture of the target surface. When exposed to strong direct light, local shadows, or when the target surface is severely dirty, it is very easy to produce false edge jitter. At the same time, two-dimensional images cannot directly reflect the physical thickness and spatial orientation of the target, and it is difficult to eliminate the observation errors caused by the chamfering and thickness of the target edges.

[0008] (2) Centering detection scheme based on depth vision:

[0009] CN112233136A discloses a method, system, device, and storage medium for truck alignment based on binocular vision. This method simultaneously acquires left and right images using a binocular camera device, calculates the disparity matrix and depth values ​​to generate a 3D point cloud of the truck, performs threshold segmentation on the 3D point cloud, and obtains the coordinates of alignment feature points through plane fitting. The current position of the alignment feature points is compared with a preset position range to generate forward, backward, or stop alignment commands. However, this scheme has the following shortcomings: First, its depth perception modality is limited to binocular vision, making it incompatible with other depth sensing mechanisms such as Time-of-Flight (ToF) cameras, structured light depth cameras, and LiDAR. Second, its point cloud processing only extracts the coordinates of a single alignment feature point and outputs only one-dimensional forward / backward alignment commands, failing to provide full-degree-of-freedom pose parameters such as lateral alignment deviation and yaw angle. Third, its computational architecture is tied to specific hardware, failing to address cross-platform compatibility issues, making seamless algorithm migration difficult during system upgrades or hardware cost reductions.

[0010] CN122392014A discloses a method for recognizing and calculating the pose of a van. It acquires an initial point cloud using a Time-of-Flight (TOF) camera, and then calculates the van's deflection angle and pose information through coordinate transformation, preprocessing, normal vector calculation, bottom surface fitting, and line fitting. Although this method uses a depth camera and outputs the deflection angle, it focuses on locating the rear door of the van to facilitate the loading robot's grasping, rather than performing full-degree-of-freedom pose detection for loading and centering scenarios.

[0011] In summary, the existing technologies mainly suffer from the following technical bottlenecks:

[0012] First, two-dimensional vision solutions are severely limited by lighting conditions. They cannot work stably under direct strong light, local shadows, or when the target surface is dirty. Furthermore, they lack depth dimension information and cannot handle geometric deviations caused by edge chamfering and thickness.

[0013] Second, existing depth vision solutions (such as CN112233136A) have a single sensing mode, supporting only binocular vision, and cannot flexibly select the optimal depth sensing mechanism according to the specific environmental conditions of the industrial site.

[0014] Third, the existing solution only outputs one-dimensional forward / backward alignment commands and does not provide full-degree-of-freedom pose parameters such as lateral alignment deviation and yaw angle, which cannot meet the needs of high-precision loading and alignment scenarios for multi-dimensional deviation detection.

[0015] Fourth, existing 3D point cloud solution solutions are usually tightly bound to specific underlying hardware acceleration architectures, lacking cross-platform compatibility, and it is difficult to achieve seamless algorithm migration when upgrading the system or reducing hardware costs. Summary of the Invention

[0016] To address the aforementioned technical problems in existing technologies, this invention proposes a vehicle centering detection method, device, and electronic equipment based on depth vision. This method enables the calculation of vehicle boundaries in three-dimensional physical space, is compatible with multiple depth sensing mechanisms and computing platforms, and outputs full-degree-of-freedom pose parameters including lateral centering deviation, yaw angle, and longitudinal positioning deviation. The specific technical solution is as follows:

[0017] A vehicle centering detection method based on depth vision includes the following steps:

[0018] Step 1: Obtain the 3D depth data of the target vehicle's working area and convert it to a unified reference coordinate system;

[0019] Step 2: Based on the prior height physical parameters of the target vehicle, a dynamic truncation threshold range is set on the depth axis, and spatial reference filtering is performed on the three-dimensional depth data.

[0020] Step 3: Perform spatial plane segmentation on the filtered and retained depth data, fit the three-dimensional plane equation of the target vehicle's working area surface and extract effective interior points to eliminate spatial outliers;

[0021] Step 4: Project the effective interior points orthogonally onto the two-dimensional physical grid along the normal vector of the three-dimensional plane equation, and use computational geometry algorithms to extract the contour point set of the working area;

[0022] Step 5: Divide the contour point set into a horizontal boundary set and a vertical boundary set. Perform line fitting on the horizontal boundary set and calculate the symmetrical centerline of the two edge lines to output the horizontal centering deviation and yaw angle. Extract statistical geometric feature points from the vertical boundary set to output the vertical positioning deviation.

[0023] Furthermore, the three-dimensional depth data in step 1 comes from the depth vision acquisition module, which is selected from one or more combinations of the following: ToF depth camera, structured light depth camera, active speckle binocular depth camera, active speckle triocular depth camera, passive binocular depth camera, passive triocular depth camera, and solid-state LiDAR.

[0024] Further, step 2 specifically involves: setting an upper limit threshold and a lower limit threshold based on the target vehicle's calibrated height, performing one-dimensional pass-through filtering along the depth axis, retaining the depth data between the upper limit threshold and the lower limit threshold, and removing track surface and ground surface data below the lower limit threshold as well as background structures and suspended noise data above the upper limit threshold.

[0025] Furthermore, in step 3, the spatial plane segmentation employs the RANSAC random sampling consensus algorithm. Three non-collinear spatial points are randomly selected from the filtered depth data to iteratively fit candidate planes. The plane containing the most valid interior points is then solved and output as the optimal spatial plane equation. ,in , , , The coefficients of the plane equation, , , These are spatial coordinates.

[0026] Furthermore, the computational geometry algorithm employs either the convex hull algorithm or the Alpha-Shape algorithm; the straight line fitting employs either the total least squares method (TLS) or the principal component analysis method (PCA).

[0027] Furthermore, it also includes: injecting the output lateral centering deviation, yaw angle, and longitudinal positioning deviation into an extended Kalman filter (EKF) containing position and velocity state parameters for second-order dynamic smoothing filtering.

[0028] A vehicle centering detection device based on depth vision, comprising:

[0029] The depth vision acquisition module is used to acquire three-dimensional depth data of the target vehicle's working area, and the three-dimensional depth data includes spatial coordinate information of physical absolute scale;

[0030] The data transmission link, using a high-bandwidth industrial vision or high-speed serial interface, is connected between the depth vision acquisition module and the data processing platform to transmit the three-dimensional depth data from the depth vision acquisition module to the data processing platform.

[0031] The data processing platform is connected to the depth vision acquisition module and is used to perform coordinate transformation, spatial filtering, plane fitting and noise reduction, dimensionality reduction projection and topological boundary extraction, and orthogonal space calculation on the three-dimensional depth data, and output the lateral centering deviation, yaw angle, and longitudinal positioning deviation.

[0032] Furthermore, the data processing platform specifically includes:

[0033] The depth data acquisition module is used to convert the acquired three-dimensional depth data of the target vehicle's working area into a preset world reference coordinate system to generate a target space point cloud;

[0034] The spatial reference filtering module is used to set a dynamic truncation threshold range on the depth axis based on the prior physical height parameters of the target vehicle to cut off the background environment point cloud and obtain the target candidate point cloud;

[0035] The spatial plane extraction module is used to perform spatial plane segmentation on the target candidate point cloud, fit the three-dimensional plane equation of the vehicle area surface, and extract the effective interior point set.

[0036] The dimension reduction projection module is used to orthogonally project the effective set of interior points onto a two-dimensional physical grid along the normal vector of the plane, and extract the spatial point set of the outer physical contour.

[0037] The geometric pose calculation module is used to divide the spatial point set into a horizontal boundary set and a vertical boundary set, calculate the horizontal centering deviation and yaw angle by fitting spatial straight lines, and calculate the vertical positioning deviation by extracting feature points.

[0038] Furthermore, the hardware architecture of the data processing platform includes any of the following:

[0039] Option 1 is a pure software computing architecture based entirely on a general-purpose central processing unit (CPU). All depth data parsing and spatial geometry solving algorithms are implemented in software programming on a general-purpose processor.

[0040] Option 2 is a heterogeneous computing architecture based on a coprocessor and a general-purpose processor, in which the underlying high-density pixel operations are performed by the coprocessor and the spatial geometry calculations are performed by the general-purpose processor; the coprocessor includes one or more of a field-programmable gate array (FPGA), a digital signal processor (DSP), or a neural network processor (NPU).

[0041] A depth vision-based vehicle centering detection electronic device, comprising:

[0042] Memory, used to store program instructions;

[0043] A processor, coupled to the memory, is configured to execute the program instructions, the program instructions being configured to perform the method of any one of claims 1 to 6.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] First, this invention eliminates the dependence on two-dimensional pixel gradients. By performing height domain truncation filtering, RANSAC spatial plane fitting, and normal orthogonal projection in three-dimensional physical space, it fundamentally eliminates edge detection errors caused by edge chamfering, thickness, and illumination changes, achieving millimeter-level high-precision centering and positioning.

[0046] Second, the method of the present invention is independent of the specific type of depth sensor and is compatible with various sensing modes such as time-of-flight (ToF) cameras, structured light depth cameras, active / passive binocular or multi-view depth cameras, and solid-state lidar. It can flexibly select the optimal sensor configuration according to the environmental conditions (lighting, field of view, distance, etc.) of the industrial site.

[0047] Third, the method of the present invention is independent of specific computing hardware architecture. It can run on both general-purpose CPU platforms and heterogeneous computing platforms such as ARM+FPGA. The underlying algorithm code is not bound to a specific manufacturer's instruction set, and has strong cross-platform portability and code reusability.

[0048] Fourth, the present invention simultaneously outputs lateral centering deviation, yaw angle and longitudinal positioning deviation, providing the vehicle's full-degree-of-freedom pose information on the loading platform, thus overcoming the shortcomings of the prior art which only outputs one-dimensional alignment commands. Attached Figure Description

[0049] Figure 1 This is a system block diagram of the vehicle alignment detection device provided in an embodiment of the present invention.

[0050] Figure 2 This is an overall flowchart of the vehicle alignment detection method provided in an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram of height domain spatial reference filtering provided in an embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram illustrating the principle of RANSAC spatial plane fitting provided in an embodiment of the present invention.

[0053] Figure 5 This is a hardware connection structure diagram of the vehicle alignment detection device provided in an embodiment of the present invention.

[0054] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0055] Figure 7 This is a schematic diagram illustrating the use of the vehicle alignment detection device provided in this embodiment of the invention in a vehicle loading scenario. Detailed Implementation

[0056] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0057] Example 1:

[0058] In this embodiment 1, a method, apparatus, and electronic device for centering detection between a depth camera and a general-purpose CPU architecture based on signal time-of-flight (ToF) are implemented.

[0059] like Figure 1 and Figure 6As shown, this embodiment is aimed at industrial sites with standardized computing platforms and high requirements for hardware cost and deployment agility. It provides a lightweight vehicle centering detection system that uses a single sensor (time-of-flight camera) and relies entirely on a general-purpose central processing unit (CPU) for 3D point cloud calculation.

[0060] The system hardware architecture in this embodiment 1 includes:

[0061] (1) Depth vision acquisition module: The ToF depth camera is used as the core sensing source. It emits high-frequency modulated near-infrared light pulses to the target area of ​​the vehicle and receives the reflected light. The absolute depth of each pixel in the scene is calculated at the physical level by using the time of flight or phase difference of light. (Value), directly outputting a point cloud matrix with three-dimensional spatial coordinates.

[0062] (2) Data processing platform: The data processing platform adopts a standardized high-performance multi-core general-purpose processor (such as an industrial control computer based on ARM Cortex-A architecture or x86 architecture). All geometric analysis and mathematical fitting algorithms are compiled and executed in pure software at the C / C++ level in conjunction with the SIMD instruction set (such as ARM NEON or Intel AVX).

[0063] (3) Data transmission link: A serial digital interface (SDI) is used to realize data transmission and communication between the depth data acquisition module and the data processing platform. The system is deployed in a gigabit local area network environment to ensure that the high-frequency spatial pose data stream has extremely low transmission latency.

[0064] Based on the above system, a vehicle centering detection method based on depth vision is implemented in this embodiment, such as... Figure 2 As shown, the steps are as follows:

[0065] Step 101: Acquire 3D depth data: The ToF depth camera emits an active light source into the vehicle's working area in a global exposure mode. The internal sensor of the camera captures the backscattered light, calculates the phase delay through the integrated circuit, directly generates a high frame rate depth map, and maps it into 3D point cloud data in the camera's physical coordinate system, realizing nanosecond-level light pulse modulation and direct depth generation.

[0066] Step 102, Spatial reference filtering based on height topology constraints: such as Figure 3 As shown, the point cloud data is loaded into the main control CPU memory via gigabit network. The CPU determines the target vehicle's height based on its known engineering calibration. Establish upper limit threshold With lower threshold Along the depth axis ( (Axis) Perform one-dimensional pass-through filtering to remove redundant track surface, background structures and floating noise data from the global point cloud.

[0067] Step 103, Spatial plane fitting and noise reduction: (e.g.) Figure 4 As shown, the CPU calls a multi-threaded parallel RANSAC algorithm to randomly select three non-collinear spatial points from the filtered target region surface point cloud to fit a candidate plane. It calculates the distance from all points to the plane, counts the number of inliers whose distance is less than a threshold, iteratively solves for and outputs the equation of the optimal spatial plane containing the most inliers.

[0068] (1)

[0069] in , , , The coefficients of the plane equation, , , Spatial coordinates;

[0070] This step effectively eliminates three-dimensional spatial protrusions caused by chamfered edges or foreign objects on the vehicle's surface.

[0071] Step 104, Dimensional Reduction Projection and Topological Boundary Extraction: The set of effective interior points constituting the plane is orthogonally projected onto a two-dimensional reference plane along the normal vector of the plane to generate a two-dimensional physical occupancy grid matrix. The spatial convex hull algorithm or the Alpha-Shape algorithm is called to calculate and extract the set of physical contour points of the outer boundary of the target working area surface.

[0072] Step 105, Orthogonal Space Calculation: Based on the world coordinate system benchmark of the vision system, the outer boundary point set is divided into a horizontal boundary set and a vertical boundary set. The CPU uses the total least squares (TLS) method to fit the equations of the left and right edge lines of the horizontal boundary set, and then calculates the symmetry axis of the two lines, outputting the horizontal centering deviation and yaw angle. Statistical centroid coordinates are extracted from the vertical boundary set, and the vertical positioning deviation is output.

[0073] After the final pose parameters are injected into an extended Kalman filter (EKF) containing position and velocity state parameters for second-order dynamic smoothing, control commands are sent to the loading actuator via a communication protocol, or the pose calculation results and acquired data images are output and displayed.

[0074] Technical effects of this embodiment 1:

[0075] This embodiment 1 completely eliminates the parallax hole problem in passive vision by directly outputting physical depth through a ToF camera. It eliminates the need for a complex FPGA heterogeneous acceleration pipeline, significantly reducing hardware material costs and system failure points. The algorithm pipeline is entirely based on the floating-point computing unit of a general-purpose CPU for spatial geometry analysis. The underlying code is not bound to the instruction set of any specific hardware vendor's coprocessor, exhibiting extremely high code reusability. By integrating a lightweight communication protocol within a closed gigabit LAN, it ensures that spatial alignment deviation and yaw angle commands can be delivered to the automation control end with sub-millisecond network latency.

[0076] Example 2:

[0077] In this embodiment 2, a high-performance centering detection method, device, and electronic device based on active trifocal vision and ARM+FPGA heterogeneous computing are implemented.

[0078] This embodiment 2 addresses harsh industrial environments characterized by strong light interference and homogeneous, textureless target surfaces, providing a high-performance heterogeneous computing solution based on an active tri-lens depth camera and deeply integrating an ARM processor and an FPGA coprocessor.

[0079] System hardware architecture:

[0080] (1) Active trinocular vision acquisition module: It consists of a high-frequency infrared speckle projector and three global shutter image sensors arranged with orthogonal baselines. To suppress interference from strong outdoor solar radiation, an ultra-narrow band interference filter is integrated at the front end of the sensor lens. The system uses nanosecond-level pulse projection and microsecond-level ultra-short exposure window to achieve hard synchronization, ensuring that high-contrast active texture images can still be acquired under strong light.

[0081] (2) The data processing platform is a highly integrated heterogeneous computing platform: it is built using a high-performance ARM architecture processor (RK3588 optional) and a programmable logic device FPGA (Xilinx Artix-7 optional). After the image sensor captures the multi-view raw video stream, it is directly connected to the FPGA logic unit through the MIPI or LVDS high-speed bus. The FPGA pre-instantiates the hardware lookup table (LUT) and DSP slices, and pipelines the image distortion correction and stereo epipolar correction. It executes the semi-global matching (SGM) algorithm in hardware and directly outputs a high-resolution dense depth map.

[0082] (3) Zero-copy transmission mechanism: After the FPGA generates the depth map, it uses the V4L2 framework and DMA-BUF mechanism through PCIe or internal high-speed bus to directly map the memory pointer of the depth map data block to the memory space of the ARM processor, thereby realizing zero-copy transmission and completely avoiding CPU intervention in data moving operations.

[0083] Core implementation steps:

[0084] Step 201, Nanosecond-level hardware triggering and low-level depth calculation (FPGA coprocessor side): After the image sensor captures the multi-view raw video stream, it is connected to the FPGA logic unit. The FPGA pipelines the image distortion correction and stereo epipolar correction as the pixel stream flows in, and executes the semi-global matching (SGM) algorithm in hardware to directly output a high-resolution dense depth map.

[0085] Step 202, Zero-copy transfer based on DMA-BUF: After the FPGA generates the depth map, the memory pointer of the depth map data is directly mapped to the memory space of the ARM processor through the high-speed bus to achieve zero-copy transfer.

[0086] Step 203, Spatial reference filtering and 3D plane extraction (ARM main control end): After the ARM multi-core processor acquires the depth data, it converts it into a 3D point cloud and performs Z-axis-based height pass-through filtering to remove the chassis and background environment. Then, the multi-threaded accelerated RANSAC algorithm is called to fit the 3D plane equation (1) of the vehicle area surface in the retained point cloud set.

[0087] Step 204, Dimensional Reduction Projection and Orthogonal Spatial Line Fitting: Project the points in the plane along the spatial normal vector onto a two-dimensional physical occupancy grid. Use a topology algorithm to extract the physical contour point set of the vehicle's outer side, dividing it into a lateral feature point set and a longitudinal feature point set. Use Total Least Squares (TLS) to fit lines to the left and right feature point sets respectively. By analyzing the spatial bisectors of the two lines, output the lateral absolute centering deviation and yaw angle; extract the statistical median coordinates from the front and rear feature point sets, outputting the longitudinal positioning deviation.

[0088] Step 205, Second-order dynamic state smoothing and distribution: The calculated deviation value and yaw angle are injected into the extended Kalman filter (EKF), and feedforward compensation is performed using the rate of change of the previous frame to eliminate measurement jumps caused by motion blur. Finally, the smoothing control command is pushed to the loading control terminal through the communication protocol.

[0089] Technical effects of this embodiment 2:

[0090] This embodiment 2 fully leverages the computing power of heterogeneous hardware, offloading high-density pixel-level computations (epipolar correction, SGM matching) to the FPGA through a hardware pipeline, while retaining complex floating-point geometric fitting and dynamic filtering on the ARM processor. Combined with zero-copy transmission and communication protocols, it achieves extremely low latency from photon incidence to pose command issuance. Utilizing the orthogonal baseline advantage and point cloud spatial resolution of the tri-view depth camera, it overcomes the blind zone limitation of parallel epipolar lines in binocular vision, simultaneously achieving high-precision centering in both the lateral and longitudinal directions.

[0091] Example 3:

[0092] In this embodiment 3, a vehicle centering detection method, device, and electronic equipment based on passive binocular white light vision and a pure general-purpose CPU architecture are implemented.

[0093] This embodiment is designed for industrial scenarios with stringent requirements for hardware material costs, adequate on-site lighting, and the desire for the vision system to be reusable for secondary vision tasks such as traditional manual monitoring or vehicle identification number (OCR). Embodiment 3 provides a system architecture that completely eliminates active light source projection and underlying hardware accelerators, relying solely on a passive binocular visible light camera and pure general-purpose CPU computing power for 3D spatial centering detection.

[0094] System hardware architecture:

[0095] (1) Passive binocular white light acquisition module: The acquisition module consists of binocular visible light (RGB or monochrome CMOS / CCD) image sensors arranged with a horizontal baseline. This embodiment 3 does not contain any active structured light or speckle projector, and relies entirely on natural ambient light or basic white light floodlight illumination in the industrial site. This module utilizes the natural physical features of the target vehicle area surface (such as seams, rust spots, scratches, edge steps, etc.) as the optical basis for stereo matching.

[0096] (2) Pure general-purpose multi-core CPU computing platform: The system is completely decoupled from specific hardware accelerators. After the visible light image data stream is accessed, image distortion correction, stereo epipolar correction, semi-global matching (SGM) and all subsequent three-dimensional geometric analysis tasks are completed on the general-purpose CPU, which relies heavily on multi-threaded concurrent scheduling and SIMD vectorized instruction set to ensure real-time operation.

[0097] Core implementation steps:

[0098] Step 301, Binocular White Light Image Capture and Adaptive Feature Enhancement: The main control CPU receives synchronized video frames from the left and right cameras. Addressing the technical challenge of passive vision failing on homogeneous surfaces, the CPU pre-calls SIMD instructions to perform Limited Contrast Adaptive Histogram Equalization (CLAHE) on the original image. This algorithm, by limiting excessive amplification of local contrast, suppresses ambient light noise while maximizing the amplification of natural textures and edge gradients on the homogeneous surface, providing high-confidence pixel features for subsequent disparity calculations.

[0099] Step 302, Epipolar Correction and Stereo Matching: The CPU uses a pre-generated nonlinear two-dimensional lookup table (LUT) to perform epipolar correction on the feature-enhanced image, ensuring strict horizontal alignment of the epipolar lines in the left and right views. Subsequently, for the epipolar-corrected image, the CPU runs a multi-threaded optimized semi-global matching (SGM) algorithm. To compensate for texture defects in passive vision, the algorithm adaptively increases the smoothing penalty coefficient during the cost aggregation stage. The disparity map is forced to maintain a continuous transition in weakly textured regions. A 2D disparity map is output and combined with a reprojection matrix (Q matrix) to convert it into a 3D point cloud with absolute physical scale.

[0100] Steps 303-305, and subsequent steps such as spatial reference filtering, RANSAC 3D plane extraction, normal orthogonal projection, TLS line fitting, and EKF smoothing are essentially the same as steps 102-105 in Example 1, and will not be repeated here.

[0101] Technical effects of this embodiment:

[0102] This embodiment employs a passive binocular visible light sensor, eliminating the expensive and lifespan-limited laser / infrared projector and its associated driving circuitry. It achieves 3D perception using only a standard industrial visible light camera and a general-purpose host, making it the architecture with the lowest hardware material cost and the longest mean time between failures (MTBF) among all embodiments. Due to the use of passive white light imaging, its original RGB / monochrome video stream possesses extremely high readability. High-definition visible light images acquired by the same hardware can be directly offloaded to other CPU cores or a lightweight NPU to simultaneously execute deep learning-based vehicle identification number (OCR), vehicle surface foreign object intrusion detection, and on-site visual scheduling and monitoring, achieving "one machine, multiple uses." The entire depth generation and spatial resolution pipeline is implemented entirely in C / C++ software code, enabling seamless portability to various x86 industrial hosts or ARM core boards, significantly reducing hardware supply chain risks and system lifecycle maintenance costs.

[0103] In a preferred embodiment of the present invention, the depth vision acquisition module and the data processing platform are physically connected via a high-bandwidth industrial vision or high-speed serial data transmission link for real-time streaming of the original video stream / depth map. To adapt to the large physical span, strong electromagnetic interference, and board-level interconnection requirements of heterogeneous computing in industrial environments such as large marshalling yards and vehicle loading sites, the data transmission link specifically adopts one or a combination of the following architectures:

[0104] Chip / board-level direct connection architecture (extremely low latency):

[0105] In highly integrated intelligent optoelectronic bases, the Mobile Industrial Processor Interface (MIPI-CSI) or Low Voltage Differential Signaling Interface (LVDS) is used to directly interface with the physical pins of ARM multi-core processors or FPGA coprocessors; or the high-speed serial computer expansion bus standard (PCIe / OCuLink) is used to achieve direct memory mapping across board levels, completing point cloud data acquisition at the system-on-a-chip (SoC) level with extremely low latency and low power consumption.

[0106] Heavy-duty industrial and automotive SerDes architecture (high interference immunity for long-distance transmission):

[0107] In a split-type loading and testing environment, when there is a large physical distance between the camera acquisition end and the computing processing end, a gigabit multimedia serial link (GMSL) or a flat panel display link (FPD-Link) is used to achieve long-distance transmission of video data, control signals and power (PoC) over a single cable using a single coaxial cable or twisted pair; or a serial digital interface (SDI, such as 3G-SDI / 12G-SDI) and vehicle audio and video transmission (AVT / AVB) are used to ensure signal integrity in heavy machinery vibration environments.

[0108] Ultra-high bandwidth industrial machine vision standard architecture (massive point cloud throughput):

[0109] For active tri-lens depth cameras with extremely high resolution or ultra-high frame rate, an industrial camera link interface (CameraLink, supporting Base / Medium / Full modes) or a coaxial fast display interface (CoaXPress, CXP-6 / CXP-12) can be used in conjunction with an image acquisition card to achieve extremely high throughput raw data retrieval; or a gigabit / 10 gigabit Ethernet interface (GigE Vision / 10GigE) and a universal serial bus interface (USB) based on TCP / UDP underlying protocols can be used to achieve plug-and-play high-speed collaboration with standardized industrial control computers.

[0110] Strong electromagnetic immune optical communication architecture (extremely harsh environment):

[0111] In extreme loading environments with strong electromagnetic interference such as the overhead contact system of ultra-high voltage electrified railways, fiber optic communication interfaces based on SFP+ / QSFP optical modules are used to convert the front-end deep photoelectric signals into pure optical signals for long-distance isolation transmission over kilometers, completely blocking the interference of spatial electromagnetic radiation on three-dimensional point cloud data at the physical medium level.

[0112] The vehicle alignment detection device provided in Examples 1-3 is applied in vehicle loading scenarios as follows: Figure 7 As shown, the system includes a target vehicle 801, a surface 802 in front of the target vehicle, a depth vision acquisition module 803, a data transmission link 804, a data processing platform 805, and a display and control terminal 806. The data processing platform 805 and the display and control terminal 806 are integrated into one unit.

[0113] Optionally, the data processing platform 805 can also be integrated with the depth vision acquisition module 803. Optionally, the depth vision acquisition module 803, the data processing platform 805, and the display and control terminal 806 can also be set up independently.

[0114] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle centering detection method based on depth vision, characterized in that, Includes the following steps: Step 1: Obtain the 3D depth data of the target vehicle's working area and convert it to a unified reference coordinate system; Step 2: Based on the prior height physical parameters of the target vehicle, a dynamic truncation threshold range is set on the depth axis, and spatial reference filtering is performed on the three-dimensional depth data. Step 3: Perform spatial plane segmentation on the filtered and retained depth data, fit the three-dimensional plane equation of the target vehicle's working area surface and extract effective interior points to eliminate spatial outliers; Step 4: Project the effective interior points orthogonally onto the two-dimensional physical grid along the normal vector of the three-dimensional plane equation, and use computational geometry algorithms to extract the contour point set of the working area; Step 5: Divide the contour point set into a horizontal boundary set and a vertical boundary set. Perform line fitting on the horizontal boundary set and calculate the symmetrical centerline of the two edge lines to output the horizontal centering deviation and yaw angle. Extract statistical geometric feature points from the vertical boundary set to output the vertical positioning deviation.

2. The method according to claim 1, characterized in that, The three-dimensional depth data in step 1 comes from the depth vision acquisition module, which is selected from one or more of the following combinations: ToF depth camera, structured light depth camera, active speckle binocular depth camera, active speckle triocular depth camera, passive binocular depth camera, passive triocular depth camera, and solid-state LiDAR.

3. The method according to claim 1, characterized in that, Step 2 specifically involves: setting an upper and lower threshold based on the target vehicle's calibrated height, performing one-dimensional pass-through filtering along the depth axis, retaining the depth data between the upper and lower thresholds, and removing track and ground surface data below the lower threshold as well as background structures and suspended noise data above the upper threshold.

4. The method according to claim 1, characterized in that, In step 3, the spatial plane segmentation uses the RANSAC random sampling consensus algorithm. Three non-collinear spatial points are randomly selected from the filtered depth data to iteratively fit candidate planes. The plane containing the most valid interior points is then solved and output as the optimal spatial plane equation. ,in , , , The coefficients of the plane equation, , , These are spatial coordinates.

5. The method according to claim 1, characterized in that, The computational geometry algorithm employs either the convex hull algorithm or the Alpha-Shape algorithm; the linear fitting employs either the total least squares method (TLS) or the principal component analysis method (PCA).

6. The method according to claim 1, characterized in that, Also includes: The output lateral centering deviation, yaw angle, and longitudinal positioning deviation are injected into an extended Kalman filter (EKF) containing position and velocity state parameters for second-order dynamic smoothing filtering.

7. A vehicle alignment detection device employing the method according to any one of claims 1 to 6, characterized in that, include: The depth vision acquisition module is used to acquire three-dimensional depth data of the target vehicle's working area, and the three-dimensional depth data includes spatial coordinate information of physical absolute scale; The data transmission link, using a high-bandwidth industrial vision or high-speed serial interface, is connected between the depth vision acquisition module and the data processing platform to transmit the three-dimensional depth data from the depth vision acquisition module to the data processing platform. The data processing platform is connected to the depth vision acquisition module and is used to perform coordinate transformation, spatial filtering, plane fitting and noise reduction, dimensionality reduction projection and topological boundary extraction, and orthogonal space calculation on the three-dimensional depth data, and output the lateral centering deviation, yaw angle, and longitudinal positioning deviation.

8. The apparatus as claimed in claim 7, characterized in that, The data processing platform specifically includes: The depth data acquisition module is used to convert the acquired three-dimensional depth data of the target vehicle's working area into a preset world reference coordinate system to generate a target space point cloud; The spatial reference filtering module is used to set a dynamic truncation threshold range on the depth axis based on the prior physical height parameters of the target vehicle to cut off the background environment point cloud and obtain the target candidate point cloud; The spatial plane extraction module is used to perform spatial plane segmentation on the target candidate point cloud, fit the three-dimensional plane equation of the vehicle area surface, and extract the effective interior point set. The dimension reduction projection module is used to orthogonally project the effective set of interior points onto a two-dimensional physical grid along the normal vector of the plane, and extract the spatial point set of the outer physical contour. The geometric pose calculation module is used to divide the spatial point set into a horizontal boundary set and a vertical boundary set, calculate the horizontal centering deviation and yaw angle by fitting spatial straight lines, and calculate the vertical positioning deviation by extracting feature points.

9. The apparatus as claimed in claim 7, characterized in that, The hardware architecture of the data processing platform includes any of the following: Option 1 is a pure software computing architecture based entirely on a general-purpose central processing unit (CPU). All depth data parsing and spatial geometry solving algorithms are implemented in software programming on a general-purpose processor. Option 2 is a heterogeneous computing architecture based on a coprocessor and a general-purpose processor, in which the underlying high-density pixel operations are performed by the coprocessor and the spatial geometry calculations are performed by the general-purpose processor; the coprocessor includes one or more of a field-programmable gate array (FPGA), a digital signal processor (DSP), or a neural network processor (NPU).

10. A vehicle centering detection electronic device based on depth vision, characterized in that, include: Memory, used to store program instructions; A processor, coupled to the memory, is configured to execute the program instructions, the program instructions being configured to perform the method of any one of claims 1 to 6.

Images