A three-camera vision device for cooperative inter-vehicle relative pose measurement

Abstract

The utility model discloses a kind of three vision devices of relative pose measurement between collaborative vehicles, install in the collaborative intersection side of initiative vehicle.The device includes three camera components, image processing component, external communication component;Three camera components include the first camera, second camera and third camera along the same straight line setting, first camera and second camera are used to form stereoscopic precision measurement area;Third camera is used to cover precision measurement area and form rough measurement area.Image processing component includes FPGA for image synchronous acquisition and pre-processing and NPU for target detection and stereoscopic matching solution.The utility model is used for vehicle autonomous collaborative berthing, intersection docking scene, through three coaxial arrangement and rough measurement / precision measurement fusion mode, realize the unity of long-distance target detection and close-range high-precision stereoscopic measurement, improve the measurement stability and real-time of vehicle collaborative intersection process, device structure is compact, applicable to vehicle vibration environment.

Description

Technical Field

[0001] This utility model relates to the field of visual measurement technology in vehicle collaborative operations, and in particular to a trinocular vision device for measuring the relative pose between vehicles. Background Technology

[0002] With the development of intelligent vehicles and unmanned autonomous systems, automatic collaborative rendezvous, approach, and docking between vehicles has become a key requirement. In such tasks, active vehicles typically need to detect and identify cooperative targets of passive vehicles at medium to long distances, and acquire high-precision relative position and attitude information during the approach phase to ensure the safety and reliability of the rendezvous and docking process.

[0003] Existing technologies mostly employ binocular vision or single-camera vision measurement solutions, but they have the following shortcomings: 1. A single camera setup struggles to balance coarse long-range measurements with precise close-range measurements. Wide-angle lenses offer a large field of view but reduce image resolution; telephoto lenses, while providing high precision, have a limited field of view.

[0004] 2. Large field-of-view lenses exhibit significant distortion, which is detrimental to high-precision ranging.

[0005] 3. The overall structure has weak seismic resistance and insufficient imaging stability under vehicle vibration conditions.

[0006] 4. Limited long-range detection capability makes it difficult to achieve reliable initial target acquisition over a large area.

[0007] Therefore, there is a need for a trinocular vision measurement device that can simultaneously possess long-distance coarse measurement capabilities and short-distance fine measurement capabilities in the same device, and meet the requirements of stable imaging and high real-time processing capabilities under vehicle vibration environments. Utility Model Content

[0008] To address the aforementioned technical problems, this invention provides a tri-vision vision device for relative pose measurement between cooperative vehicles. This device is installed on the cooperative rendezvous side of the active vehicle and used in conjunction with a cooperative target on the cooperative rendezvous side of the passive vehicle. Through a three-camera coaxial arrangement, an FPGA+NPU dual-processing architecture, and a coarse / fine measurement dual-mode fusion design, this device achieves unified long-range target detection and close-range high-precision stereo measurement, improving the safety, stability, and real-time performance of the cooperative vehicle rendezvous process.

[0009] The technical solution adopted in this utility model is as follows: The tri-lens vision device for relative pose measurement between cooperative vehicles is installed on the cooperative rendezvous side of the active vehicle and includes: a tri-lens camera component (1), an image processing component (2), an external communication component (3), a power conversion component (4), and an integrated component (5). The tri-lens camera assembly (1) includes a first camera (11), a second camera (12), and a third camera (13) arranged along the same straight line, wherein: The first camera (11) and the second camera (12) have the same focal length and are set by a fixed baseline distance to form a precise three-dimensional measurement area; The field of view of the lens used by the third camera (13) is greater than the field of view corresponding to the precise stereo measurement area, and is used to form a coarse detection area covering the area.

[0010] The third camera (13) is positioned between the first camera (11) and the second camera (12), and the imaging field of view of the third camera (13) is larger than that of the first camera (11) and the second camera (12), which is used to achieve large-area coarse measurement; The first camera (11) and the second camera (12) use lenses with the same focal length and a maximum accurate ranging distance not less than the minimum measuring distance threshold to achieve high-precision stereo measurement. The minimum measuring distance threshold is determined by the binocular baseline length and the lens focal length, and is the minimum usable ranging distance of the system.

[0011] Each camera in the tri-lens camera assembly (1) includes an optical lens, a lens mount, an optical sensor chip, and an aviation plug-in electrical connector.

[0012] The image processing component includes a first image processor (21) and a second image processor (22), wherein, The first image processor (21) is electrically connected to each camera of the three-lens camera assembly (1). The first image processor (21) is an FPGA and includes: circuit units for synchronous triggering, video signal reception, video signal format conversion and data output. It sends synchronization signals to the three cameras through the RS-485 interface, receives video signals through the SDI interface and outputs processed image data to the second image processor 22 through the PCIe interface.

[0013] The second image processor (22) is electrically connected to the first image processor (21), and is connected to the first image processor (21) and the external communication component (3), respectively; the second image processor (22) is an embedded multi-core NPU processor, including an image target detection processing module and a stereo measurement module for binocular measurement, wherein: The image target detection and processing module includes an image computing circuit based on a convolution acceleration unit, which processes the passive vehicle or cooperative target image data acquired by the third camera 13 and outputs coarse pose parameters to the vehicle control system.

[0014] The stereo measurement module performs stereo matching processing based on the binocular image signals from the first and second cameras, and provides relatively accurate pose information to the vehicle control system.

[0015] The external communication component (3) is used to realize bidirectional communication between the image processing component (2) and the vehicle control system. It adopts a full-duplex CAN communication interface with a communication rate of 500 kbps and a standard frame format.

[0016] The power conversion component (4) converts the externally input 28 V DC power into 12 V and 5 V to power the tri-lens camera component, image processing component and external communication component.

[0017] The integrated component (5) includes a printed circuit board (PCB) and a housing, wherein the three-lens camera component, image processing component, external communication component and power conversion component are all mounted on the PCB; the PCB provides three independent SDI signal connections, three independent RS-485 synchronous trigger connections and one CAN communication connection; the PCB is fixed in the housing by bolts to form a shock-resistant integrated structure.

[0018] Compared with the prior art, the beneficial effects that this utility model can achieve are: 1. Multi-scale ranging capability with near and far distance fusion. By combining a large field-of-view third camera with a fixed baseline binocular camera, continuous coverage of long-range coarse measurement and short-range fine measurement is achieved, taking into account both initial acquisition and high-precision measurement requirements.

[0019] 2. High measurement accuracy and stability. The first and second cameras use the same focal length and an optimized stereo baseline, while the third camera's field of view covers the precision measurement area, guiding the target into the stereo imaging zone and improving the continuity and accuracy of ranging.

[0020] 3. Highly vibration-resistant integrated structural design. The tri-lens camera assembly, image processor, and communication circuitry are integrated into the same PCB and fixed within a rigid housing, significantly improving the imaging and measurement stability of the device under vehicle vibration environments.

[0021] 4. High real-time dual-processing architecture. The FPGA is responsible for image synchronization triggering and high-speed preprocessing, while the NPU is responsible for deep learning object detection and stereo matching calculation, enabling the system to have high frame rate and low latency real-time processing capabilities.

[0022] 5. Intelligent switching of measurement modes. The image processor can automatically switch between coarse and fine measurement modes according to the target area, without manual intervention, and improve the system's ability to adapt to different distances and orientation changes.

[0023] 6. Enhanced long-range detection capability. Combining the coarse-detection camera with a deep learning detection model can expand the initial detection range and improve the ability to identify long-range targets and resist occlusion.

[0024] 7. High system integration and convenient deployment. The device adopts an integrated installation structure and provides independent SDI, RS-485 and CAN communication interfaces, facilitating direct integration with vehicle systems. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this utility model, the accompanying drawings used in the description will be briefly introduced below. Obviously, the drawings in the following description are one embodiment of this utility model. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort: Figure 1 This is a schematic diagram of the electrical relationships of the trinocular vision device of this utility model.

[0026] Figure 2 This is a schematic diagram of the external appearance of the trinocular vision device of this utility model.

[0027] Figure 3 This is a schematic diagram of the field of view of the trinocular camera component of this utility model. Detailed Implementation

[0028] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present utility model.

[0029] like Figure 1 As shown, this utility model provides a trinocular vision device for measuring the relative pose between cooperative vehicles. The trinocular vision device for measuring the relative pose between cooperative vehicles is installed on the cooperative rendezvous side of the active vehicle and is used in conjunction with the cooperative target set on the cooperative rendezvous side of the passive vehicle. The device is characterized by including: a trinocular camera component (1), an image processing component, an external communication component (3), a power conversion component (4), and an integrated component (5).

[0030] The tri-lens camera assembly (1) includes a first camera (11), a second camera (12) and a third camera (13) for acquiring images of the measured environment. The first camera (11), the third camera (13) and the second camera (12) are arranged sequentially along the same straight line, wherein: the focal length of the first camera (11) and the second camera (12) are the same and are set by a fixed baseline distance to form a precise stereo measurement area; the field of view of the lens used by the third camera (13) is larger than the field of view corresponding to the precise stereo measurement area to form a coarse detection area covering the area.

[0031] The third camera (13) is positioned between the first camera (11) and the second camera (12), and the imaging field of view of the third camera (13) is larger than that of the first camera (11) and the second camera (12), which is used to achieve large-area coarse measurement; The first camera (11) and the second camera (12) use lenses with the same focal length and a maximum accurate distance measurement distance not less than the minimum measurement distance threshold to achieve high-precision stereo measurement.

[0032] In this embodiment, each camera in the tri-lens camera assembly (1) includes: an optical lens, a lens mount, an optical image sensor chip, and an aviation plug-in electrical connector, in order to meet the requirements of long-term stable operation under vehicle vibration environment.

[0033] like Figure 2 As shown, in this embodiment, the third camera (13) is positioned between the first camera (11) and the second camera (12), and all three are arranged along the same straight line inside the housing. Specifically, the distance Rab between the first camera (11) and the second camera (12) is the length of the stereo baseline, and Rab is 0.6 m, which can meet the requirement of a maximum accurate binocular ranging of not less than 15 m. The spacings Rac and Rcb between the third camera (13) and the two side cameras are designed to be equal to ensure structural symmetry and ease of installation.

[0034] In this embodiment, the third camera (13) can use a wide-angle lens with a field of view of not less than 110°, and its imaging range is larger than the precise stereo measurement area, which is used to form a coarse detection area; the coarse detection area at least covers or surrounds the precise stereo measurement area, so as to guide the target from the coarse measurement area to the fine measurement area, thereby realizing the imaging relationship of "large coarse measurement field of view and small fine measurement field of view".

[0035] Specifically, such as Figure 3As shown, the field of view of the first camera (11) is the area shown in A1–A–A2; the field of view of the second camera (12) is the area shown in B1–B–B2; and the field of view of the third camera (13) is the area shown in C1–C–C2. Among them, the fields of view of the first camera (11) and the second camera (12) overlap, namely the area shown in B1–D–A2. This area simultaneously meets the binocular imaging conditions and can obtain clear and complete stereo matching features. When the second image processor detects that the target falls into this overlapping area, it determines that the binocular imaging is effective and starts the stereo vision measurement model to calculate the precise spatial position and attitude of the target. This area corresponds to the precise stereo measurement area.

[0036] When the target is not located within the overlapping area of ​​B1–D–A2, the second image processor performs target detection based on the wide-angle image of the third camera (13). Since the target image has fewer pixels and lower resolution in this area, the relative position and orientation of the target can only be estimated by combining the target's prior size and image coordinates. Its measurement accuracy is lower than that of the precision measurement area, thus corresponding to the coarse detection area. The field of view C1–C–C2 of the third camera corresponds to the coarse detection area, which is larger than the precise stereo measurement area.

[0037] The image processing component includes a first image processor (21) and a second image processor (22), wherein, The first image processor (21) is an FPGA, including: circuit units for synchronous triggering, video signal reception, video signal format conversion and data output, sending synchronous signals to the camera through the RS-485 interface, receiving video signals through the SDI interface and outputting processed image data to the second image processor through the PCIe interface.

[0038] The second image processor (22) is an embedded multi-core NPU processor, including an image target detection processing module and a stereo measurement module for binocular measurement, wherein: The image target detection and processing module includes an image computing circuit based on a convolution acceleration unit, which processes the passive vehicle or cooperative target image data acquired by the third camera and outputs coarse pose parameters to the vehicle control system.

[0039] The stereo measurement module performs stereo matching processing based on the binocular image signals from the first and second cameras, and supplies relatively accurate pose information to the vehicle control system.

[0040] Specifically, in this embodiment, the first image processor (21) adopts a field-programmable gate array (FPGA), and sends synchronization trigger signals to the first camera (11), the second camera (12), and the third camera (13) through three independent RS-485 (Recommended Standard 485, i.e. 485 communication standard) interfaces respectively; and receives the video signal output by the three-lens camera component through three independent SDI (Serial Digital Interface, i.e. serial digital interface) interfaces and converts it into YUV422 format; and sends the preprocessed video data to the second image processor (22) through the PCIe interface.

[0041] The second image processor (22) adopts an embedded multi-core NPU processor, which can be a Rockchip RK3588 processor, and is connected to the FPGA and the external communication component (3). Coarse measurement stage: The second image processor processes the image captured by the third camera through its built-in image processing circuit and outputs parameters reflecting the approximate relative position of the target, which are used to guide the target into the fine measurement area; Precision measurement phase: After the target enters the precision measurement area, stereo matching and position and attitude calculation are performed based on the first and second cameras, and the precise relative pose and control commands are output to the vehicle control system.

[0042] Furthermore, in the coarse measurement stage, target detection can be performed on the image from the third camera to obtain a rough relative pose of the passive vehicle or its cooperative target on the rendezvous side; in the fine measurement stage, stereo feature extraction and geometric measurement can be performed on the binocular images from the first and second cameras to obtain a precise relative pose. The above coarse and fine measurement processing methods are exemplary and not limited thereto. The above image processing is implemented by hardware circuitry, and the specific processing methods are not within the protection scope of this utility model.

[0043] The external communication component (3) is used to realize bidirectional communication between the image processing component (2) and the vehicle control system, and adopts a full-duplex CAN communication interface with a communication rate of 500 kbps and a standard frame format. Through this interface, attitude measurement results and working status information can be sent to the vehicle control system, and mode commands or system configuration parameters issued by the vehicle control system can be received.

[0044] The power conversion component (4) converts the externally input 28V DC power into 12V and 5V to power the tri-lens camera component, image processing component, and communication component. In this embodiment, a DC-DC converter is used to achieve voltage conversion, ensuring long-term stable operation of the system.

[0045] The integrated component (5) includes a printed circuit board (PCB) and a housing, wherein the tri-lens camera assembly, image processing assembly, external communication assembly, and power conversion assembly are all mounted on the PCB; the PCB provides three independent SDI signal connections, three independent RS-485 synchronous trigger connections, and one CAN communication connection; the PCB is fixed to the housing with bolts to form a shock-resistant integrated structure. This integrated design gives the device good shock resistance, environmental adaptability, and system reliability.

[0046] In this embodiment, the integrated component (5) is laid out on a printed circuit board (PCB) according to the electrical connection relationship between the three-lens camera component (1), the image processing component (2), the external communication component (3), and the power conversion component (4). While meeting the device size and system heat dissipation requirements, the integrated space of the PCB is optimized, and a structural protective shell is designed for the PCB and its components. The PCB is rigidly connected to the shell via threaded fixing holes and screws, and the threaded fixing holes and screw positions are reinforced with adhesive to form an integrated unit. The shell is made of high-strength material, providing reliable mechanical support and protection in vehicle vibration environments.

[0047] The device's workflow is as follows: After the external power supply is input, a stable 12V and 5V DC power is provided through the power conversion component (4); The FPGA sends a synchronization trigger signal to the three cameras to achieve synchronized exposure; Three cameras simultaneously output SDI video, which is received and converted by the FPGA. Video data is transmitted to the NPU via PCIe; The NPU performs coarse testing and guides the system into the fine testing area for fine stereo matching and attitude calculation. The measurement results are sent to the vehicle control system via the CAN interface.

[0048] As mentioned above, through the coaxial arrangement of three cameras, the dual processing architecture of FPGA+NPU, and the fusion of coarse and fine measurement modes, this device can simultaneously achieve large-scale coarse-grained detection and small-scale high-precision three-dimensional measurement in the same system, making it suitable for scenarios such as vehicle collaborative rendezvous and docking guidance.

[0049] The above description is merely a specific embodiment of this utility model, but the protection scope of this utility model is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this utility model, and these modifications or substitutions should all be covered within the protection scope of this utility model. Therefore, the protection scope of this utility model should be determined by the scope of the claims.

Claims

1. A trinocular vision device for measuring relative pose between cooperative vehicles, installed on the cooperative rendezvous side of an active vehicle, characterized in that, include: The three-lens camera assembly (1) consists of a first camera (11), a third camera (13), and a second camera (12) arranged sequentially along the same straight line. The first camera (11) and the second camera (12) have the same focal length and are set by a fixed baseline distance to form a precise stereo measurement area. The third camera (13) has a field of view that is larger than the field of view corresponding to the precise stereo measurement area, and is used to form a rough detection area covering the area. The image processing component (2) includes a first image processor (21) and a second image processor (22), wherein: the first image processor (21) is electrically connected to each camera of the tri-lens camera component (1) and is used to receive synchronously acquired image data; the second image processor (22) is connected to the first image processor (21) and the external communication component (3) respectively. External communication component (3) is used to realize bidirectional communication between image processing component (2) and vehicle control system; The power conversion component (4) is used to convert the external input power and supply it to the tri-lens camera component (1), the image processing component (2) and the external communication component (3); The integrated component (5) includes a printed circuit board and a housing. The three-lens camera component (1), image processing component (2), external communication component (3) and power conversion component (4) are mounted on the printed circuit board. The printed circuit board is fixedly installed in the housing, forming an integrated structure.

2. The trinocular vision device for cooperative vehicle relative pose measurement according to claim 1, characterized in that, The third camera (13) is positioned between the first camera (11) and the second camera (12), and the imaging field of view of the third camera (13) is greater than that of the first camera (11) and the second camera (12), which is used to achieve large-area coarse measurement; the first camera (11) and the second camera (12) use lenses with the same focal length and the maximum accurate ranging distance is not less than the minimum measuring distance threshold, which is used to achieve high-precision stereo measurement; wherein, the minimum measuring distance threshold is determined by the binocular baseline length and the lens focal length, and is the minimum usable ranging distance of the system.

3. The trinocular vision device for cooperative vehicle relative pose measurement according to claim 2, characterized in that, The distance Rab between the first camera (11) and the second camera (12) is the length of the stereo baseline, and the spacings Rac and Rcb between the third camera (13) and the two side cameras are designed to be equal.

4. The trinocular vision device for cooperative vehicle relative pose measurement according to claim 3, characterized in that, Each camera in the tri-lens camera assembly (1) includes an optical lens, a lens mount, an optical image sensor chip, and an aviation plug-in electrical connector.

5. A trinocular vision device for cooperative vehicle-to-vehicle relative pose measurement according to claim 4, characterized in that, The third camera (13) may use a wide-angle lens with a field of view of not less than 110°.

6. A trinocular vision device for cooperative vehicle relative pose measurement according to claim 1, characterized in that, The first image processor (21) is an FPGA, including: circuit units for synchronous triggering, video signal reception, video signal format conversion and data output, sending synchronous signals to the camera through the RS-485 interface, receiving video signals through the SDI interface and outputting processed image data to the second image processor through the PCIe interface.

7. A trinocular vision device for measuring relative pose between collaborative vehicles according to claim 1, characterized in that, The second image processor (22) is an embedded multi-core NPU processor, including an image target detection processing module and a stereo measurement module for binocular measurement, wherein: The image target detection and processing module includes an image computing circuit based on a convolution acceleration unit, which processes the passive vehicle or cooperative target image data acquired by the third camera and outputs coarse pose parameters to the vehicle control system. The stereo measurement module performs stereo matching processing based on the binocular image signals from the first and second cameras, and provides relatively accurate pose information to the vehicle control system.

8. A trinocular vision device for cooperative vehicle-to-vehicle relative pose measurement according to claim 1, characterized in that, The external communication component (3) includes a full-duplex CAN communication interface with a communication rate of 500 kbps and a standard frame format.

9. A trinocular vision device for measuring relative pose between collaborative vehicles according to claim 1, characterized in that, The power conversion component (4) converts the externally input 28 V DC power into 12 V and 5 V DC power, and supplies power to the tri-lens camera component (1), image processing component (2) and external communication component (3) through the printed circuit board.

10. A trinocular vision device for cooperative vehicle-to-vehicle relative pose measurement according to claim 1, characterized in that, The printed circuit board provides three independent SDI signal connections, three independent RS-485 synchronous trigger signal connections, and one CAN signal connection.

Images