Visual fusion positioning terminal and vehicle-mounted positioning system

Abstract

The utility model relates to a visual fusion positioning terminal and a vehicle-mounted positioning system. The visual fusion positioning terminal comprises a visual processing chip, a Beidou RTK module, an IMU module and a video data access module, wherein the Beidou RTK module and the IMU module are connected with the visual processing chip, and the video data access module is connected with a plurality of cameras. The video data access module is used for accessing real-time video data of a plurality of cameras and inputting the real-time video data to the visual processing chip; the IMU module is internally provided with an array type MEMS sensor and is used for outputting spatial attitude data to the visual processing chip; the Beidou RTK module is used for outputting satellite positioning data to the visual processing chip; the visual processing chip fuses the spatial attitude data, the satellite positioning data and the real-time video data of each camera and outputs fused positioning data; according to the technical scheme, the positioning technical scheme of Beidou satellite navigation, inertial navigation and visual positioning is fused, accurate positioning can be provided for the engineering vehicle, and the positioning stability is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the positioning technical field, in particular to a visual fusion positioning terminal and a vehicle-mounted positioning system. BACKGROUND

[0002] At present, in various engineering vehicle applications, a visual positioning system is usually used for positioning. The visual positioning system is mainly a positioning navigation system based on image recognition and computer vision technology. The visual positioning system realizes accurate identification and positioning of position and direction by perceiving visual features in the environment, collecting real-time images for feature extraction, feature matching and pose estimation and the like.

[0003] However, although the visual positioning system can provide high-precision visual positioning function, in some special scenarios, the visual positioning system is prone to positioning loss during use, which affects application safety and production efficiency. CONTENT OF THE UTILITY MODEL

[0004] The application aims to solve one of the above technical defects, and provides a visual fusion positioning terminal to improve positioning accuracy.

[0005] The visual fusion positioning terminal comprises a visual processing chip, a Beidou RTK module, an IMU module and a video data access module connected with a plurality of cameras, respectively.

[0006] The video data access module is configured to input real-time video data of the plurality of cameras to the visual processing chip.

[0007] The IMU module is internally provided with an array type MEMS sensor, and is configured to output spatial attitude data to the visual processing chip.

[0008] The Beidou RTK module is configured to output satellite positioning data to the visual processing chip.

[0009] The visual processing chip fuses the spatial attitude data, the satellite positioning data and the real-time video data of the plurality of cameras to output fusion positioning data.

[0010] In an embodiment, the visual fusion positioning terminal further comprises a communication module configured to provide a communication function for the terminal.

[0011] In an embodiment, the communication module comprises a Beidou short message module, a mobile communication module, a WiFi module and a BT module.

[0012] In an embodiment, the visual fusion positioning terminal further comprises a peripheral interface module connected with the visual processing chip, and configured to provide an interface for connecting external devices for the terminal.

[0013] In one embodiment, the peripheral interface module includes one or more of a USB module, a CAN bus module, an RS485 module, a UART module, and a GPIO module.

[0014] In one embodiment, the visual fusion positioning terminal further comprises an audio amplification module and a display interface respectively connected to the visual processing chip; the audio amplification module is configured to connect to a loudspeaker, and the display interface is configured to connect to a display.

[0015] In one embodiment, the video data access module includes at least one AHD-to-MIPI module configured to convert real-time video data captured by multiple AHD cameras into MIPI protocol and input the converted video data into the visual processing chip.

[0016] In one embodiment, the visual fusion positioning terminal further comprises a storage device connected to the visual processing chip; the storage device includes an SD card or a SATA hard disk.

[0017] In one embodiment, the visual fusion positioning terminal further comprises a PMU unit connected to the visual processing chip and configured to connect to a power supply.

[0018] A vehicle-mounted positioning system includes the visual fusion positioning terminal and an external camera, a driver state monitoring camera, a vehicle driving state monitoring camera, and an in-vehicle monitoring camera connected to the visual fusion positioning terminal.

[0019] The technical solution of the present application has the following technical effects:

[0020] The visual processing chip fuses video data of multiple cameras to provide a visual positioning function, and based on a satellite navigation signal of a Beidou RTK module to perform real-time accurate positioning; in the case of loss of the satellite navigation signal, the IMU module can be used to perform accurate positioning for a short time. The technical solution fuses positioning technologies of Beidou satellite navigation, inertial navigation, and visual positioning, and can provide accurate positioning for engineering vehicles and improve the positioning stability of engineering vehicles.

[0021] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0023] Figure 1 FIG. 1 is a circuit structure diagram of a visual fusion positioning terminal according to one embodiment.

[0024] Figure 2 is a circuit structure diagram of a visual fusion positioning terminal of another embodiment;

[0025] Figure 3 is a camera layout schematic diagram of an example. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and letters in the drawing figures and the following description denote the same or like elements or components. The embodiments described below are examples of the present application, and are not intended to limit the present application.

[0027] As those skilled in the art will readily appreciate from the disclosure herein, ordinary reference to the singular form "a", "an", and "the" includes the plural reference unless the context clearly dictates otherwise. As is also apparent from the disclosure herein, ordinary reference to "including" or "comprising" means "including or comprising, but not limited to".

[0028] As those skilled in the art will readily appreciate from the disclosure herein, ordinary reference to the singular form "a", "an", and "the" includes the plural reference unless the context clearly dictates otherwise. As is also apparent from the disclosure herein, ordinary reference to "including" or "comprising" means "including or comprising, but not limited to".

[0029] Reference Figure 1 is shown, Figure 1It is a circuit structure diagram of a visual fusion positioning terminal in an embodiment, comprising: a visual processing chip, a Beidou RTK (Real-time kinematic) module, an IMU (Inertial Measurement Unit) module connected to the visual processing chip respectively, and a video data access module connected to multiple cameras. Among them, the video data access module is used to input the real-time video data of the multiple cameras to the visual processing chip; the IMU module is built-in with an array type MEMS (MEMS, Micro-Electro-Mechanical System) sensor, which is used to output spatial attitude data to the visual processing chip; the Beidou RTK module is used to output satellite positioning data to the visual processing chip; the visual processing chip fuses the spatial attitude data, the satellite positioning data and the real-time video data of each multi-camera to output fusion positioning data; and the Beidou RTK module and the IMU module are connected to the visual processing chip through a UART (Universal Asynchronous Receiver / Transmitter) interface respectively.

[0030] The scheme of the above embodiment provides visual positioning function by using the video data of multiple cameras by the visual processing chip, and at the same time fuses satellite navigation signals based on the Beidou RTK module to realize real-time accurate positioning. In the case of loss of satellite navigation signals, accurate positioning in a short time can also be realized through the IMU module. The positioning terminal fuses the positioning technology schemes of Beidou satellite navigation, inertial navigation and visual positioning, and can realize driver state monitoring, lane deviation, collision avoidance, pedestrian protection and other driving assistance functions, as well as vehicle driving attitude perception function, thereby providing accurate positioning for engineering vehicles and improving the positioning stability of engineering vehicles.

[0031] Illustratively, the visual processing chip can adopt an AI vision processor, which can be equipped with a CPU and a GPU, and is built-in with an NPU. The CPU can be an eight-core CPU equipped with four-core A76 and four-core A55, the GPU can be an ARM G610MP4 GPU, and the NPU can be a 6TOPs algorithm NPU. Thus, the visual processing chip has the characteristics of high algorithm, low power consumption, super multimedia, rich data interface, etc.

[0032] Illustratively, the Beidou RTK module can support positioning and orientation functions, support BDS, GPS, GLONASS, Galileo, QZSS, IRNSS satellite system signal tracking, support single Beidou operation, be configured with anti-narrowband interference and anti-continuous wave interference technology, and adopt miniaturized design. Thus, full-system, multi-frequency point and high-precision RTK positioning can be realized.

[0033] Reference Figure 2 as shown, Figure 2 is a circuit structure diagram of a visual fusion positioning terminal of another embodiment; in order to make the technical solutions of the present application clearer, the following will be combined with the accompanying Figure 2 to set forth more detailed embodiments.

[0034] In one embodiment, as Figure 2 shown, the visual fusion positioning terminal of the present application can also include a communication module connected to the visual processing chip, for realizing the communication function of the terminal with the outside; for example, the communication module can include: a Beidou short message module, a mobile communication module, a WiFi module and a BT module.

[0035] Among them, the Beidou short message module realizes the short message service function based on the Beidou system, the mobile communication module provides the 4G / 5G mobile network communication function, the WiFi module can provide the WiFi communication function, and the BT module can provide the short distance Bluetooth wireless communication function; for example, the Beidou short message module, the WiFi module and the BT module are connected to the visual processing chip through the UART interface, and the mobile communication module is connected to the visual processing chip through the USB interface.

[0036] In one embodiment, as Figure 2 shown, the visual fusion positioning terminal of the present application can also include a peripheral interface module connected to the visual processing chip, for providing the terminal with an interface for connecting external devices; for example, the peripheral interface module can include a USB module, a CAN bus module, a RS485 module, a UART module, a GPIO module, etc.; can respectively provide the external device function connected to the corresponding interface; for example, the USB module is connected to the visual processing chip through the USB interface, the CAN bus module, the RS485 module and the UART module are connected to the visual processing chip through the UART interface, and the GPIO module is connected to the visual processing chip through the GPIO interface.

[0037] In one embodiment, as Figure 2 shown, the visual fusion positioning terminal of the present application can also include a storage device connected to the visual processing chip; wherein the storage device includes an SD card or a SATA hard disk, for example, the SD card is connected to the visual processing chip through the SDIO interface via the SD card slot, and the SATA hard disk is connected to the visual processing chip through the SATA interface; for the parameters of the storage device, the storage space of the terminal can be configured to be not less than 8GB, the running memory is not less than 2GB, and the SD card slot supporting plugging, thereby providing various storage functions for the positioning terminal, such as providing large-capacity monitoring video storage function.

[0038] In one embodiment, as Figure 2As shown, the visual fusion positioning terminal of the present application can further include an audio amplification module and a display interface connected to the visual processing chip respectively; for example, the display interface is an HDMI (High Definition Multimedia Interface, high-definition multimedia interface) interface or an LDVS (Low Voltage Differential Signaling, low voltage differential signaling) interface, and the audio amplification module is connected to the visual processing chip through an I2S interface; wherein the audio amplification module is used to connect a loudspeaker, and the display interface is used to connect a display, thereby realizing audio and video output functions.

[0039] In one embodiment, as shown, Figure 2 The visual fusion positioning terminal of the present application further includes a PMU (Power Management Unit, power management unit) unit connected to the visual processing chip, which is used to connect a power supply; for example, a vehicle-mounted power supply can be used for power supply, and the power supply voltage of the positioning terminal is 12V, and the PMU unit can be converted into 5V / 4V / 3.3V / 1.8V and other voltages for use by various components.

[0040] In one embodiment, as shown, Figure 2 For the video data access module, it can include at least one AHD (Analog High Definition, analog high-definition) to MIPI (Mobile Industry Processor Interface, mobile industry processor interface) module, which is used to convert the real-time video data shot by multiple AHD cameras into MIPI protocol and input into the visual processing chip; for example, through the AHD to MIPI module, the real-time video data of multiple high-definition cameras can be accessed into the visual processing chip, so as to obtain positioning information based on visual positioning technology.

[0041] The embodiments of the vehicle-mounted positioning system are described below.

[0042] The vehicle-mounted positioning system of the present embodiment includes the visual fusion positioning terminal of any of the preceding embodiments, and a vehicle exterior camera, a driver state monitoring camera, a vehicle driving state monitoring camera and an in-vehicle monitoring camera connected to the visual fusion positioning terminal; preferably, each of the above cameras uses an AHD camera. Referring to Figure 3 As shown, Figure 3A schematic diagram of an example camera layout, wherein the out-of-vehicle camera X1 is placed outside the vehicle to monitor the out-of-vehicle state in real time, the driver state monitoring camera X2 is aimed at the driver to monitor the driver state in real time, the vehicle driving state monitoring camera X3 is placed inside the vehicle to monitor the driver's platform in real time, and the in-vehicle monitoring camera X4 is placed inside the vehicle to shoot the in-vehicle environment and monitor the in-vehicle environment. Further, the positioning system can also be configured with an IPS (In-Plane Switching) display and a speaker, wherein the IPS display is connected to the visual processing chip through the display interface, and the speaker is connected to the audio amplification module; the display can display various state information and real-time video images, etc., and the speaker can perform voice warning, etc.

[0043] For example, the technical parameters of the out-of-vehicle camera can be as follows: the resolution is not less than 1280*720, the pixel is not less than 720p, the recording frame rate is not less than 25fps, and the camera angle is not less than 200° horizontally, 210° diagonally, and 140° vertically. Meanwhile, various image combination modes, image calibration splicing, vehicle signal automatic switching display screen, and other holographic image functions are configured.

[0044] For example, the technical parameters of the driver state monitoring camera can be as follows: the resolution is not less than 1280*720, the pixel is not less than 720p, and the driver state monitoring warning functions such as smoking warning, phone warning, unfastened seat belt warning, distraction driving warning, fatigue driving warning, and intentional lens obstruction warning are configured.

[0045] For example, the technical parameters of the vehicle driving state monitoring camera can be as follows: the resolution is not less than 1280*720, the pixel is not less than 720p, and the functions such as lane deviation warning, front collision warning, safe distance reminder, pedestrian collision warning, and front vehicle departure reminder are configured, as well as the functions such as daytime and nighttime WDR high-definition recording, front and in-vehicle monitoring dual-channel recording online real-time video, etc.

[0046] For example, the technical parameters of the in-vehicle monitoring camera can be as follows: the resolution is not less than 1280*720, the pixel is not less than 720p, and the recording frame rate is not less than 25fps.

[0047] For example, the technical parameters of the IPS display can be as follows: the screen size is not less than 7 inches, the screen resolution is not less than 1920X720 pixels, the screen viewing angle is greater than 170° vertically and horizontally, a capacitive touch screen is configured, and multi-point touch and touch gesture functions are set.

[0048] The vehicle-mounted positioning system described above combines the positioning capabilities of Beidou RTK high-precision positioning, inertial navigation positioning and visual positioning, and can be used for state monitoring and auxiliary driving (lane deviation detection, lane automatic cruise, collision pre-warning, pedestrian detection, etc.) functions of the driver of the engineering vehicle. Based on the storage device, it can also provide a large-capacity monitoring video storage function, and through the communication module, it can also realize a one-key alarm function and a real-time uploading function of alarm information.

[0049] The above only describes some embodiments of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A visual-inertial positioning terminal, characterized by, Comprise: A visual processing chip, and a Beidou RTK module, an IMU module and a video data access module connected to the visual processing chip respectively; The video data access module is used for accessing real-time video data of multiple cameras to the visual processing chip; The IMU module is built-in with an array MEMS sensor, which is used for outputting spatial attitude data to the visual processing chip; The Beidou RTK module is used for outputting satellite positioning data to the visual processing chip; The visual processing chip fuses the spatial attitude data, satellite positioning data and real-time video data of each multi-camera to output fused positioning data.

2. The visual fusion positioning terminal according to claim 1, c h a r a c t e r i z e d by Also include: A communication module for providing communication function for the terminal.

3. The visual fusion positioning terminal according to claim 2, wherein The communication module includes a Beidou short message module, a mobile communication module, a WiFi module and a BT module.

4. The visual fusion positioning terminal according to claim 1, wherein Also include: A peripheral interface module connected to the visual processing chip, which is used for providing interface for connecting external devices for the terminal.

5. The visual fusion positioning terminal according to claim 4, characterized in that, The peripheral interface module includes one or more of USB module, CAN bus module, RS485 module, UART module and GPIO module.

6. The visual fusion positioning terminal according to claim 1, wherein Also include: An audio amplification module and a display interface connected to the visual processing chip respectively; wherein the audio amplification module is used for connecting a loudspeaker, and the display interface is used for connecting a display.

7. The visual fusion positioning terminal according to claim 1, wherein The video data access module includes at least one AHD to MIPI module, which is used for converting real-time video data of multiple AHD cameras into MIPI protocol and inputting to the visual processing chip.

8. The visual fusion positioning terminal according to claim 1, wherein Also include: A storage device connected to the visual processing chip; wherein the storage device includes an SD card or a SATA hard disk.

9. The visual fusion positioning terminal according to claim 1, wherein Also include: A PMU unit connected to the visual processing chip, which is used for connecting a power supply.

10. A vehicle location system, characterized by Comprise: The visual fusion positioning terminal of any one of claims 1-9, and an outside camera, a driver state monitoring camera, a vehicle driving state monitoring camera and an in-vehicle monitoring camera connected to the visual fusion positioning terminal.

Images