Automatic driving truck sensor configuration system applied to port

Abstract

The invention provides an automatic driving truck sensor configuration system applied to a port. The automatic driving truck comprises a tractor and a trailer, and comprises an ultra-long-distance sensing module which comprises a first group of sensing cameras and a first group of sensing radars and is used for sensing long-distance obstacles of the automatic driving truck; the 360-degree panoramic sensing module comprises a second group of sensing cameras and a second group of sensing radars and is used for sensing obstacles around the automatic driving truck by 360 degrees; and the close-range blind compensation sensing module comprises a third group of sensing cameras and a third group of sensing radars and is used for sensing obstacles which cannot be detected due to shielding of the tractor and the trailer, and the pitch angles of at least part of the sensing cameras in the second group of sensing cameras and at least part of the sensing radars in the second group of sensing radars are adjustable. According to the scheme, the problem that in the prior art, a setting scheme for a sensor of an automatic driving truck is difficult to support implementation of some functions of the automatic driving truck in a port scene is solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to an autonomous driving truck sensor configuration system and an autonomous driving truck used in a port. Background Art

[0002] In order to realize the operation of self-driving trucks, it is generally necessary to install sensors at corresponding positions of the self-driving trucks, such as perception radars, perception cameras, etc. Generally, perception radars are set at the front, sides, and rear of the truck. This setting method cannot perceive some blind spots and cannot support the realization of some functions of self-driving trucks in port scenarios. For example, it cannot support high-position target detection tasks, such as container vertical loading and unloading status detection, suspended mechanical equipment detection, etc.

[0003] For example, for a truck towing a trailer in a port scenario, the trailer itself obstructs the view and creates a blind spot directly behind the trailer. The layout of sensors in the prior art makes it difficult to detect obstacles directly behind the vehicle in a reversing scenario. Summary of the invention

[0004] The main purpose of this application is to provide a sensor configuration system and an autonomous driving truck for use in ports, so as to at least solve the problem that the sensor setting scheme for autonomous driving trucks in the prior art is difficult to support the realization of some functions of autonomous driving trucks in port scenarios.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a sensor configuration system for an autonomous driving truck applied to a port is provided, wherein the autonomous driving truck includes a tractor and a trailer, and includes: an ultra-long-range perception module, including a first group of perception cameras and a first group of perception radars, for sensing obstacles at a long distance of the autonomous driving truck; a 360-degree panoramic perception module, including a second group of perception cameras and a second group of perception radars, for sensing obstacles around the autonomous driving truck in 360 degrees; a close-range blind spot perception module, including a third group of perception cameras and a third group of perception radars, for sensing obstacles that cannot be detected due to obstruction by the tractor and the trailer, and the pitch angles of at least some of the second group of perception cameras and at least some of the second group of perception radars are adjustable.

[0006] Optionally, the first group of perception cameras includes a first camera and a second camera, and the first group of perception radars includes a first millimeter-wave radar, wherein the first camera and the second camera are both installed inside the cockpit and fixed behind the front windshield, and the first millimeter-wave radar is installed in the center of the front bumper or near the center.

[0007] Optionally, the second group of perception cameras includes a third camera, a fourth camera and a fifth camera, and the second group of perception radars includes a first laser radar, a second laser radar, a second millimeter-wave radar and a third millimeter-wave radar, wherein the third camera is installed inside the cockpit and fixed behind the front windshield, the fourth camera, the fifth camera, the first laser radar and the second laser radar are installed on both sides of the roof sensor bracket, and the second millimeter-wave radar and the third millimeter-wave radar are respectively installed on the left and right side enclosures and facing the rear side.

[0008] Optionally, the third group of perception cameras includes a sixth camera, a seventh camera, an eighth camera, a ninth camera and a tenth camera, and the third group of perception radars includes a third laser radar, a fourth laser radar, a fifth laser radar and a sixth laser radar, wherein the sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera are respectively installed on the front bumper, the right side wall, the rear wall, the left side wall and the rear bumper of the trailer, and the third laser radar, the fourth laser radar, the fifth laser radar and the sixth laser radar are respectively installed on the front bumper, the right side wall, the rear wall and the left side wall.

[0009] Optionally, the sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera are all fisheye cameras.

[0010] Optionally, the system further includes an operator and a gateway, the operator communicates with the gateway, the perception cameras in the ultra-long-range perception module and the 360-degree panoramic perception module are connected to the gateway via GMSL2, some perception cameras in the close-range blindness compensation perception module are connected to the gateway via GMSL2, and the perception radars in the ultra-long-range perception module, the 360-degree panoramic perception module and the close-range blindness compensation perception module are connected to the gateway via Ethernet.

[0011] Optionally, the system also includes a repeater, which communicates with the gateway. The third group of perception cameras of the close-range blind spot perception module includes a perception camera installed on the rear bumper of the trailer. The perception camera installed on the rear bumper of the trailer is connected to the repeater via GMSL2.

[0012] Optionally, the close-range blind spot perception module includes a perception camera installed on the rear enclosure and a laser radar installed on the rear enclosure, and the operator receives image data collected by the perception camera installed on the rear enclosure and laser point cloud data collected by the laser radar installed on the rear enclosure, and determines the loading and unloading status of the cargo based on the image data and the laser point cloud data.

[0013] Optionally, the operator determines the loading and unloading status of the cargo according to the image data and the laser point cloud data, including: pre-processing the image data and the laser point cloud data to intercept the image data and the laser point cloud data whose height is greater than a threshold, so as to obtain intercepted image data and intercepted laser point cloud data; processing the intercepted image data using a neural network algorithm to extract container features or quay crane spreader features in the image, and performing feature fusion of the container features or the quay crane spreader features with the intercepted laser point cloud data to obtain fused features; determining the three-dimensional position and boundary information of the container or the quay crane spreader according to the fused features, and calculating the three-dimensional speed information of the container or the quay crane spreader in combination with historical data; determining the loading and unloading status of the cargo according to the three-dimensional position, boundary information, and three-dimensional speed information of the container or the quay crane spreader.

[0014] According to another aspect of the present application, there are provided an autonomous driving truck for use in a port, any one of the sensor configuration systems for the autonomous driving truck for use in a port, and an autonomous driving truck body.

[0015] Applying the technical solution of the present application, the self-driving truck includes a tractor and a trailer, including: an ultra-long-range perception module, including a first group of perception cameras and a first group of perception radars, for sensing long-range obstacles of the self-driving truck; a 360-degree panoramic perception module, including a second group of perception cameras and a second group of perception radars, for sensing obstacles around the self-driving truck in 360 degrees; a close-range blind spot perception module, including a third group of perception cameras and a third group of perception radars, for sensing obstacles that cannot be detected due to obstruction by the tractor and the trailer, and the pitch angles of at least some of the second group of perception cameras and at least some of the second group of perception radars are adjustable. In this solution, an ultra-long-range perception module, a 360-degree panoramic perception module and a close-range blindness-compensation perception module are set up to realize the perception of close-range and long-range obstacles of the self-driving truck. In addition, due to the setting of the close-range blindness-compensation perception module, obstacles in the blind spots can also be perceived, and the pitch angle is adjustable to meet the needs of high-position target detection, thereby solving the problem that the existing technology of setting sensors for self-driving trucks is difficult to support the realization of some functions of self-driving trucks in port scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A schematic diagram of a sensor configuration system for an autonomous driving truck applied to a port according to an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram of sensor deployment positions of a sensor configuration system for an autonomous driving truck applied to a port according to an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram of sensor electrical connections of a sensor configuration system for an autonomous driving truck applied to a port according to an embodiment of the present application is shown;

[0020] Figure 4 A schematic diagram of the field of view coverage of an ultra-long-range perception module sensor of an autonomous driving truck sensor configuration system applied to a port according to an embodiment of the present application is shown;

[0021] Figure 5 A schematic diagram of the sensor field of view coverage of a 360-degree panoramic perception module of a sensor configuration system for an autonomous driving truck applied to a port according to an embodiment of the present application is shown;

[0022] Figure 6 A schematic diagram of the sensor field of view coverage of a close-range blind spot perception module of an autonomous driving truck sensor configuration system applied to a port according to an embodiment of the present application is shown.

[0023] The above drawings include the following reference numerals:

[0024] 10. Ultra-long-range perception module; 11. The first group of perception cameras; 110. The first camera; 111. The second camera; 12. The first group of perception radars; 120. The first millimeter-wave radar; 20. 360-degree panoramic perception module; 21. The second group of perception cameras; 210. The third camera; 211. The fourth camera; 212. The fifth camera; 22. The second group of perception radars; 220. The first laser radar; 221. The second laser radar; 222. The second millimeter-wave radar; 223. The third millimeter-wave radar; 30. Close-range blind spot perception module; 31. The third group of perception cameras; 310. The sixth camera; 311. The seventh camera; 312. The eighth camera; 313. The ninth camera; 314. The tenth camera; 32. The third group of perception radars; 320. The third laser radar; 321. The fourth laser radar; 322. The fifth laser radar; 323. The sixth laser radar. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] As introduced in the background technology, in the prior art, perception radars are generally set at the front end, both sides and rear end of the truck. This setting method cannot perceive some blind spots and cannot support the realization of some functions of autonomous driving trucks in port scenarios. In order to solve the problem that the setting scheme of sensors for autonomous driving trucks is difficult to support the realization of some functions of autonomous driving trucks in port scenarios, the embodiments of the present application provide a sensor configuration system and an autonomous driving truck for use in ports.

[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0030] The schematic diagram of the sensor configuration system for an autonomous driving truck applied to a port provided in the embodiment of the present application is as follows: Figure 1 As shown, the self-driving truck includes a tractor and a trailer, including an ultra-long-range perception module 10, a 360-degree panoramic perception module 20, and a short-range blind spot perception module 30, wherein:

[0031] The ultra-long-range sensing module 10 includes a first set of sensing cameras 11 and a first set of sensing radars 12, which are used to sense long-range obstacles of the above-mentioned self-driving truck;

[0032] A 360-degree panoramic perception module 20, including a second set of perception cameras 21 and a second set of perception radars 22, for sensing obstacles around the autonomous driving truck at 360 degrees;

[0033] The close-range blind spot sensing module 30 includes a third group of sensing cameras 31 and a third group of sensing radars 32, which are used to sense obstacles that cannot be detected due to obstruction by the tractor and the trailer, and the pitch angles of at least some of the sensing cameras in the second group of sensing cameras 21 and at least some of the sensing radars in the second group of sensing radars 22 are adjustable.

[0034] The ultra-long-range perception module 10 is mainly used to detect obstacles in the distance, providing a long-range field of view for the self-driving truck, so that it can perceive and plan the path in advance. The "long-range" here can refer to the long distance in front that the truck can cover when driving. The 360-degree panoramic perception module 20 provides an all-round field of view to ensure that the truck can perceive obstacles within a 360-degree range around it. This perception module is particularly important for the port environment because the port scene may contain potential sources of danger from all directions, such as other vehicles, pedestrians, and cargo. The short-range blind spot perception module 30 is mainly used to detect short-range obstacles that cannot be detected by other modules due to the truck's own occlusion. In particular, the pitch angles of some perception cameras in the second group of perception cameras and some radars in the second group of perception radars are adjustable, which solves the problem of rear blind spots when the truck is reversing, as well as the detection of high-level targets such as container loading and unloading equipment in ports.

[0035] like Figure 1 As shown, the first group of perception cameras 11 includes a first camera 110 and a second camera 111, and the first group of perception radars 12 includes a first millimeter wave radar 120, wherein, as shown in FIG. Figure 2 As shown, the first camera 110 and the second camera 111 are both installed inside the cockpit and fixed behind the front windshield, and the first millimeter-wave radar 120 is installed in the center of the front bumper or near the center.

[0036] In addition to setting up two telephoto cameras, namely the first camera and the second camera, the ultra-long-range sensing module can also be set up with three telephoto cameras or four telephoto cameras. If cost factors are taken into consideration, only two telephoto cameras can be set up.

[0037] Specifically, the models that may be selected for the telephoto camera include but are not limited to the following:

[0038] Sony IMX492: This is a high-performance CMOS image sensor with high resolution and fast response time, suitable for long-range detection.

[0039] Bosch SDCR8: This camera has a wide dynamic range of up to 120dB and can work in various lighting conditions while supporting high-speed data transmission.

[0040] FLIR Boson 640: This is a thermal imaging camera that provides clear images in low-light or no-light environments, making it ideal for nighttime port operations.

[0041] Specifically, the first millimeter-wave radar is installed in the center of the front bumper or near the center because the center of the front bumper can ensure the maximum coverage of the radar, especially for the detection of obstacles in front. The central position can reduce the shielding of the radar signal by the vehicle in front or other obstacles, and improve the detection efficiency and accuracy of the radar. In addition, installing the radar in the center of the front bumper or near the center can effectively reduce the shielding of the radar signal by the vehicle's own structure and reduce the blind spot range, especially in the complex environment of the port, which can better detect low obstacles in front or suddenly appearing pedestrians and vehicles. The installation in the central position can reduce the impact of bumps on the radar during vehicle driving and improve the stability of radar data.

[0042] like Figure 1 As shown, the second group of perception cameras 21 includes a third camera 210, a fourth camera 211 and a fifth camera 212, and the second group of perception radars 22 includes a first laser radar 220, a second laser radar 221, a second millimeter-wave radar 222 and a third millimeter-wave radar 223, wherein, as shown in FIG. Figure 2 As shown, the third camera 210 is installed inside the cockpit and fixed behind the front windshield, the fourth camera 211, the fifth camera 212, the first laser radar 220 and the second laser radar 221 are installed on both sides of the roof sensor bracket, and the second millimeter-wave radar 222 and the third millimeter-wave radar 223 are respectively installed on the left and right side enclosures and facing the side rear.

[0043] Specifically, the first and second laser radars are installed on both sides of the roof sensor bracket. They can not only provide information in front, but also assist the fourth and fifth cameras to jointly build a three-dimensional model of the vehicle's surrounding environment, especially the detection and identification of obstacles on the side and rear. Millimeter-wave radars have good penetration and working stability in adverse weather conditions. The second and third millimeter-wave radars are installed on the left and right sides of the vehicle and face the side and rear. This layout can effectively detect moving obstacles on the side and rear, such as other vehicles or pedestrians. Especially when the self-driving truck needs to perform side or reverse operations, obstacle detection on the side and rear is crucial.

[0044] like Figure 1 As shown, the third group of perception cameras 31 includes a sixth camera 310, a seventh camera 311, an eighth camera 312, a ninth camera 313 and a tenth camera 314, and the third group of perception radars 32 includes a third laser radar 320, a fourth laser radar 321, a fifth laser radar 322 and a sixth laser radar 323, wherein, as shown Figure 2As shown, the sixth camera 310, the seventh camera 311, the eighth camera 312, the ninth camera 313 and the tenth camera 314 are respectively installed on the front bumper, the right side wall, the rear wall, the left side wall and the rear bumper of the trailer, and the third laser radar 320, the fourth laser radar 321, the fifth laser radar 322 and the sixth laser radar 323 are respectively installed on the above-mentioned front bumper, the above-mentioned right side wall, the above-mentioned rear wall and the above-mentioned left side wall.

[0045] Specifically, the third group of perception cameras includes five cameras, namely the sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera. The sixth camera is installed at the front bumper to capture wide-angle images in front of the vehicle. This position can ensure the coverage of the field of view directly in front of the vehicle and provide guarantee for the detection of obstacles in front. The seventh camera is installed on the right side to capture wide-angle images on the right side of the vehicle. This position can ensure the monitoring of the right side of the vehicle, especially the field of view requirements when the vehicle is turning or approaching obstacles. The eighth camera is installed on the rear side to capture wide-angle images behind the vehicle. This position is crucial for monitoring the rear environment, supporting reversing operations or detecting rear obstacles. The ninth camera is installed on the left side to capture wide-angle images on the left side of the vehicle. Similar to the right side camera, it is used to ensure the monitoring of the left side of the vehicle. The tenth camera is installed on the rear bumper of the trailer. The camera at this position is to eliminate the rear field of view blind spot in the towing state, ensuring complete field of view coverage even behind the trailer, and supporting automatic driving operations in the trailer state. The five cameras are configured to provide wide-angle image data of the environment surrounding the autonomous truck, especially to solve the problem of close-range blind spots, to support autonomous driving functions in complex port environments.

[0046] Specifically, the third group of perception radars includes four laser radars, namely the third laser radar, the fourth laser radar, the fifth laser radar and the sixth laser radar. The third laser radar is installed on the front bumper for high-precision point cloud data collection of the front environment. The fourth laser radar is installed on the right side for point cloud data collection of the right side of the vehicle. The fifth laser radar is installed on the rear side for point cloud data collection of the rear environment of the vehicle, and together with the eighth camera, provides details of the rear field of view. The sixth laser radar is installed on the left side for point cloud data collection of the left side of the vehicle. The above four laser radars are configured on the front bumper, right side, rear side and left side of the truck to provide high-precision point cloud data for distance measurement and obstacle detection.

[0047] Furthermore, the sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera are all fisheye cameras. Fisheye cameras can provide a wider viewing angle than ordinary cameras, and can capture a wider range of environmental images around the vehicle, providing protection for detecting obstacles, pedestrians and other vehicles close to the vehicle, especially in complex environments such as ports, which can effectively reduce blind spots and improve safety. In addition, due to the wide field of view, fisheye cameras can collect a large amount of environmental information in the same frame of the image. In port scenes, fisheye cameras can simultaneously monitor dynamics in multiple directions, such as container loading and unloading, other vehicles or pedestrian movements. The use of fisheye cameras significantly enhances the perception capabilities of autonomous driving trucks in port environments, especially in the detection of blind spots, which helps to improve the efficiency and safety of truck operations in ports.

[0048] like Figure 3 As shown, the system further includes a computing unit and a gateway. The computing unit communicates with the gateway. The perception cameras in the ultra-long-range perception module and the 360-degree panoramic perception module are connected to the gateway via GMSL2. Some perception cameras in the close-range blindness compensation perception module are connected to the gateway via GMSL2. The perception radars in the ultra-long-range perception module, the 360-degree panoramic perception module and the close-range blindness compensation perception module are connected to the gateway via Ethernet (ETH).

[0049] Among them, GMSL2 (Gigabit Multimedia Serial Link 2, high-speed serial communication interface): mainly used for data transmission between cameras, sensors and other multimedia devices in the automotive industry. GMSL2 is a standardized data transmission protocol that can transmit data at gigabit speeds, with low latency, high reliability and good noise suppression capabilities, making it very suitable for high-speed data transmission in complex environments inside vehicles. The GMSL2 interface reduces the number and complexity of wiring harnesses by converting parallel data streams into serial data streams for transmission, reducing the cost and difficulty of wiring inside the vehicle. At the same time, GMSL2 technology also supports long-distance data transmission, which allows sensors and devices in the vehicle to be flexibly distributed in different locations without worrying about signal attenuation or integrity issues in data transmission. In autonomous vehicles, the GMSL2 interface can be used to connect sensors such as high-resolution cameras and lidars, and transmit large amounts of data captured by these sensors to the vehicle's computing platform for processing and analysis. Since autonomous vehicles need to perceive the surrounding environment in real time and accurately, the high-speed transmission capability of GMSL2 is particularly important, ensuring that the vehicle can obtain and process environmental information in a timely manner when driving at high speeds, thereby improving the safety and reliability of autonomous driving.

[0050] The features of GMSL2 in the field of data transmission include the following:

[0051] 1. High-speed data transmission: GMSL2 provides a transmission rate of up to 1.5Gbps, suitable for high-resolution video transmission and other high-speed data communication needs.

[0052] 2. Serial communication: GMSL2 uses serial communication to achieve high-speed data transmission through a single cable, which helps to simplify system design and reduce cost and complexity.

[0053] 3. Long-distance transmission: GMSL2 supports a transmission distance of up to 20 meters, which is suitable for long-distance data communication in automobiles.

[0054] 4. Low latency: GMSL2 has lower transmission latency, ensuring the reliability of real-time data transmission.

[0055] 5. Compatibility: GMSL2 supports a variety of camera and sensor types, as well as multiple data formats, improving the flexibility and scalability of the system.

[0056] 6. Anti-interference performance: GMSL2 has strong anti-interference ability and is suitable for electromagnetic interference in automotive environments.

[0057] 7. Standardization: GMSL2 is a standardized interface that helps achieve interoperability between different manufacturers and devices.

[0058] In short, GMSL2 is a high-speed, low-latency, long-distance, and anti-interference serial communication interface in the field of data transmission, and is widely used in multimedia device data transmission in the automotive industry.

[0059] Of course, in addition to GMSL2, wireless transmission technologies such as Wi-Fi or 5G can also be used, which are convenient and flexible. LVDS (Low Voltage Differential Signaling) can also be used. LVDS is a technology for short-distance high-speed data transmission, commonly used in display interfaces. Different data transmission methods can be selected according to the specific needs and design of the vehicle.

[0060] Among them, Ethernet (ETH) is a common local area network (LAN) technology used for data transmission between devices in a local area network. The working principle of Ethernet is mainly based on the Carrier Sense Multiple Access with Collision Detection (CSMA / CD) mechanism. In CSMA / CD, the device listens to the network before sending data to check whether other devices are transmitting data. If the network is idle, the device starts to transmit data; if the network is busy, the device will wait until the network is idle. During the transmission process, if two devices start transmitting at the same time, it will cause a data conflict. At this time, the conflicting device will stop transmitting and wait for a random time before trying to resend.

[0061] Ethernet technology has undergone several upgrades to support higher data transmission rates. From the initial 10Mbps (10BASE-T) standard, it has developed to 100Mbps (100BASE-TX), 1Gbps (1000BASE-T), 10Gbps, 40Gbps, 100Gbps, and even higher rates. These upgrades are mainly achieved by improving physical layer technologies such as signal coding, modulation, and transmission distance. Ethernet is widely used in various fields, especially in autonomous driving technology, where Ethernet is used to connect various sensors and computing units to achieve high-speed data transmission and processing.

[0062] Of course, in addition to ETH, sensor data transmission in self-driving trucks can also use a variety of communication technologies, and the specific choice depends on factors such as data transmission rate, transmission distance, anti-interference ability, cost and power consumption. In this embodiment, optical fiber communication can also be used. Optical fiber communication has high bandwidth, low latency and long-distance transmission capability, which is suitable for high data transmission rate and long-distance communication requirements. For scenes that require high-speed transmission of large amounts of data, optical fiber communication is a more ideal choice. In this embodiment, wireless communications, such as Wi-Fi, Bluetooth, ZigBee, etc., can also be used. Among them, Wi-Fi can provide a higher data transmission rate and is suitable for data transmission within a short distance; while Bluetooth and ZigBee have advantages in low power consumption and short-distance data transmission.

[0063] Specifically, Figure 3As shown, the system includes three gateways, namely, a first gateway, a second gateway, and a third gateway. The first laser radar 220, the second laser radar 221, the third laser radar 320, the fourth laser radar 321, the fifth laser radar 322, and the sixth laser radar 323 are connected to the first gateway via Ethernet, the first millimeter wave radar 120, the second millimeter wave radar 222, and the third millimeter wave radar 223 are connected to the second gateway via Ethernet, and the first camera 110, the second camera 111, the third camera 210, the fourth camera 211, the fifth camera 212, the sixth camera 310, the seventh camera 311, the eighth camera 312, and the ninth camera 313 are connected to the third gateway via GMSL2.

[0064] Specifically, the above-mentioned first millimeter-wave radar, second millimeter-wave radar and third millimeter-wave radar use millimeter-wave radars with strong penetration, high resolution and strong anti-interference ability. In this embodiment, 60GHz, 77GHz and 79GHz can be used. The 60GHz millimeter-wave radar is mainly used for short-range, high-precision object detection and positioning. In autonomous driving vehicles, this radar is becoming more and more important due to its high resolution, fast response and ability to provide stable performance in complex environments. The millimeter-wave radar of the 77GHz millimeter-wave radar is mainly used for long-range target detection. It has high resolution and accuracy, and can achieve a longer detection distance and a smaller detection angle, which is conducive to more accurate identification and tracking of targets. Compared with the 77GHz radar, the 79GHz radar has a wider bandwidth, which means higher resolution and can identify objects more accurately.

[0065] The detection range of the first millimeter-wave radar, the second millimeter-wave radar, and the third millimeter-wave radar is 200 meters. In this embodiment, according to the specific needs of the autonomous driving truck in the port environment, a small millimeter-wave radar can be selected. The small millimeter-wave radar is usually designed to be very compact and can be as small as the size of a palm or even smaller. This radar is suitable for installation in narrow spaces of vehicles, such as inside the bumper, near the headlights, or on the roof, to achieve all-round environmental perception. The weight of a small millimeter-wave radar is usually between a few hundred grams and 1 kilogram. The lightweight design makes it easy to integrate into various vehicles and equipment without significantly increasing the overall weight. A medium-sized millimeter-wave radar can also be selected. The size of a medium-sized millimeter-wave radar is usually larger than that of a small radar, but it remains within the range of easy installation. The size may be close to the size of an ordinary mobile phone box and weigh between 1 and 5 kilograms, depending on the components and functions inside the radar.

[0066] Specifically, the gateway plays the role of data aggregation and transmission, while the operator is responsible for processing and analyzing the aggregated data to realize the environmental perception and decision-making functions of the autonomous driving truck. The gateway is responsible for receiving sensor data from each module, performing preliminary processing such as data format conversion and data compression, and then sending the processed data to the operator, which further processes the data to realize comprehensive perception of the environment and target detection, classification and tracking, providing a basis for autonomous driving decision-making.

[0067] like Figure 3 As shown, the system also includes a repeater, which communicates with the third gateway. The third group of perception cameras of the close-range blind spot perception module includes a perception camera installed on the rear bumper of the trailer. The tenth camera 314 installed on the rear bumper of the trailer is connected to the repeater via GMSL2.

[0068] Specifically, a repeater is a device used to extend the data transmission distance. In the present technical solution, the repeater is used to connect the tenth camera on the rear bumper of the trailer and the third gateway located in the tractor. Since there may be a large distance between the trailer and the tractor, the directly connected wiring harness may not be able to guarantee the stability and efficiency of data transmission. Therefore, a repeater is introduced to enhance the data transmission capability and ensure high-quality image data transmission even over long distances.

[0069] Specifically, a repeater is a network device that is mainly used to extend the transmission distance of the network. During the data transmission process, the signal will gradually weaken due to the attenuation of the physical medium. The repeater can receive this weakened signal, amplify it and resend it, thereby achieving the purpose of extending the signal transmission distance. For the sensor configuration scheme of the autonomous driving truck in this embodiment, the repeater is particularly important because it can ensure that the data collected by the fisheye camera deployed on the rear bumper of the trailer can be stably transmitted to the gateway device located in the tractor. Considering that the distance between the trailer and the tractor may be long, direct data transmission may affect the integrity and real-time nature of the data due to signal attenuation. The addition of a repeater can solve this problem well and ensure the comprehensiveness and reliability of the perception of the entire autonomous driving system.

[0070] There are several common types of repeaters:

[0071] Wireless Repeater: Mainly used in wireless networks, such as Wi-Fi networks, it can receive wireless signals, amplify and resend them, thereby expanding the coverage of the wireless network. This type of repeater is usually used to enhance Wi-Fi signals in homes, offices or public places.

[0072] Ethernet Repeater: It is used in wired networks, such as Ethernet, to receive, amplify and resend Ethernet signals to extend the effective transmission distance of network cables. In industrial automation and video surveillance systems, Ethernet repeaters are widely used for long-distance signal transmission.

[0073] Optical Fiber Repeater: Used in fiber-optic communication networks, this type of repeater can enhance and regenerate fiber-optic signals, overcoming the attenuation and distortion of fiber-optic signals during long-distance transmission.

[0074] RF Repeater: Used in RF communications, such as radio signals, satellite communications, etc., this type of repeater can amplify and resend received RF signals to expand the communication range or improve signal quality.

[0075] Various types of repeater products differ in design and function to adapt to different network environments and communication needs. When choosing a repeater, you need to consider many factors, such as signal type, transmission distance, network speed, and environmental factors (such as interference and temperature).

[0076] Specifically, in the close-range blind spot perception module, the third group of perception cameras includes a fisheye camera installed on the rear bumper of the trailer, namely the tenth camera. The deployment position and type of this camera are designed for the blind spot directly behind the trailer. The fisheye camera can cover a larger area behind the trailer due to its wide field of view, eliminating the blind spot directly behind the trailer, thereby supporting the towing vehicle connected to the trailer to complete the automatic driving reversing task.

[0077] Furthermore, the close-range blind spot compensation perception module includes a perception camera installed on the rear enclosure and a laser radar installed on the rear enclosure. The above-mentioned operator receives image data collected by the above-mentioned perception camera installed on the rear enclosure and laser point cloud data collected by the above-mentioned laser radar installed on the rear enclosure, and determines the loading and unloading status of the goods based on the above-mentioned image data and the above-mentioned laser point cloud data.

[0078] Specifically, the operator determines the loading and unloading status of the cargo based on the received image data and laser point cloud data, which means that the system must not only detect the presence of the cargo, but also determine whether it is being loaded and unloaded by analyzing the cargo's location, motion status and other information, such as whether it is being lifted or lowered from the vehicle. By fusing the data from the camera and lidar, targets in three-dimensional space can be more accurately identified, including their shape, size, position and motion status, which is critical for safely and efficiently operating autonomous trucks in complex and dynamic port environments. This can help trucks make reasonable obstacle avoidance and action decisions to ensure that no collisions or damage occur during loading and unloading operations.

[0079] Furthermore, the above-mentioned operator determines the loading and unloading status of the cargo according to the above-mentioned image data and the above-mentioned laser point cloud data, including: pre-processing the above-mentioned image data and the above-mentioned laser point cloud data to intercept the above-mentioned image data and the above-mentioned laser point cloud data whose height is greater than a threshold, so as to obtain intercepted image data and intercepted laser point cloud data; using a neural network algorithm to process the above-mentioned intercepted image data, extracting container features or quay crane hoist features in the image, and performing feature fusion of the above-mentioned container features or the above-mentioned quay crane hoist features with the above-mentioned intercepted laser point cloud data to obtain fused features; determining the three-dimensional position and boundary information of the container or the quay crane hoist according to the above-mentioned fused features, and calculating the three-dimensional speed information of the above-mentioned container or the above-mentioned quay crane hoist in combination with historical data; determining the loading and unloading status of the cargo according to the above-mentioned three-dimensional position, the above-mentioned boundary information, and the above-mentioned three-dimensional speed information of the above-mentioned container or the above-mentioned quay crane hoist.

[0080] Specifically, first, the system preprocesses the raw data collected from the fisheye camera and lidar. The purpose of preprocessing is to remove noise, correct sensor data, and convert the data into a format that the algorithm can process. Then, the system will intercept those parts of the processed image data and laser point cloud data that exceed the threshold according to the preset height threshold. The height threshold is set to focus on detecting high targets such as containers or quay cranes. Use a neural network algorithm to analyze the intercepted image data and identify the features of the containers or quay cranes. This process involves image recognition and feature extraction. The neural network can identify the shape, color, texture and other features of a specific object. Subsequently, the extracted image features are fused with the position information in the intercepted laser point cloud data to form a fusion feature. Based on the fusion feature, the precise position and boundary information of the container or quay crane in three-dimensional space are calculated, which can not only determine the approximate position of the target object, but also its specific size and surrounding space range.

[0081] Specifically, in the environment of the autonomous driving truck at the port, the fisheye camera uses a fisheye camera with high resolution, wide field of view, and strong night vision to ensure that environmental information can be accurately captured under various complex lighting conditions. In this embodiment, a fisheye camera with a resolution of 12 million pixels, 21 million pixels, and a field of view of 180 degrees or 360 degrees can be selected.

[0082] The main features of fisheye cameras include:

[0083] Ultra-wide-angle field of view: Fisheye cameras have a wider viewing angle than ordinary cameras, usually up to 180 degrees or wider, and can capture nearly hemispherical scenes. For autonomous vehicles, this means being able to obtain more comprehensive environmental information around the vehicle.

[0084] Image distortion: Due to its ultra-wide-angle characteristics, images taken by fisheye cameras usually have significant barrel distortion, making the edges of the image appear distorted. This distortion needs to be corrected by software algorithms in some applications.

[0085] Close-range shooting capability: Fisheye cameras are able to maintain good image clarity when shooting at close distances, which is especially important for close-range obstacle detection in autonomous vehicles.

[0086] High Resolution: Modern fisheye cameras tend to have high resolution, providing clear image details, which is important for identifying and classifying objects that are far away or small in size.

[0087] Low-light performance: Some fisheye cameras are equipped with high-performance photosensitive elements that can maintain good image quality even in low-light environments, which is particularly critical for autonomous driving scenarios at night or in low illumination.

[0088] Durability and protection level: Fisheye cameras are usually installed on the outside of a vehicle, so they need to have a certain level of protection to prevent adverse weather conditions such as rain, dust, etc. from affecting the camera's performance.

[0089] Specifically, the system will calculate the three-dimensional speed information of the target object in combination with historical data. The above historical data refers to the three-dimensional position and boundary information obtained during the historical period. The system will consider the movement of the target object over a period of time in the past to predict its current movement speed and direction. Based on the above three-dimensional position, boundary information and three-dimensional speed information, the system can determine the loading and unloading status of the goods. For example, if it is detected that the container is moving towards the cargo box of the truck, and the speed information shows that it is slowing down and approaching a standstill, then the system can infer that the loading process is taking place. Conversely, if the container is moving out of the cargo box of the truck and there is three-dimensional speed information away from the truck, then the system will determine that it is in the unloading process.

[0090] More specifically, the schematic diagram of the ultra-long-range sensing module sensor field of view coverage is as follows: Figure 4 As shown, blue represents the field of view coverage of the camera, and orange represents the field of view coverage of the millimeter-wave radar, that is, the field of view coverage of the first camera 110, the second camera 111 and the first millimeter-wave radar 120 is shown, wherein the first camera 110 is used for ultra-long-distance obstacle detection and high-speed road early response, the second camera 111 is used for long-distance obstacle and signal light detection, and the first millimeter-wave radar 120 is used for dynamic obstacle detection in front. 360 panoramic perception module sensor field of view coverage diagram as shown Figure 5As shown, blue represents the camera's field of view coverage, green represents the laser radar's field of view coverage, and orange represents the millimeter-wave radar's field of view coverage, which shows the field of view coverage of the third camera 210, the fourth camera 211, the fifth camera 212, the first laser radar 220, the second laser radar 221, the second millimeter-wave radar 222, and the third millimeter-wave radar 223. Among them, the third camera 210 is used for front wide-field obstacle detection, the fourth camera 211 and the fifth camera 212 are used for rear obstacle detection to ensure safe lane changes, the first laser radar 220 and the second laser radar 221 are used for nearly 360-degree obstacle detection, and the second millimeter-wave radar 222 and the third millimeter-wave radar 223 are used for rear dynamic obstacle detection to ensure safe lane changes. The schematic diagram of the field of view coverage of the close-range blind spot perception module sensor is shown in Figure 6 As shown, blue represents the field of view coverage of the camera, and green represents the field of view coverage of the laser radar, that is, the field of view coverage of the sixth camera 310, the seventh camera 311, the eighth camera 312, the ninth camera 313, the tenth camera 314, the third laser radar 320, the fourth laser radar 321, the fifth laser radar 322 and the sixth laser radar 323 are shown, among which the fifth laser radar 322 and the eighth camera 312 are used for cargo loading and unloading status detection, the sixth camera 310, the seventh camera 311, the ninth camera 313, the third laser radar 320, the fourth laser radar 321, and the sixth laser radar 323 are used to eliminate blind spots around the vehicle body, and the tenth camera 314 is used to eliminate blind spots when reversing.

[0091] Based on the same inventive concept, an embodiment of the present application also provides an autonomous driving truck for use in ports, including: an autonomous driving truck sensor configuration system for use in ports, and an autonomous driving truck body.

[0092] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0093] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, combinations and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A sensor configuration system for autonomous driving trucks used in ports, characterized in that: Self-driving trucks include tractors and trailers, including: An ultra-long-range perception module, including a first set of perception cameras and a first set of perception radars, for sensing long-range obstacles of the autonomous driving truck; A 360-degree panoramic perception module, including a second set of perception cameras and a second set of perception radars, for sensing obstacles around the autonomous driving truck in 360 degrees; The close-range blind spot perception module includes a third group of perception cameras and a third group of perception radars, which are used to perceive obstacles that cannot be detected due to obstruction by the tractor and the trailer, and the pitch angles of at least some of the perception cameras in the second group of perception cameras and at least some of the perception radars in the second group of perception radars are adjustable.

2. The system according to claim 1, characterized in that The first group of perception cameras includes a first camera and a second camera, and the first group of perception radars includes a first millimeter-wave radar, wherein the first camera and the second camera are both installed inside the cockpit and fixed behind the front windshield, and the first millimeter-wave radar is installed in the center of the front bumper or near the center.

3. The system according to claim 1, characterized in that The second group of perception cameras includes a third camera, a fourth camera and a fifth camera, and the second group of perception radars includes a first laser radar, a second laser radar, a second millimeter-wave radar and a third millimeter-wave radar, wherein the third camera is installed inside the cockpit and fixed behind the front windshield, the fourth camera, the fifth camera, the first laser radar and the second laser radar are installed on both sides of the roof sensor bracket, and the second millimeter-wave radar and the third millimeter-wave radar are respectively installed on the left and right side enclosures and face the side rear.

4. The system according to claim 1, characterized in that The third group of perception cameras includes a sixth camera, a seventh camera, an eighth camera, a ninth camera and a tenth camera, and the third group of perception radars includes a third laser radar, a fourth laser radar, a fifth laser radar and a sixth laser radar, wherein the sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera are respectively installed on the front bumper, the right side wall, the rear wall, the left side wall and the rear bumper of the trailer, and the third laser radar, the fourth laser radar, the fifth laser radar and the sixth laser radar are respectively installed on the front bumper, the right side wall, the rear wall and the left side wall.

5. The system according to claim 4, characterized in that The sixth camera, the seventh camera, the eighth camera, the ninth camera and the tenth camera are all fisheye cameras.

6. The system according to claim 1, characterized in that The system also includes a calculator and a gateway, the calculator communicates with the gateway, the perception cameras in the ultra-long-range perception module and the 360-degree panoramic perception module are connected to the gateway via GMSL2, some perception cameras in the close-range blindness compensation perception module are connected to the gateway via GMSL2, and the perception radars in the ultra-long-range perception module, the 360-degree panoramic perception module, and the close-range blindness compensation perception module are connected to the gateway via Ethernet.

7. The system according to claim 6, characterized in that The system also includes a repeater, which communicates with the gateway. The third group of perception cameras of the close-range blind spot perception module includes a perception camera installed on the rear bumper of the trailer. The perception camera installed on the rear bumper of the trailer is connected to the repeater via GMSL2.

8. The system according to claim 6, characterized in that The close-range blind spot perception module includes a perception camera installed on the rear enclosure and a laser radar installed on the rear enclosure. The operator receives image data collected by the perception camera installed on the rear enclosure and laser point cloud data collected by the laser radar installed on the rear enclosure, and determines the loading and unloading status of the cargo based on the image data and the laser point cloud data.

9. The system according to claim 8, characterized in that The computing unit determines the loading and unloading status of the cargo according to the image data and the laser point cloud data, including: Preprocess the image data and the laser point cloud data to extract the image data and the laser point cloud data whose height is greater than a threshold, so as to obtain the intercepted image data and the intercepted laser point cloud data; The captured image data is processed using a neural network algorithm to extract container features or quay crane spreader features in the image, and the container features or quay crane spreader features are fused with the captured laser point cloud data to obtain fused features; Determine the three-dimensional position and boundary information of the container or the quay crane spreader according to the fusion feature, and calculate the three-dimensional speed information of the container or the quay crane spreader in combination with historical data; The loading and unloading status of the cargo is determined according to the three-dimensional position of the container or the quay crane spreader, the boundary information, and the three-dimensional speed information.

10. An autonomous driving truck used in a port, characterized in that: A sensor configuration system for an autonomous driving truck used in a port as described in any one of claims 1 to 9, and an autonomous driving truck body.

Images