Automated trailer camera calibration

Abstract

A method for calibrating extrinsic parameters (182) of a trailer camera (132d, 132e, 132f) supported by a trailer (106) attached to a towing vehicle (102). The method includes determining a three-dimensional feature map (162) from one or more vehicle images (133) received from a camera (132a, 132b, 132c) supported by the towing vehicle, and identifying a reference point (163) within the three-dimensional feature map. The method includes detecting the reference point within one or more trailer images received from the trailer camera after the vehicle and trailer have moved a predetermined distance in a forward direction. The method further includes determining a trailer camera position (172) of the trailer camera (132d, 132e, 132f) relative to the three-dimensional feature map (162), and determining a trailer reference point (184) based on the trailer camera position. The method further includes determining extrinsic parameters (182) of the trailer camera relative to the trailer reference point.

Description

Technical Field

[0001] This disclosure relates to an automated camera calibration method and system for vehicle trailer applications. Specifically, it relates to a method for calibrating one or more cameras mounted on a trailer attached to a tractor unit. Background Technology

[0002] A trailer is typically a non-powered vehicle towed by a powered tractor. Trailers can be general-purpose trailers, pop-up campers, travel trailers, livestock trailers, flatbed trailers, enclosed dispatch vehicles, and boat trailers, among others. The tractor can be a car, crossover, truck, van, SUV, RV, or any other vehicle configured to attach to and tow the trailer. The trailer can be attached to the powered vehicle using a trailer hitch. The receiving hitch is mounted on the tractor and connects to the trailer hitch to form a connection. The trailer hitch can be a ball-and-socket connector, a tow bar and gooseneck, or a trailer jack. Other attachment mechanisms may also be used. In addition to the mechanical connection between the trailer and the powered vehicle, in some examples, the trailer can also be electrically connected to the tractor. This electrical connection allows the trailer to draw power from the taillight circuitry of the powered vehicle, enabling the trailer to have taillights, turn signals, and brake lights synchronized with the lights of the powered vehicle.

[0003] Some trailers may be equipped with accessory trailer cameras mounted by the driver. The mounted trailer cameras can be wired or wireless and can communicate with the towing vehicle. Many vehicle-trailer applications (such as, but not limited to, trailer hitch-up assist and trailer reversing assist) rely on calibration parameters for cameras supported by both the vehicle and the trailer. Therefore, a system is desired that can provide automatic camera calibration parameters for one or more accessory cameras supported by the trailer for use in vehicle-trailer applications. Summary of the Invention

[0004] One aspect of this disclosure provides a method for calibrating eigenparameters of a trailer-mounted camera. The trailer-mounted camera is supported by a trailer attached to a tractor. The method includes determining a three-dimensional feature map at data processing hardware (e.g., a controller performing a calibration algorithm) based on one or more vehicle images received from the camera supported by the tractor. In some examples, determining the three-dimensional feature map includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure-of-motion (SfM) algorithm. The camera may be positioned in front of the vehicle to capture the environment in front of the vehicle. In some examples, one or more vehicle images may be received from more than one camera positioned to capture the environment in front of and to the sides of the vehicle. The method includes identifying reference points within the three-dimensional feature map at the data processing hardware. The method also includes detecting, at the data processing hardware, the reference points within one or more trailer images received from the trailer-mounted camera after the vehicle and trailer have moved a predetermined distance in a forward direction. The method includes determining the trailer-mounted camera position relative to the three-dimensional feature map at the data processing hardware. Additionally, the method includes determining a trailer reference point based on the trailer camera's position at the data processing hardware. Finally, the method includes determining eigenvalues ​​of the trailer camera relative to the trailer reference point at the data processing hardware.

[0005] Implementations of this aspect of the disclosure may include one or more of the following optional features. In some implementations, the method includes sending eigenvalues ​​of a trailer camera to one or more vehicle systems, causing the vehicle systems to use the eigenvalues ​​to perform actions. The eigenvalues ​​may define the center position of the trailer camera and the direction of travel of the trailer camera. In some implementations, a trailer reference point is located at a predetermined distance from the trailer camera. The trailer reference point may overlap with the trailer camera position.

[0006] The method may further include associating an identifier with each of the reference points identified in one or more vehicle images; wherein detecting a reference point within one or more trailer images received from a trailer camera includes determining an identifier associated with each of the reference points.

[0007] In some embodiments, the method further includes, before detecting a reference point within one or more trailer images received from the trailer camera at the data processing hardware: sending a command to the user interface and receiving images from the trailer camera after the vehicle has traveled a predetermined distance. The command causes the user display to prompt the vehicle driver to drive the vehicle in a forward direction.

[0008] Another aspect of this disclosure provides an exemplary arrangement of operation for a method for calibrating eigenparameters of a trailer-mounted camera. The trailer-mounted camera is supported by a trailer attached to a tractor unit. The method includes determining a three-dimensional feature map at data processing hardware (e.g., a controller performing a calibration algorithm) based on one or more vehicle images received from the camera supported by the tractor unit. The method includes identifying a reference point within the three-dimensional feature map at the data processing hardware. The method includes determining a vehicle pose relative to a first origin within the three-dimensional feature map at the data processing hardware. The method also includes detecting a reference point within one or more trailer images received from the trailer-mounted camera at the data processing hardware. The method includes determining a trailer-mounted camera position relative to the three-dimensional feature map at the data processing hardware. Additionally, the method includes determining a first trailer-mounted camera pose at the data processing hardware based on the trailer-mounted camera position. The method includes determining a trailer reference point at the data processing hardware relative to the trailer-mounted camera position, which is a trailer reference pose. Furthermore, the method includes determining a second trailer-mounted camera pose relative to the trailer reference pose at the data processing hardware. The method includes determining eigenvalues ​​of the trailer camera relative to a trailer reference posture at the data processing hardware.

[0009] Embodiments of this aspect of the disclosure may include one or more of the following optional features. In some embodiments, a trailer reference point is located at a predetermined distance from the trailer camera. The trailer reference point may overlap with the trailer camera position. Intrinsic parameters may define the center position of the trailer camera and the direction of travel of the trailer camera.

[0010] In some examples, the method further includes associating an identifier with each of the reference points identified in one or more vehicle images, wherein detecting a reference point within one or more trailer images received from a trailer camera includes determining an identifier associated with each of the reference points.

[0011] The method may further include sending extrinsic parameters of a trailer camera to one or more vehicle systems, causing the vehicle systems to use the extrinsic parameters to perform actions.

[0012] In some implementations, determining a three-dimensional feature map based on one or more vehicle images received from a camera supported by a tractor includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure of motion (SfM) algorithm.

[0013] Before detecting a reference point within one or more trailer images received from a trailer camera at the data processing hardware, the method may include sending an instruction to a user interface that causes a user display to prompt the vehicle driver to drive the vehicle in a forward direction and to receive images from the trailer camera after the vehicle has traveled a predetermined distance.

[0014] Another aspect of this disclosure provides a system comprising data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. These operations include the methods described above.

[0015] Details of one or more embodiments of this disclosure are set forth in the accompanying drawings and the following description. Other aspects, features, and advantages will be apparent from the specification, the drawings, and the claims. Attached Figure Description

[0016] Figure 1A This is a top view of an exemplary vehicle trailer system.

[0017] Figure 1B yes Figure 1A A top view of an exemplary vehicle trailer system, wherein the vehicle and trailer are at an angle relative to each other.

[0018] Figure 2 yes Figure 1A A schematic diagram of an exemplary vehicle trailer system is shown.

[0019] Figure 3 It has reference points extracted from images of one or more vehicle cameras. Figure 1A A top view of an exemplary vehicle trailer system.

[0020] Figure 4 It has the ability to extract images from one or more trailer cameras. Figure 3 reference point Figure 1A A top view of an exemplary vehicle trailer system.

[0021] Figure 5 It has a trailer reference point Figure 1A A top view of an exemplary vehicle trailer system.

[0022] Figure 6 and 7 This is a schematic diagram of an exemplary arrangement of operations for determining the eigenvalues ​​of a trailer camera.

[0023] In the various figures, the same reference numerals denote the same elements. Detailed Implementation

[0024] A towing vehicle (such as, but not limited to, cars, crossovers, trucks, vans, SUVs, and RVs) can be configured to tow a trailer. The towing vehicle is attached to the trailer via a trailer hitch. A system capable of calibrating accessory cameras supported by the trailer is desired.

[0025] refer to Figure 1A , 1B In some embodiments, the vehicle-to-trailer system 100 includes a tractor 102 having a towing ball 104 supported by a vehicle hook-up bar 105. The vehicle-to-trailer system 100 also includes a trailer 106 having a trailer hitch connector 108 supported by a trailer hitch link 109. The vehicle towing ball 104 is coupled to the trailer hitch connector 108. The tractor 102 may include a drive system 110 that maneuvers the tractor 102 across a road surface based on drive commands, for example, having x, y, and z components. As shown, the drive system 110 includes right front wheels 112, 112a, left front wheels 112, 112b, right rear wheels 112, 112c, and left rear wheels 112, 112d. The drive system 110 may also include other wheel configurations. The drive system 110 may also include a braking system (not shown) and an acceleration system (not shown), the braking system including brakes associated with each wheel 112, 112a-d, and the acceleration system configured to regulate the speed and direction of the tractor 102. Furthermore, the drive system 110 may include a suspension system (not shown) including tires, tire air, springs, shock absorbers, and links associated with each wheel 112, 112a-d, which connects the tractor 102 to its wheels 112, 112a-d and allows relative movement between the tractor 102 and the wheels 112, 112a-d.

[0026] The tractor 102 can move across the road surface through various combinations of motions relative to three mutually perpendicular axes defined by the tractor 102: lateral axis X V Front and rear axis Y V and the central vertical axis Z V Horizontal axis X V It extends between the right and left sides of tractor 102. Along the front and rear axle Y... V The forward drive direction is designated as F. V This is also known as forward motion. Furthermore, along the forward / backward direction Y... V The rearward or rearward drive direction is specified as R. V This is also known as rearward motion. In some examples, the tractor 102 includes a suspension system (not shown) that, when adjusted, causes the tractor 102 to move around an X-axis. V Axis and / or Y V Inclined along the axis, or along the central vertical axis Z V sports.

[0027] Furthermore, trailer 106 follows tractor 102 across the road surface via various combinations of motion relative to three mutually perpendicular axes defined by trailer 106: trailer lateral axis X T Trailer front and rear axle Y T And the vertical axis Z of the trailer center T Trailer lateral axis X T The trailer steering axis 107 extends between the right and left sides of the trailer 106. In some examples, the trailer 106 includes a front axle (not shown) and a rear axle 107. In this case, the trailer lateral axis X... T Extending between the right and left sides of trailer 106 along the midpoint of the front and rear axles (i.e., the virtual steering axis). Along the trailer's front and rear axis Y... T The forward drive direction is designated as F. T This is also known as forward motion. Furthermore, along the forward / backward direction Y... T The trailer's rearward or rearward drive direction is designated as R. T This is also known as rearward movement. Therefore, the movement of the vehicle-trailer system 100 includes the tractor 102 moving along its lateral axis X. V Front and rear axis Y V and the central vertical axis Z V The movement of the trailer 106 along its trailer transverse axis X T Trailer front and rear axle Y T And the vertical axis Z of the trailer center T The movement of the tractor. Therefore, when the tractor 102 is in the forward direction F V When turning during movement, the trailer 106 follows. During turning, the tractor 102 and the trailer 106 form a trailer angle α, which is the angle between the front and rear axles of the vehicle, Y. V and the front and rear axle Y of the trailer T The angle between them.

[0028] The towing vehicle 102 may include a user interface 120. The user interface 120 may include a display 122, knobs, and buttons, which serve as input mechanisms. In some examples, the display 122 may show knobs and buttons. In other examples, the knobs and buttons are a combination of knobs and buttons. In some examples, the user interface 120 receives one or more driver commands from the driver and / or displays one or more notifications to the driver via one or more input mechanisms or the touchscreen display 122. In some examples, the display 122 displays an image 133 of the environment surrounding the vehicle towing system 100.

[0029] The vehicle towing system 100 includes a sensor system 130 that provides sensor data 131 of the vehicle towing system 100 and its surroundings to assist the driver while driving. The sensor system 130 may include different types of sensors that may be used individually or together to generate environmental perception of the vehicle towing system 100, which is used to enable the towing vehicle 102 to drive and assist the driver in making intelligent decisions based on objects and obstacles detected by the sensor system 130. The sensor system 130 may include one or more cameras 132, 132a-f supported by the vehicle towing system 100. In some embodiments, the towing vehicle 102 includes a front vehicle camera 132a (i.e., the first camera) mounted to provide a view of the forward driving path of the towing vehicle 102, or in other words, the front vehicle camera 132a captures images 133 of the environment ahead of the towing vehicle 102. In some examples, sensor system 130 also includes side vehicle cameras 132c, 132d (i.e., the third and fourth cameras), each mounted to provide side images 133 of the side environment of the towing vehicle 102. Side vehicle cameras 132c, 132d may each be mounted on a side mirror, side door, or side frame of the towing vehicle 102. Sensor system 130 also includes one or more cameras 132d-f supported by trailer 106. For example, a trailer rear camera 132d (i.e., the fourth camera) is mounted to provide a view of the rear environment of trailer 106. Additionally, side trailer cameras 132e, 132f (i.e., the fifth and sixth cameras) are mounted to each provide side images 133 of the side environment of trailer 106. Trailer cameras 132d-e may be connected to vehicle 102 (e.g., vehicle controller 140) via a wired connection or wirelessly. As shown in the figure, the vehicle-to-trailer system 100 includes six cameras 132: three cameras 132a-c supported by the towing vehicle 102 and three cameras 132d-e supported by the trailer 106; however, the vehicle 102 may include at least one or more cameras 132, and the trailer may include at least one or more cameras 132, wherein each camera 132 is positioned to capture the environment of the vehicle-to-trailer system 100. Each camera 132 may include intrinsic parameters (e.g., focal length, image sensor format, and principal point) and extrinsic parameters (e.g., extrinsic parameters define the position of the camera center and the camera's direction of travel relative to a reference point).

[0030] The sensor system 130 may also include other sensors 134 that detect vehicle motion (i.e., velocity, angular velocity, position, etc.). These other sensors 134 may include an inertial measurement unit (IMU) configured to measure the vehicle's linear acceleration (using one or more accelerometers) and rotational rate (using one or more gyroscopes). In some examples, the IMU also determines a forward direction reference for the tractor 102. Therefore, the IMU determines the pitch, roll, and yaw of the tractor 102. Other sensors 134 may also include, but are not limited to, radar, sonar, LIDAR (optical remote sensing that may require measuring the properties of scattered light to find the distance and / or other information of distant targets), LADAR (laser detection and ranging), ultrasound, HFL (high-resolution 3D flash LIDAR), etc.

[0031] User interface 120 and sensor system 130 communicate with vehicle controller 140. Vehicle controller 140 includes computing device (or data processing hardware) 142 (e.g., a central processing unit with one or more computing processors) that communicates with non-transitory memory or hardware memory 144 (e.g., hard disk, flash memory, random access memory), capable of storing executable instructions on the computing processor(s). The controller may be supported by tractor 102, trailer 106, or both tractor 102 and trailer 106. In some examples, controller 140 executes calibration algorithm 150 to calibrate one or more trailer cameras 132, 132d-f. As shown, vehicle controller 140 is supported by tractor 102; however, vehicle controller 140 may also be detached from tractor 102 and communicate with tractor 102 via a network (not shown).

[0032] In some implementations, calibration algorithm 150 is configured to simplify the calibration process for trailer cameras 132d-f attached to trailer 106 by the driver, or for trailer cameras 132d-f previously mounted on trailer 106 but not calibrated. Because trailer cameras 132d-f can be connected to a controller via wired or wireless means, it can be difficult for the driver to calibrate non-intrinsic parameters of the trailer cameras 132d-f (e.g., position and rotation relative to a reference point). Thus, calibration algorithm 150 provides an automated method for calibrating the camera 132d-f, requiring only that the driver drive in the forward direction.

[0033] The controller 140 receives intrinsic parameters (e.g., focal length, image sensor format, principal point, and distortion parameters) of each of the vehicle cameras 132a-c and each of the trailer cameras 132d-f from the cameras 132, since each camera 132, 132a-f is aware of its intrinsic parameters. As for the extrinsic parameters of the vehicle cameras 132a-c, their positions and rotations are known since these cameras are mounted on the vehicle 102; therefore, the extrinsic parameters of the vehicle cameras 132a-c are known. However, the extrinsic parameters of the trailer cameras 132d-f are unknown because these cameras 132d-f are placed on the trailer 106 by the driver. Therefore, the calibration algorithm 150 automatically calibrates the extrinsic parameters 182 of each of the trailer cameras 132d-f relative to the trailer reference point 184.

[0034] refer to Figure 2-5 The calibration algorithm 150 includes three phases 160, 170, and 180. During the first phase 160, the calibration algorithm 150 constructs a three-dimensional feature map 162 based on images 133 captured by vehicle cameras 132a-c, and determines the vehicle pose 164 (e.g., position and orientation) relative to a vehicle reference point 166 based on the three-dimensional feature map 162. The vehicle reference point 166 may be a predetermined point within the vehicle 102. In some examples, the vehicle reference point 166 is located on the vehicle's lateral axis X. V The front and rear axles of the vehicle Y V and the vertical axis Z of the vehicle center V At the intersection. Calibration algorithm 150 can use visual ranging (VO) algorithm, simultaneous localization and mapping (SLAM) algorithm and / or structure of motion (SfM) algorithm to determine a 3D feature map. The VO, SLAM and SfM framework is a well-established theory and allows calibration algorithm 150 to locate one or more reference points 163 in real time in a self-generated 3D point cloud map based on the received image 133. The VO method extracts image feature points 163 and tracks the extracted image feature points 163 in the image sequence. Examples of feature points 163 may include, but are not limited to, edges, corners or blobs of objects within image 133. The VO method can also directly use pixel intensity in the image sequence as visual input. The SLAM method constructs or updates a map of the unknown environment while simultaneously tracking one or more targets or image feature points 163 (e.g., reference points). In other words, the SLAM method uses the received image 133 as the sole source of external information to construct a representation of the environment including image feature points 163. The SfM method estimates the 3D structure of object or image feature points 163 based on the received image 133 (i.e., 2D image).

[0035] Once the calibration algorithm 150 has determined the three-dimensional feature map 162, the calibration algorithm 150 determines the vehicle pose 164 relative to the origin within the three-dimensional feature map 162.

[0036] During the second phase 170, calibration algorithm 150 repositions the trailer cameras 132d-f relative to the 3D feature map 162. In other words, calibration algorithm 150 determines the trailer camera position 172 associated with each trailer camera 132d-f, where the trailer camera position 172 is relative to the 3D feature map 162. In some examples, calibration algorithm 150 instructs display 122 to prompt the driver in the forward direction F. V F T A vehicle trailer system 100 is used. During forward movement of the vehicle trailer system 100, trailer cameras 132d-f capture images 133, and a calibration algorithm 150 analyzes the captured trailer images 133 to identify one or more image features 162 identified by the vehicle images 133 in a first phase 160. Once the calibration algorithm 150 identifies one or more of the image features 162, it determines a trailer camera pose 174 for each trailer camera 132d-f relative to a three-dimensional feature map 162. Each trailer camera pose 174 includes a (x, y, z) position and orientation with respect to the origin defined within the three-dimensional feature map 162.

[0037] During the third phase 180, the calibration algorithm 150 estimates the trailer camera eigenvalues ​​182 relative to the trailer reference point 184 based on the camera pose 174 determined during the second phase 170. The trailer reference point 184 can be any point within the trailer 106, such as the camera position, the center of the camera position, or any predetermined location within the trailer 106.

[0038] In some implementations, calibration algorithm 150 optimizes the trailer camera eigenparameters 182 and pose of trailer reference point 184 by minimizing the sum of all reprojection errors, while keeping the coordinates of all feature points 163 of the 3D map 162 fixed. Reprojection errors provide a qualitative measure of accuracy. Reprojection error is the image distance between the projected point and the measured point. Reprojection error is used to determine how close the estimated reconstruction of the actual projection of the 3D trailer reference point 184 is to the actual projection of that point.

[0039] Once the third phase 180 is completed, the calibration algorithm 150 transmits or sends the trailer camera eigenvalues ​​182 and orientation of the trailer reference point 184 to one or more vehicle-to-trailer systems that rely on these eigenvalues ​​182 and orientation to make system decisions. For example, the calibration algorithm 150 transmits or sends the trailer camera eigenvalues ​​182 and orientation of the trailer reference point 184 to a path planning system, a trailer reversing assist system, a trailer hitch-up assist system, or any other system that relies on these eigenvalues ​​182 and orientation.

[0040] The calibration algorithm 150 performs an automatic calibration method that reduces the complexity of driver operation and uses vehicle cameras 132a-c to calibrate trailer cameras 132d-f. The calibration algorithm 150 enables the implementation of new features for the trailer 106.

[0041] Figure 6 Provided for use Figure 1A-5 This is an exemplary arrangement of the operation of method 600 for calibrating the eigenparameters 182 of trailer cameras 132d, 132e, and 132f. The trailer cameras 132d, 132e, and 132f are supported by a trailer 106 attached to a tractor unit 102. At block 602, method 600 includes determining a three-dimensional feature map 162 at data processing hardware (e.g., a controller 140 executing calibration algorithm 150) based on one or more vehicle images 133 received from cameras 132a, 132b, and 132c supported by the tractor unit 102. In some examples, determining the three-dimensional feature map 162 includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure-of-motion (SfM) algorithm. Cameras 132a, 132b, and 132c may be positioned at the front of vehicle 102 to capture the environment in front of vehicle 102. In some examples, one or more vehicle images 133 may be received from more than one camera 132a, 132b, 132c positioned to capture the front and side environment of vehicle 102. At block 604, method 600 includes identifying reference points 163 within a 3D feature map 162 at data processing hardware. At block 606, method 600 includes detecting, at data processing hardware 150, points along the forward direction F of vehicle 102 and trailer 106. V F TAfter moving a predetermined distance, a reference point 163 is obtained from one or more trailer images 133 received from trailer cameras 132d, 132e, and 132f. At block 608, method 600 includes determining, at data processing hardware 150, the trailer camera positions 172 of trailer cameras 132d, 132e, and 132f relative to the 3D feature map 162. Additionally, at block 610, method 600 includes determining a trailer reference point 184 at data processing hardware based on the trailer camera positions 172. Finally, at block 612, method 600 includes determining, at data processing hardware 150, eigenvalues ​​182 of trailer cameras 132d, 132e, and 132f relative to the trailer reference point 184.

[0042] In some embodiments, method 600 includes sending eigenvalues ​​182 of trailer cameras 132d, 132e, 132f to one or more vehicle systems, causing the vehicle systems to perform actions using the eigenvalues ​​182. The eigenvalues ​​182 may define the center position and direction of travel of the trailer cameras 132d, 132e, 132f. In some embodiments, a trailer reference point 184 is located at a predetermined distance from the trailer cameras 132d, 132e, 132f. The trailer reference point 184 may overlap with the trailer camera position 172.

[0043] Method 600 may further include associating an identifier with each of the reference points 163 identified in one or more vehicle images 133; wherein detecting the reference points 163 within one or more trailer images 133 received from trailer cameras 132d, 132e, 132f includes determining an identifier associated with each of the reference points 163.

[0044] In some embodiments, method 600 further includes, before detecting a reference point 163 within one or more trailer images 133 received from trailer cameras 132d, 132e, 132f at data processing hardware 150: sending a command to user interface 120 and receiving images 133 from trailer cameras 132d, 132e, 132f after vehicle 102 has traveled a predetermined distance. This predetermined distance allows vehicle 102 to align with trailer 106 such that the trailer angle is zero. The command causes user display 122 to prompt the driver of vehicle 102 to drive vehicle 102 in a forward direction.

[0045] Figure 7 Provided for use Figure 1A-5This is an exemplary arrangement of the operation of method 700 for calibrating the eigenparameters 182 of trailer cameras 132d, 132e, and 132f. The trailer cameras 132d, 132e, and 132f are supported by a trailer 106 attached to a tractor unit 102. At block 702, method 700 includes determining a three-dimensional feature map 162 at data processing hardware (e.g., a controller 140 executing calibration algorithm 150) based on one or more vehicle images 133 received from cameras 132a, 132b, and 132c supported by the tractor unit 102. At block 704, method 700 includes identifying a reference point 163 within the three-dimensional feature map 162 at data processing hardware 150. At block 706, method 700 includes determining a vehicle pose 164 at data processing hardware 150 relative to a first origin within the three-dimensional feature map 162. At block 708, method 700 includes detecting, at data processing hardware 150, a reference point 163 within one or more trailer images 133 received from trailer cameras 132d, 132e, 132f. At block 710, method 700 includes determining, at data processing hardware 150, trailer camera positions 172 of trailer cameras 132d, 132e, 132f relative to a 3D feature map 162. Additionally, at block 712, method 700 includes determining, at data processing hardware 150, a first trailer camera pose 174 of trailer cameras 132d, 132e, 132f relative to a vehicle pose 164 within the 3D feature map 162. At block 714, method 700 includes determining a trailer reference point 184 at data processing hardware 150 based on trailer camera positions 172. At block 716, method 700 includes determining a trailer reference pose 186 at data processing hardware 150 relative to a trailer reference point 184 at trailer camera position 172. Additionally, at block 718, method 700 includes determining a second trailer camera pose 174 at data processing hardware 150 relative to trailer reference pose 186 for trailer cameras 132d, 132e, and 132f. At block 720, method 700 includes determining eigenvalues ​​182 at data processing hardware 150 for trailer cameras 132d, 132e, and 132f relative to trailer reference pose 186.

[0046] In some embodiments, the trailer reference point 184 is located at a predetermined distance from the trailer cameras 132d, 132e, and 132f. The trailer reference point 184 may overlap with the trailer camera position 172. The eigenvalue parameter 182 may define the center position of the trailer cameras 132d, 132e, and 132f and their direction of travel.

[0047] In some examples, method 700 also includes associating an identifier with each of the reference points 163 identified in one or more vehicle images 133, wherein detecting the reference points 163 within one or more trailer images 133 received from trailer cameras 132d, 132e, 132f includes determining an identifier associated with each of the reference points 163.

[0048] Method 700 may further include sending eigenvalues ​​182 of trailer cameras 132d, 132e, 132f to one or more vehicle systems, causing the vehicle systems to use the eigenvalues ​​182 to perform actions.

[0049] In some implementations, determining a three-dimensional feature map 162 based on one or more vehicle images 133 received from cameras 132a, 132b, 132c supported by the tractor 102 includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure of motion (SfM) algorithm.

[0050] Before detecting a reference point 163 within one or more trailer images 133 received from trailer cameras 132d, 132e, 132f at data processing hardware 150, method 700 may include sending a command to user interface 120 that causes a user display to prompt the driver of vehicle 102 in the forward direction F V F T The vehicle 102 is driven and images 133 are received from trailer cameras 132d, 132e, and 132f after the vehicle 102 has traveled a predetermined distance.

[0051] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuits, integrated circuits, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system, said programmable system including at least one programmable processor, at least one input device, and at least one output device, said programmable processor being dedicated or general-purpose, coupled to receive data and instructions from and send data and instructions to the storage system.

[0052] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0053] The embodiments of the subject matter and functional operation described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations thereof. Furthermore, the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that implement machine-readable propagating signals, or a combination thereof. The terms "data processing apparatus," "computing device," and "computing processor" cover all means, devices, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagating signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[0054] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in sequential order, or requiring all shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0055] Many embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other embodiments are also within the scope of the appended claims.

Claims

1. A method for calibrating eigenparameters of a trailer-mounted camera, the trailer-mounted camera being supported by a trailer attached to a tractor, the method comprising: A three-dimensional feature map is determined at the data processing hardware based on one or more images of the tractor unit received from a camera supported by the tractor unit. The reference points within the 3D feature map are identified at the data processing hardware. The reference point is detected at the data processing hardware within one or more trailer images received from the trailer camera after the tractor and the trailer have moved a predetermined distance in the forward direction; The position of the trailer camera relative to the three-dimensional feature map is determined at the data processing hardware; The trailer reference point is determined at the data processing hardware based on the position of the trailer camera. as well as The eigenvalues ​​of the trailer camera relative to the trailer reference point are determined at the data processing hardware.

2. The method according to claim 1, wherein, The trailer reference point is located at a predetermined distance from the trailer camera.

3. The method according to claim 1, wherein, The trailer reference point overlaps with the position of the trailer camera.

4. The method according to claim 1, wherein, The non-eigenvalue parameter defines the center position of the trailer camera and the forward direction of the trailer camera after the tractor and the trailer have moved a predetermined distance in the forward direction.

5. The method according to claim 1, further comprising: Associate the identifier with each of the reference points identified in the one or more tractor images; Detecting reference points within one or more trailer images received from the trailer camera includes determining an identifier associated with each of the reference points.

6. The method according to claim 1, further comprising: The non-intrinsic parameters of the trailer camera are sent to one or more vehicle systems, thereby enabling the vehicle systems to perform actions using the non-intrinsic parameters.

7. The method according to claim 1, wherein, Determining a 3D feature map based on one or more images of a tractor unit received from a camera supported by the tractor unit includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure of motion (SfM) algorithm.

8. The method of claim 1, further comprising, before detecting the reference point within one or more trailer images received from the trailer camera at the data processing hardware: Send a command to the user interface, the command causing the user display to prompt the driver of the tractor to drive the tractor in the forward direction; After the tractor has traveled a predetermined distance, images are received from the trailer camera.

9. A method for calibrating eigenvalues ​​of a trailer-mounted camera, the trailer-mounted camera being supported by a trailer attached to a tractor, the method comprising: A three-dimensional feature map is determined at the data processing hardware based on one or more images of the tractor unit received from a camera supported by the tractor unit. The reference points within the 3D feature map are identified at the data processing hardware. The tractor posture relative to a first origin within the three-dimensional feature map is determined at the data processing hardware. The reference point is detected at the data processing hardware within one or more trailer images received from the trailer camera after the tractor and the trailer have moved a predetermined distance in the forward direction; The position of the trailer camera relative to the three-dimensional feature map is determined at the data processing hardware; A first trailer camera pose is determined at the data processing hardware relative to the tractor pose within the three-dimensional feature map; The trailer reference point is determined at the data processing hardware based on the position of the trailer camera. The trailer reference posture relative to the position of the trailer camera is determined at the data processing hardware. A second trailer camera posture relative to the trailer reference posture is determined at the data processing hardware. as well as The eigenvalues ​​of the trailer camera relative to the trailer reference posture are determined at the data processing hardware.

10. The method according to claim 9, wherein, The trailer reference point is located at a predetermined distance from the trailer camera.

11. The method according to claim 9, wherein, The trailer reference point overlaps with the position of the trailer camera.

12. The method according to claim 9, wherein, The eigenvalues ​​define the center position of the trailer camera and the forward direction of the trailer camera.

13. The method of claim 9, further comprising: Associate the identifier with each of the reference points identified in the one or more tractor images; Detecting the reference points within one or more trailer images received from the trailer camera includes determining an identifier associated with each of the reference points.

14. The method of claim 9, further comprising: The non-intrinsic parameters of the trailer camera are sent to one or more vehicle systems, thereby enabling the vehicle systems to perform actions using the non-intrinsic parameters.

15. The method according to claim 9, wherein, Determining a 3D feature map based on one or more images of a tractor unit received from a camera supported by the tractor unit includes performing at least one of the following: a visual ranging (VO) algorithm, a simultaneous localization and mapping (SLAM) algorithm, or a structure of motion (SfM) algorithm.

16. The method of claim 9, further comprising, before detecting the reference point within one or more trailer images received from the trailer camera at the data processing hardware: Send a command to the user interface, the command causing the user display to prompt the driver of the tractor to drive the tractor in the forward direction; After the tractor has traveled a predetermined distance, images are received from the trailer camera.

Images