TRAILER ALIGNMENT DETECTION FOR DOCK AUTOMATION USING A VISION SYSTEM AND DYNAMIC DEPTH FILTERING

Abstract

Systems and methods for determining the alignment of a trailer with respect to a loading dock or vehicle dock door using dynamic depth filtering. Image and position data are captured by a 3D camera system with a field of view at least partially oriented downwards. As a trailer approaches the dock or loading door, the captured image data includes the trailer's top surface. A dynamic height range is determined based on the estimated height of the trailer's top surface in the image data, and a dynamic depth filter is applied to filter out image data corresponding to heights outside the dynamic height range. From the depth-filtered image data, the trailer's angular position and / or lateral displacement are determined.

Description

AI DETECTION TRAILER ALIGNMENT FOR 1A AUTOMATION OF MUF.TTRS USING A VISION SYSTEM AND DYNAMIC DEPTH FILTERING Field of Invention The present invention relates to systems and methods for monitoring vehicles, including, for example, monitoring vehicles approaching a maneuvering dock door. Background of the Invention Loading operations at a laser-guided vehicle (LGV) facility require that trailer trucks align themselves within a certain tolerance with the center of a facility gate so that the LGV can enter. Similarly, when a trailer truck approaches a dock door (e.g., to load and / or unload cargo), the trailer truck must reverse toward the dock door with the trailer aligned with the center of the dock door. Summary of the Invention In some implementations, the systems and methods described herein detect and monitor the alignment of a truck parked at a dock door. Computer vision mechanisms, including, for example, a three-dimensional (3D) camera, are used to determine the position of a truck approaching the dock door. The truck's current position is compared to known / defined tolerances, and in some implementations, feedback is provided to the vehicle driver or personnel at the maneuvering facility. In other implementations, the information obtained and determined by monitoring the truck is used to provide input and, in some cases, control signals to automated systems and actuators.In some implementations, the system automatically provides feedback to the truck driver in the form of instructions or other guidance on how to move the truck to align and center it with the dock door. In some implementations, the system is configured to provide information to an automated truck driving system that is configured to automatically adjust the vehicle's position and driving to align and center the truck with the dock door based on the information received and / or pzcrQn / pznz / q / Y instructions. - 2 of the control signal. In some implementations, the systems and methods utilize a computer-implemented program and a 3D camera system installed above a door at the facility or on a loading dock door. In some implementations, the system is configured to provide full-width detection of a truck using relatively few sensors. The system, in some implementations, is configured to perform automatic self-calibration of the camera system and provide detailed feedback. Additionally, in some implementations, the system is configured to automatically identify the truck from the image data captured by the camera system (e.g.,, detecting a license plate or other truck identification feature in the captured image data), to automatically determine if the trailer's rear door is open or closed when the trailer backs up to the maneuvering dock door and, if the rear doors are open, to provide a visual analysis of the load in the trailer. In one embodiment, the invention provides a method for determining the alignment of a trailer with respect to a maneuvering dock or vehicle dock door. Image data and position data from a field of view are captured by a 3D camera system. As a trailer approaches the maneuvering dock or door, the image data includes the trailer's top surface. A dynamic height range is determined based on the estimated height of the trailer's top surface in the image data. The dynamic height range includes a variety of vertical positions that encompass the estimated height of the trailer's top surface. A dynamic depth filter is then applied to the captured image data to filter out image data corresponding to heights outside the dynamic height range.Based on the image data filtered in depth, the angular position and / or lateral displacement of the trailer is determined. In some implementations, the determined angular position and / or lateral displacement of the trailer are used to provide instructions to a driver or autonomous driving systems to adjust the trailer's alignment with the dock / gate. In some implementations, the determined angular position and / or lateral displacement are used to generate control inputs for other automated systems. In some implementations, -3 An alarm signal is automatically generated when the specified angular position and / or lateral displacement are outside the defined tolerances. In another embodiment, the invention provides a trailer alignment determination system comprising a support arm, a 3D camera system, and a controller. The support arm is installed relative to a maneuvering dock or a vehicle dock door. The 3D camera system is attached to a distal end of the support arm and positioned with its field of view at least partially oriented downwards. The controller is configured to receive image data and position data from the 3D camera system, where the position data indicates pixel positions in 3D space within the image data. The controller then determines a dynamic height range based on the estimated height of the trailer's top surface in the image data and applies a dynamic depth filter to exclude image data corresponding to heights outside the dynamic height range.After applying the dynamic depth filter, the controller determines an angular position of the trailer and / or a lateral displacement of the trailer based on the depth-filtered image data. Other aspects of the invention will become evident upon consideration of the detailed description and accompanying drawings. Brief Description of the Figures of the Invention Figure 1A is a perspective view of a camera system according to one implementation. Figure IB is a side elevation view of the camera system in Figure 1A. Figure 2A is an elevation view of a maneuvering dock equipped with the camera system of Figures 1A and 1B. Figure 2B is a side view of the maneuvering dock in Figure 2A. Figure 3 is a block diagram of a control system for the camera system of Figures 1A and 1B. Figure 4A is a flowchart of a method for determining the position and orientation of a trailer approaching the maneuvering dock using the camera system in Figure 1A and the control system in Figure 3. Figure 4B is a flowchart of a method for determining whether the approaching trailer is within alignment tolerance thresholds -4 using the camera system of Figure 1A and the control system of Figure 3. Figure 4C is a flowchart of a method that monitors the rear door of an approaching trailer using the camera system in Figure 1A and the control system in Figure 3. Figure 4D is a flowchart of a method for accessing additional information about the approaching trailer based on image data captured using the camera system in Figure 1A and the control system in Figure 3. Figure 5A is a schematic aerial view of a first example of a trailer approaching a maneuvering dock, where the trailer is not aligned with the maneuvering dock. Figure 5B is a schematic aerial view of a second example of a trailer approaching a maneuvering dock, where the trailer aligns with the maneuvering dock. Figure 6 is a flowchart of a self-calibration method for the camera system of Figure 1A using the control system of Figure 3. pzcrQn / pznz / q / Yi Detailed Description of the Invention Before explaining in detail any embodiment of the invention, it should be understood that the invention is not limited in its application to the construction details and arrangement of components set forth in the following description or illustrated in the following drawings. The invention is susceptible to other embodiments and can be practiced or carried out in various ways. Figures 1A and 1B illustrate an example of a camera and mounting system 100 for capturing image data of a vehicle (e.g., a truck with a trailer) approaching a gate or a maneuvering dock. The system includes a support arm 101 extending horizontally from a height-adjustable bracket 103. In this example, bracket 103 is installed on a wall, and the height of support arm 101 can be adjusted relative to bracket 103. At a distal end of support arm 101, a camera bracket 105 attaches a 3D camera 107 to the support arm 101. In some implementations, the 3D camera 107 is positioned facing directly downward to capture aerial image data of a vehicle passing beneath it. However, in the example in Figures 1A and 1B, the camera bracket 105 positions the 3D camera 107 at a slight angle. -5 angle with respect to the vertical such that the 3D camera 107 can capture image data from the top of an approaching vehicle and also capture image data from the front and / or rear side of an approaching vehicle (as described in more detail below). Figures 2A and 2B illustrate an example of the camera and mounting system 100 installation. This example illustrates a truck-trailer maneuvering dock with a docking door 201 and a raised docking platform 203. The camera and mounting system 100 are installed above a docking door 201 with the support arm 101 extending perpendicularly from the wall. As illustrated in the side view of Figure 2B, a truck trailer 207 is operated in reverse toward the docking platform 203 for loading and / or unloading cargo. The positioning of the 3D camera 107 provides an image field of view 205 that captures image data of the top / roof of the trailer 207, the rear of the trailer 207 (e.g., rear cargo doors, rear bumper, license plate, etc.), and also at least some objects and / or markings on the docking platform 203.However, in other implementations, the 3D camera 107 can be positioned, oriented and / or configured to provide a different field of view (e.g., positioning the camera to point straight down instead of at a slight angle to the vertical). Figure 2B also illustrates a calibration target 209 positioned on the docking platform 203. As described in more detail below, the calibration target 209 may include an object, structure, or other marking (e.g., a symbol affixed in paint) that is of a known size and located in a known position relative to the dock door 201. For example, in some implementations, the calibration target may be part of the dock door itself (e.g., one or both of the side rails of an overhead door). In some implementations, the position and size of the calibration target are known and / or predefined based on the manufacturing of the dock door 201. In other implementations, the calibration target is positioned, and a user / technician provides information to the camera system regarding the relative location of the calibration target 209.Furthermore, although calibration target 209 is shown in the example in Figure 2B placed on docking platform 203, in some other implementations, calibration target 209 may be placed in a different location than pzcpQn / cznz / q / Yi. -6 includes, for example, the surface of the road in front of docking platform 203. As illustrated in the example in Figure 2B, the calibration target 209 is placed within the field of view 205 of the 3D camera 207. As described in more detail below, in some implementations, the 3D camera system is configured to use the calibration target 207 to self-calibrate in order to determine the position and orientation of its field of view 205 with respect to the dock door 201. Consequently, the 3D camera system is then able to determine the position and orientation of other objects within the field of view 205 of the 3D camera 107 with respect to the dock door 201. Figure 3 illustrates an example of a control system for a position / orientation monitoring system using the 3D camera as illustrated in the examples in Figures IA to 2B. A controller 301 includes an electronic processor 303 communicatively coupled to a computer-readable non-transient memory 305. The memory 305 stores data and instructions which, when executed by the electronic processor 303, provide functionality to the controller 301 (including, for example, the functionality described herein). The electronic processor 303 is also communicatively coupled to a 3D camera 307, a display 309, and an alarm 311. The 3D camera 307 is configured to capture image data and provide both image and depth information to the electronic controller 303. For example, in some implementations, the 3D camera 307 includes at least two cameras and a projector. The projector of the 3D camera 307 projects a structured light pattern onto surfaces within the camera's field of view, and the two cameras capture stereo image data of the surfaces and the projected structured light pattern. Based on predefined knowledge of the structured light pattern and its appearance in the captured image data, the 3D camera can determine the location in 3D space for each pixel in the captured image data. In some implementations, the electronic processor 303 is configured to cause the display 309 to provide an image, graphic, or text output to a user on the display 309. For example, in some implementations, the electronic processor 303 is configured to cause the display 309 to show image data of the top surface of a trailer approaching the maneuvering dock with pzcpQn / cznz / q / Yi - 7 Additional graphical information superimposed on the trailer image. For example, the electronic processor 303 can cause a rectangular outline of the trailer and / or a graphical indication of a trailer centerline to be superimposed on the image data on display 309. In some implementations, the electronic processor 303 causes display 309 to provide a graphical user interface (GUI) and also receives user input from a user input device (e.g., a mouse, a keyboard, and / or a touch-sensitive display screen). As discussed later, controller 301 is configured to determine the alignment of an approaching trailer with respect to the dock. In some implementations, controller 301 is also configured to generate an alarm condition in response to the determination that the trailer's current alignment with respect to the dock exceeds a defined tolerance (e.g., the trailer is not sufficiently aligned with the dock as it approaches). In some implementations, electronic processor 303 is configured to detect this alarm condition and display a notification of the trailer misalignment on display 309. In some implementations, the system may include another alarm device 311 in addition to, or instead of, displaying the notification on display 309.For example, alarm 311 may include a speaker configured to emit an audible alarm in response to an alarm signal generated by electronic processor 303 and / or a light configured to emit a visible light (e.g., a solid-colored light, a flashing light pattern, etc.) in response to the alarm signal. Furthermore, in some implementations, the electronic processor 303 is communicatively coupled to other systems and / or actuators 313 and configured to generate data and / or control signals for those other systems / actuators 313. For example, in an automated or partially automated installation, the electronic processor 303 can be configured to generate a control signal instruction to an automated loading / unloading system (e.g., a lifting device, a conveyor, etc.) in response to the identification of an approaching truck / trailer and / or the trailer's determined position / alignment. In some implementations, the truck approaching the maneuvering dock may include an autonomous or semi-autonomous vehicle, and the electronic processor 303 can be configured to transmit data or control signals to the vehicle based on a detected position / orientation of the trailer relative to the gate / dock of pzcpQn / cznz / q / Yi -8 maneuvers and make the vehicle adjust its position as it continues its approach. Finally, in some implementations, the 301 controller includes or is communicatively coupled to a 315 wireless transceiver (e.g., a Wi-Fi transceiver, a cellular antenna) or other digital communication device. The digital communication device (e.g., the 315 wireless transceiver) allows the 301 controller to communicate with other computer systems, including, for example, a remote transportation / logistics server for tracking freight and transport vehicles. In some implementations, the 301 controller is configured to use the 315 wireless transceiver to communicate with one or more systems or controllers of the approaching vehicle.For example, controller 301 can transmit image data to the vehicle, which is then displayed on a user screen inside the vehicle operator's cab (such as, for example, the image data and user interface elements discussed earlier with reference to screen 309), can transmit alarm notifications to the vehicle to trigger a visual or audible alarm inside the vehicle operator's cab (e.g., indicating trailer misalignment with respect to the maneuvering dock), and / or can transmit control signals or data to cause an autonomous vehicle driving system to automatically correct the vehicle / trailer position / alignment with respect to the maneuvering dock. The electronic controller 301 is configured to receive 3D image and depth / position data from the 3D camera 307, to determine the relative position / alignment of an approaching vehicle based on the data captured from the 3D camera 307, and to determine whether the relative position / alignment is within a defined tolerance range. In some implementations, the electronic controller 301 is also configured to perform additional functions based on the determined position / alignment, including, for example, issuing an alarm signal in response to the determination that the position / alignment exceeds the defined tolerance. Figures 4A through 4D illustrate several examples of methods implemented by controller 301 to capture / process image data and provide functionality based on image data analysis. Starting with Figure 4A, controller 301 receives image and depth data captured by 3D camera 307 (stage 401). In some implementations, 3D camera 307 is configured to capture and process image data and provides a three-dimensional mapping of pzcpQn / cznz / q / Yi -9 The captured image data is sent to controller 301. In other implementations, the 3D camera 307 provides raw image data to controller 301, and controller 301 is configured to process the captured image data to determine an appropriate 3D mapping (step 403). Controller 301 also applies an edge smoothing filter to the image data (step 405). To isolate the shape of the trailer's top / roof from the rest of the captured image data, controller 301 first calculates a dynamic range threshold corresponding to the height of the trailer's top (step 407) and filters the image data based on this dynamic range threshold (step 409). Controller 301 is configured to determine the dynamic range threshold by estimating the trailer's height in 3D coordinate space. In some implementations, the 301 controller is configured to estimate trailer height based on captured image data. For example, the 301 controller can be configured to capture and / or store a model depth frame of the area within the 3D camera 307's field of view without a trailer on or approaching the maneuvering dock. The depth frame includes a 2D matrix indicating the vertical position / depth of the highest image pixel in each vertical column of the 3D-mapped image data (e.g., the depth of the highest image pixel from an aerial / top-down perspective). The 3D camera 307 will detect new objects entering its field of view by comparing a depth frame from a newly captured / mapped 3D image with the model depth frame.When a new object is detected, its height is estimated by calculating an average pixel depth in the depth box corresponding to the new object. In some implementations, handler 301 is configured to calculate the new object's height as the average depth of all pixels in the depth box corresponding to the new object, while in other implementations, handler 301 is configured to calculate the new object's height as the average depth of a smaller subset of pixels corresponding to the new object. In addition to, or instead of, using captured 3D image data to determine the height of the approaching trailer, in some implementations, the 301 controller is configured to determine the trailer height by accessing stored information about the approaching trailer on a computer system. For example, in some implementations, the 301 controller may be configured to identify pzcpQn / cznz / q / Yi - 10 The trailer is detected based on the wireless signal received from the approaching trailer or, as described in more detail below, based on image analysis of the license plate or other identifying markings on the approaching trailer. In some such implementations, the 301 controller can be configured to determine the trailer height by identifying the trailer and accessing stored height information for the trailer from a computer system. Once the height of the new object (e.g., the trailer) has been determined, a dynamic range is defined based at least in part on this determined height. Figure 2B illustrates an example of the dynamic range determined for trailer 207 approaching the maneuvering dock. In the example in Figure 2B, line 211 indicates the determined height of the top of trailer 207. The dynamic range is defined to include all pixels within a defined distance above and below the height of trailer 211. In Figure 2B, the maximum height of the dynamic range is indicated by line 213, and the minimum height of the dynamic range is indicated by line 215. In some implementations, the maximum and minimum of the dynamic range are defined as the predefined distance above and below the determined average height of the trailer. In some implementations, the width of the dynamic range can be fine-tuned or adjusted (i.e.,(automatically, semi-automatically, or manually in various implementations) to optimize the system for operating variables that include, for example, the camera's position / orientation and the slope of the road surface near the maneuvering dock. In other implementations, the 301 controller can be configured to dynamically determine the dynamic range width based on the captured image data. For example, in implementations where the trailer height is determined based on an analysis of a depth frame (as discussed above), the 301 controller can be configured to determine a dynamic range that is defined to include the highest and lowest pixels corresponding to the top of the trailer in the depth frame. Once the controller defines the dynamic range threshold (step 407), the 3D-mapped image data is filtered based on the defined dynamic range threshold (step 409) to remove all image data outside the defined dynamic range (e.g., by setting the pixel color value to a defined white color). In some implementations, controller 301 is configured to use dynamic range filtering to generate a binary image where pzcpQn / cznz / q / Yi - 11 All image data pixels detected within the dynamic range are reset to a first color (e.g., black) and all image data pixels detected outside the dynamic range are reset to a second color (e.g., white). Controller 301 then applies an edge-search algorithm to find a contour (e.g., profile) of the trailer's top (step 411) and, based on the defined profile, to determine the location of each upper corner of the trailer within the field of view of 3D camera 307 (step 413). Based on the assumption that the trailer's top will have a rectangular shape, controller 301 then applies a rectangular approximation to fit the shape to the trailer's profile (step 415). In some implementations, the edge-search algorithm and the rectangular approximation are applied to the image data in 3D space. However, in some implementations, to reduce computational complexity, a downward-facing two-dimensional image frame is generated after dynamic range filtering (e.g.,where the color of each pixel is defined based on whether a vertical column in 3D space includes image data within the dynamic range) and the edge search algorithm (step 411), trailer top corner localization (step 413) and rectangular approximation (step 415) are all applied only to the two-dimensional image frame. Once the profile of the upper part of the trailer is determined (in 3D coordinate space or in a single 2D horizontal plane), controller 301 then calculates the angular position of the trailer (step 417) and the lateral displacement of the trailer (step 41), as illustrated in the example in Figure 4B. In some implementations, the controller is configured to calculate the angular position and lateral displacement of the trailer by first calculating the centerline of the trailer profile (determined in step 415 of Figure 4A). As illustrated in the examples in Figures 5A and 5B, the centerline 503 is calculated as the line in a 2D horizontal plane (or in 3D space) that extends along the center of the trailer profile 501 in the direction parallel to the length of the trailer.After determining the profile 501 of the upper part of the trailer in the captured image data (using the method in Figure 4) and determining the centerline 503, the controller 301 calculates the angular position α of the trailer as the angle between the centerline 501 and the leading edge 505 of the maneuvering dock. The controller 301 also calculates the lateral displacement x of the trailer as the lateral distance between the center point 507 of the maneuvering dock and the point where the centerline 503 of the trailer's profile 501 intersects the pzcpQn / cznz / q / Yi. - 12 front edge 505 of the maneuvering dock. In some implementations, the controller 301 is also configured to calculate the distance y between the front edge 505 of the maneuvering dock and the nearest point of the trailer profile 501. In the example in Figure 5B, the trailer profile 501 is positioned aligned with the maneuvering dock. The angular position θi is 90° (i.e., the centerline 503 is perpendicular to the front edge 505 of the maneuvering dock), and the lateral displacement x is 0. Conversely, in the example in Figure 5A, the angular position α is less than 90°, and the lateral displacement x is greater than 0. In some implementations, perfect alignment between the trailer and the maneuvering dock is not required. However, the alignment must be close enough to allow loading / unloading of cargo and to prevent collisions between the truck / trailer and any nearby structures. Therefore, in some implementations, the controller 301 defines an angle tolerance and a runout tolerance. The angular tolerance defines the magnitude of deviation from 90° allowed for the angular position α.The runout tolerance defines the magnitude of deviation allowed between the trailer's centerline 503 and the maneuvering dock's center point 507. For example, in some implementations, the angular tolerance is defined as + / - 1° from the perpendicular, and the runout tolerance is defined as + / - 4 inches. Returning to Figure 4B, after calculating the trailer's angular position α (step 417) and lateral displacement x (step 419), controller 301 compares these calculated metrics with the defined tolerance thresholds. Specifically, controller 301 compares a calculated angle error (i.e., the magnitude of the difference between 90° and the calculated angular position cc) with the angle tolerance threshold (step 421) and compares the lateral displacement x with the runout tolerance threshold (step 423). If either comparison exceeds the respective tolerance threshold, controller 301 determines that a misalignment error condition is present and issues an error alarm signal (step 425).As discussed above, in some implementations, the error alarm signal can, for example, cause an error notification to be displayed on one or more display screens and / or trigger one or more audible / visual alarm devices. Additionally, in some implementations, the 301 controller can be configured for pzcpQn / cznz / q / Yi. - 13 Transmit the determined alignment / position data for the trailer on the truck to a user interface and / or other system(s) / actuator(s) (step 427). For example, in some implementations, controller 301 is configured to cause a display screen (e.g., screen 309) to show an aerial image of the trailer (based on image data captured by 3D camera 307) with the determined profile 501 of the trailer and / or the centerline 503 superimposed on the image of the trailer (step 429). In some implementations, controller 301 also causes the display screen to provide a numerical or graphical indication of metrics including, for example, angular position a, angle error, lateral displacement xy, and / or distance y.Additionally, in some implementations, the 301 controller is configured to provide instructions to a truck driver to guide him into alignment within defined tolerances. The methods illustrated in Figures 4I and 4B focus primarily on image data of the top of the trailer. However, as discussed above, controller 301 in some implementations can be configured to analyze image data corresponding to other surfaces / perspectives, including, for example, image data corresponding to the rear of the trailer. As illustrated in the example in Figure 4C, controller 301 can be configured to process the captured image data to identify a set of pixels in the 3D-mapped image data that correspond to the rear of the trailer (step 431).For example, in some implementations, the 301 controller can be configured to locate image data corresponding to the rear of the trailer after identifying the profile of the top of the trailer by applying region growth techniques and / or edge searches that extend from the rear edge of the profile of the top of the trailer. In some implementations, controller 301 is configured to use this portion of the 3D-mapped image data to determine whether the trailer's rear door is open. For example, if controller 301 determines that the 3D-mapped image data includes any image data mapped to locations beyond the rear of the trailer and within a defined distance below the top of the trailer (i.e., image data for locations that would be inside the trailer) (step 433), then controller 301 determines that the trailer's rear door should already be open. - 14 (step 435). Conversely, if the 3D mapped image data includes a set of adjacent image pixels in a vertical plane extending downward from the rear edge of the trailer's top profile (step 433), then controller 301 determines that the trailer's rear door is closed (step 435). In some implementations, the 301 controller is configured to respond to the determination that the trailer door is closed by transmitting a notification signal (e.g., to a graphical user interface display on the truck or in the dock area) that instructs the driver and / or dock personnel to open the rear trailer door (step 437). Alternatively or additionally, in some implementations, the 301 controller can be configured to transmit control signals to one or more actuators to control automated (or semi-automated) equipment based on whether the rear trailer door is closed. For example, the trailer can be equipped with actuators to automatically open and close the rear trailer doors.In some such implementations, the 301 controller is configured to respond to the determination that the trailer door is closed by transmitting a control signal to the trailer's rear door actuators, causing the trailer doors to automatically open in response. In addition, in some implementations, the 301 controller can be configured to respond to the determination that the trailer door is closed by applying image processing techniques to a portion of the trailer interior that is within the unobstructed field of view of the 3D Camera. For example, the 301 controller can be configured to determine a trailer load status based on image data captured from inside the trailer (e.g., whether the trailer is currently loaded, how much space is left in the trailer, where the load is currently placed inside the trailer, etc.) (step 439).In some of such implementations, the 301 controller is configured to use load status information determined based on image analysis of the trailer interior to display information to the truck driver and / or dock personnel on a display screen (stage 441) and / or control the operation of one or more autonomous or semi-autonomous load / unload systems. In some implementations, controller 301 is configured to repeat pzcpQn / cznz / q / Yi - The 15 stages described above with reference to Figures 4A, 4B, and 4C use the most recent image data as the vehicle / trailer continues to approach. In some implementations, controller 301 is configured to have 3D camera 307 capture new image / depth data and repeat the image analysis according to a defined period. In other implementations, the 3D camera is configured to capture image / depth data at a defined frame rate (e.g., a defined frequency), and upon completion of the analysis of a captured 3D image frame, controller 301 accesses the most recent image / depth data frame captured by 3D camera 307 and repeats the analysis. Finally, as noted above, in some implementations, the 301 controller can be configured to analyze the captured image data to access other identifying information about the approaching trailer and / or vehicle. Figure 4D illustrates an example of a method used by the 301 controller to access trailer-specific information stored in a computer system (e.g., a cloud-based transportation and logistics system). The 301 controller analyzes the captured image data to locate a unique trailer identification indicator on the trailer body (step 439). For example, the 301 controller can be configured to determine the trailer's identity based on image data from a license plate on the trailer, an alphanumeric code printed or affixed to the trailer body (e.g., on the side, rear, or top of the trailer), or a digital code (e.g.,a QR code) printed or affixed to the trailer body. Once the trailer's unique identification indicator has been located in the captured image data (step 439), the controller uses the unique indicator to access trailer-specific information from a computer system that includes, for example, data identifying the trailer size / model (step 443), the identity of the truck driver (step 445), the list of cargo to be unloaded from the trailer (step 447), and the list of new cargo to be loaded onto the trailer (step 449). As noted above, in some implementations, the 301 controller can be configured to use trailer-specific information that identifies the trailer size / model to determine the trailer height (for dynamic range image filtering purposes) and calculate the appropriate size / proportions of the trailer top for the rectangular approximation used to determine the top profile pzcpQn / cznz / q / Yi - 16 of the trailer (stage 451). In addition, the 301 controller can be configured to use the accessed information regarding the cargo to be unloaded from the trailer and / or the new cargo to be loaded onto the trailer to provide instructions to dock personnel and / or automated loading / unloading systems (Step 453). For example, in some implementations, this information accessed from the computer system may also include a load map of the trailer interior that identifies (a) the location of the particular cargo to be unloaded from the trailer and / or (b) the location within the trailer where the new cargo will be loaded. In implementations where cargo is loaded / unloaded using automated systems, the accessed load map can be used to guide the automated systems.In implementations where dock personnel manually load / unload cargo, the 301 controller can be configured to display graphical instructions on screen 309 to guide dock personnel even while the trailer is still approaching the dock. For example, the 301 controller can be configured to provide an output indicating the location of the new cargo to be loaded onto the trailer in a warehouse (or dock area) so that dock personnel can prepare the new cargo for loading. The 301 controller can also be configured to provide a graphical indication on screen 309 of the location on the trailer where the new cargo will be loaded (e.g., its position relative to another cargo already loaded on the trailer).In situations where cargo is to be unloaded from the trailer, controller 301 can be configured to provide a graphical indication on display 309 of the location within the trailer of the cargo to be unloaded and, where applicable, the location of the cargo that will remain in the trailer. In some implementations, the system is also configured to analyze captured 3D image data to assess the quality of the trailer floor and determine if any obstructions or debris are present. In some such implementations, the system is configured to generate an alarm signal upon determining that obstructions or debris are present on the trailer floor. The methods in Figures 4A, 4B, and 4C are illustrated as a loop that repeats for multiple different image frames as the trailer approaches. In some implementations, the method in Figure 4D can also be repeated pzcpQn / cznz / q / Yi - 17 several times. Alternatively, in some implementations, the method in Figure 4D is performed only once for each approaching trailer. For example, controller 301 can be configured to analyze the captured image data with each repeated iteration of the method in Figure 4A until the unique identifier of the trailer is detected in the captured image data. Then, after the unique identifier has been located in the image data, the trailer-specific information is accessed from the computer system, and the controller no longer attempts to locate the unique identifier for any subsequent image frames until a new trailer approaches the maneuvering dock. The mechanisms described in the preceding examples function to determine the location and orientation of a vehicle / trailer using the image / depth data captured by the 3D camera. Therefore, in some implementations, these mechanisms operate to determine the vehicle / trailer's location and orientation relative to a 3D coordinate frame defined by the 3D camera's position and orientation. To determine the vehicle / trailer's position and orientation relative to a gate / dock, controller 301 must also know the gate / dock's position relative to the same 3D coordinate frame. As described above with reference to Figure 2B, in some implementations, controller 301 is configured to self-calibrate to determine the gate / dock's position relative to the 3D camera 307's 3D coordinate frame using a calibration target. An example of a self-calibration process performed by controller 301 in some implementations is illustrated in Figure 6. Controller 301 captures image and depth data from 3D camera 307 (step 601) and analyzes the captured data to detect a calibration target (e.g., calibration target 209 in Figure 2B) in the captured image data (step 603). Controller 301 then determines the position and orientation of 3D camera 307 with respect to the calibration target by determining the position and orientation of the calibration target in the 3D coordinate frame of 3D camera 307 (step 605). In some implementations, the 301 controller is configured to detect the calibration target by mapping the image data in 3D space and then applying an image search algorithm (e.g., edge search followed by shape fitting) to detect an object in the 3D image data pzcpQn / cznz / q / Yi - 18 mappings that match the known size / shape of the calibration target. In other implementations, the system is configured to detect the calibration target in the original image data captured by the 3D camera (before mapping the image data to the 3D coordinate space) and then determine the location of the calibration target in the 3D coordinate frame based on the apparent size and shape of the calibration target in the original image data. Additionally, in some implementations, the calibration target itself includes a unique visual feature that makes it easier to detect in the image data.For example, the calibration target may have a unique color that distinguishes it from other objects and surfaces in the maneuvering dock area, or it may be covered with a reflective material, so that color and / or brightness filtering can be applied to the captured image data to locate the calibration target. The location and orientation of the calibration target relative to the dock / haulage door is known or, in some implementations, determined manually and provided as user input to controller 301. For example, in some implementations, when installing 3D camera 307, a technician may also install the calibration target within the camera's field of view, then measure the position / orientation of the installed calibration target relative to a specific point on the dock (e.g., the center point 507 on the leading edge 505 of the dock), and provide the measurement as input to controller 301 via a user interface. In other implementations, the calibration target may be directly integrated into the dock door structure (e.g.,(In some implementations, the side rails of an overhead door can be used as a calibration target). Returning to Figure 6, after detecting the calibration target in the captured image data (step 603) and determining the position / orientation of the calibration target in the 3D coordinate system of the 3D camera 307 (step 605), the controller then accesses or receives as input the known (or previously determined) position and orientation of the calibration target with respect to the maneuvering dock (step 607) and thereby determines the position / orientation of the maneuvering dock with respect to the 3D coordinate frame used by the 3D camera 307 (step 609). Consequently, by determining the position / orientation of the maneuvering dock (e.g., a center point 507 of the maneuvering dock) and the pzcpQn / cznz / q / Yi - 19 Trailer position / orientation (e.g., using the method in Figure 4A) in the same 3D coordinate frame, the controller 301 is able to determine the trailer position / orientation with respect to the maneuvering dock. In some implementations, by using this self-calibration technique, the controller 301 5 is able to determine the trailer position / orientation with respect to the maneuvering dock even if the maneuvering dock itself is not in the image data captured by the 3D camera. Accordingly, the examples described above provide, among other things, a camera-based system and methods for determining the alignment of a vehicle and / or trailer with respect to a gate or maneuvering dock based on image data captured by a 3D camera. Other features and advantages are set forth in the following claims.

Claims

1. A method for determining the alignment of a trailer, the method comprising: capturing, by means of a 3D camera system, image data and position data of a field of view, wherein the image data includes the upper surface of the trailer; determining the estimated height of the upper surface of the trailer; determining a dynamic height range based on the estimated height of the upper surface of the trailer, wherein the dynamic height range includes a range of vertical positions that include the estimated height of the upper surface of the trailer; applying a depth dynamic filter to filter image data corresponding to heights outside the dynamic height range from the image data captured by the 3D camera system; and determining, based on the depth-filtered image data, at least one selected from a group consisting of an angular position of the trailer and the lateral displacement of the trailer.

2. The method of claim 1, wherein applying the dynamic depth filter includes setting a first plurality of pixels to a first color value, wherein the first plurality of pixels includes all pixels in the captured image data corresponding to positions within the defined dynamic height range; and setting a second plurality of pixels to a second color value, wherein the second plurality of pixels includes all pixels in the captured image data corresponding to positions outside the defined dynamic height range.

3. The method of claim 2, further comprising generating a two-dimensional binary image, wherein the two-dimensional binary image includes a downward perspective view of the image data captured by the 3D camera with all pixels including either the first color value or the second color value, and wherein all pixels within the two-dimensional binary image set to the first color value represent objects within the dynamic height range and all pixels set to the second color value represent objects outside the dynamic height range. pzcpQn / cznz / q / Yi 4. The method of claim 1, further comprising estimating the position of a profile of the upper surface of the trailer by applying a rectangular approximation to depth-filtered image data to identify image data corresponding to a rectangular shape of the upper surface of the trailer.

5. The method of claim 1, further comprising estimating the position of a profile of the upper surface of the trailer by applying edge-search processing to the depth-filtered image data to locate edges corresponding to the profile of the upper surface of the trailer.

6. The method of claim 5, wherein estimating the position of the profile of the upper part of the trailer further includes determining the location of at least one corner of the upper surface of the trailer based on the intersection of two edges detected by edge-search processing.

7. The method of claim 1, wherein the 3D camera system is installed in a location relative to a maneuvering dock, wherein the upper surface of the trailer in the field of view of the 3D camera system includes the upper surface of a trailer approaching the maneuvering dock in reverse, wherein determining at least one selected from the group consisting of the angular position of the trailer and the lateral displacement of the trailer includes determining, based on the estimated position of the profile of the upper surface of the trailer, a centerline of the trailer, and calculating the angular position of the trailer as an angle of the centerline of the trailer with respect to a front edge of the maneuvering dock, the method further comprises comparing the angular position of the trailer with an angle tolerance threshold,where the angle tolerance threshold defines the maximum permissible magnitude of the difference between the angular position of the trailer and the angle perpendicular to the front edge of the maneuvering dock.

8. The method of claim 1, wherein the 3D camera system is installed in a location relative to a maneuvering dock, wherein the upper surface of the trailer in the field of view of the 3D camera system includes the upper surface of a trailer approaching the maneuvering dock in reverse, wherein determining at least one selected from the group consisting of the angular position of the trailer and the lateral displacement of the trailer includes determining, based on the estimated position of the profile of the upper surface of the trailer, a centerline of the trailer, and calculating the lateral displacement as the difference between a center point of a line corresponding to the maneuvering dock and a point where the centerline of the trailer intersects the line corresponding to the maneuvering dock, the method further comprising comparing the lateral displacement of the trailer with a runout tolerance threshold,where the runout tolerance threshold defines the maximum permissible magnitude of lateral displacement.

9. The method of claim 1, further comprising generating an alarm signal in response to the determination that at least one selected from the group consisting of the trailer's angular position and the trailer's lateral displacement exceeds a tolerance threshold.

10. The method of claim 1, further comprising: generating instructions for adjusting the trailer alignment based on at least one selected from the group consisting of the trailer's angular position and the trailer's lateral displacement; and transmitting the instructions to at least one selected from a group consisting of a driver of a vehicle coupled to the trailer and an autonomous driving system of the vehicle coupled to the trailer.

11. The method of claim 1, further comprising: detecting a calibration target in the image data captured by the 3D camera system; determining the position of the calibration target in a 3D coordinate frame of the 3D camera system based on the captured image data and captured position data; and determining the location and orientation of a maneuvering dock in the 3D coordinate frame of the 3D camera system based on the determined position of the calibration target in the 3D coordinate frame of the 3D camera system, wherein the location and orientation of the maneuvering dock relative to the calibration target are known, and wherein determining at least one selected from the group consisting of the angular position of the trailer and the lateral displacement of the trailer includes determining the angular position of the trailer with respect to the maneuvering dock in the 3D coordinate frame of the 3D camera system.and determining the lateral displacement of the trailer with respect to the maneuvering dock in the 3D coordinate frame of the 3D camera system.

12. The method of claim 1, wherein determining the estimated height -23 of the upper surface of the trailer includes calculating the average height of a plurality of image pixels corresponding to the upper surface of the trailer.

13. A trailer alignment determination system, the system comprising: a support arm installed at a location relative to at least one selected from a group consisting of a maneuvering dock and a gate; a 3D camera system attached to a distal end of the support arm and positioned with a field of view at least partially oriented downwards; and a controller configured to receive image data and position data from the 3D camera system, the position data indicating positions in 3D pixel space in the image data, wherein the image data includes the top surface of a trailer, determine the estimated height of the top surface of the trailer, and determine a dynamic height range based on the estimated height of the top surface of the trailer, wherein the dynamic height range includes a range of vertical positions that include the estimated height of the top surface of the trailer.Apply a dynamic depth filter to filter the image data corresponding to heights outside the dynamic height range from the image data captured by the 3D camera system, and determine, based on the depth-filtered image data, at least one selected from a group consisting of an angular position of the trailer and a lateral displacement of the trailer.

14. The system of claim 13, wherein the controller is configured to generate a binary image based on the application of the dynamic depth filter, wherein all image data pixels in the binary image corresponding to positions within the dynamic height range are set to a first color value, and wherein all image data pixels in the binary image corresponding to positions outside the dynamic height range are set to a second color value.

15. The system of claim 14, wherein the controller is further configured to estimate the position of the upper surface profile of the trailer in the captured image data by applying edge-search processing to the generated binary image and locating the position of at least one corner of the upper surface of the trailer based on the intersection between the edges detected by the edge-search processing.

16. The system of claim 14, wherein the controller is further configured to estimate the position of the upper surface profile of the trailer in the captured image data by applying a rectangular approximation to the generated binary image to identify image data corresponding to the rectangular shape of the upper surface of the trailer.

17. The system of claim 13, wherein the 3D camera system is attached to the distal end of the support arm at an angle relative to the downward vertical such that the central axis of the field of view of the 3D camera system extends partly downward and partly forward.

18. The system of claim 13, further comprising a height-adjustable mounting bracket, wherein a proximal end of the support arm is adjustablely coupled to the height-adjustable mounting bracket, and wherein the height of the support arm with respect to the height-adjustable mounting bracket is selectively adjustable.

19. The system of claim 13, further comprising a calibration target placed at a known location and orientation with respect to the maneuvering dock, wherein the controller is further configured to detect the calibration target in the image data captured by the 3D camera system; determining the position of the calibration target in a 3D coordinate frame of the 3D camera system based on the captured image data and the captured position data;and determining, based on the determined position of the calibration target in the 3D coordinate frame of the 3D camera system, the location and orientation in the 3D coordinate frame of at least one selected from the group consisting of the maneuvering dock and the gate, and wherein the controller is configured to determine at least one selected from the group consisting of the angular position of the trailer and the lateral displacement of the trailer by determining the angular position of the trailer with respect to the maneuvering dock in the 3D coordinate frame of the 3D camera system, and determining the lateral displacement of the trailer with respect to the maneuvering dock in the 3D coordinate frame of the 3D camera system.

Images