Systems, methods, and storage media for a vehicle

Abstract

The present invention relates to a system, method and storage medium for a vehicle, the system comprising: at least one sensor, at least one attachment configured to be removably attached to an auxiliary vehicle, the auxiliary vehicle configured to be attached to a main vehicle; at least one computer readable medium storing computer executable instructions; at least one first processor communicatively coupled to the at least one sensor and configured to execute the computer executable instructions to perform operations comprising: receiving, from the at least one sensor, data corresponding to an environment of the auxiliary vehicle; determining, based on the environment of the auxiliary vehicle, at least one dimension of the auxiliary vehicle and a relative position of the at least one sensor with respect to the main vehicle; providing, based on the at least one dimension of the auxiliary vehicle and the relative position of the at least one sensor, the relative position of the at least one dimension and the at least one sensor to the main vehicle.

Description

TECHNICAL FIELD

[0001] The present specification relates to dynamic positioning sensors for vehicles (e.g., towed). BACKGROUND

[0002] Autonomous vehicles use a suite of sensors and communication devices to update their position state while navigating their environment from one destination to the next. Towed vehicles obstruct the line of sight of the sensors, reducing the vehicle's positioning accuracy and perception of the surrounding environment. As the towed vehicle (particularly in maneuvers requiring high accuracy such as turns) greatly reduces the position accuracy of the autonomous vehicle, trajectory planning is also hindered. SUMMARY

[0003] A system for a vehicle, comprising: at least one sensor comprising at least one attachment configured to be removably attached to a secondary vehicle, the secondary vehicle configured for attachment to a primary vehicle; at least one computer-readable medium storing computer-executable instructions; at least one first processor communicatively coupled to the at least one sensor and configured to execute the computer-executable instructions, the execution performing operations comprising: receiving, from the at least one sensor, data corresponding to an environment of the secondary vehicle; determining, based on the environment of the secondary vehicle, at least one dimension of the secondary vehicle and a relative position of the at least one sensor with respect to the primary vehicle; and providing, based on the at least one dimension of the secondary vehicle and the relative position of the at least one sensor, the at least one dimension and the relative position of the at least one sensor to the primary vehicle.

[0004] A method for a vehicle, comprising: receiving, from at least one sensor configured to be removably attached to a secondary vehicle, data corresponding to an environment of the secondary vehicle, the secondary vehicle configured for attachment to a primary vehicle; determining, based on the environment of the secondary vehicle, at least one dimension of the secondary vehicle and a relative position of the at least one sensor with respect to the primary vehicle; and providing, based on the at least one dimension of the secondary vehicle and the relative position of the at least one sensor, the at least one dimension and the relative position of the at least one sensor to the primary vehicle.

[0005] A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a first device, the at least one program comprising instructions, which, when executed by the at least one processor, cause the first device to perform the method. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 An example of an autonomous vehicle with autonomous capabilities is shown.

[0007] Figure 2 A computer system is shown.

[0008] Figure 3 An example architecture for an autonomous vehicle is shown.

[0009] Figure 4 An example of inputs and outputs that can be used by a perception module is shown.

[0010] Figure 5 An example system diagram of a towing system and communication of the towing system with an example autonomous vehicle computer system is shown.

[0011] Figure 6A A side view of an example autonomous vehicle with integrated sensors is shown.

[0012] Figure 6B A side view of an example autonomous vehicle connected to a towed vehicle with temporary sensors is shown.

[0013] Figure 6C A top view of an example autonomous vehicle connected to a towed vehicle with temporary sensors is shown.

[0014] Figure 7 A perspective view of a towed vehicle with attached temporary sensors connected to an autonomous vehicle is shown.

[0015] Figure 8 A flowchart for processing for providing at least one dimension of a towed vehicle to an autonomous vehicle using a towing system is shown in accordance with one or more embodiments. DETAILED DESCRIPTION

[0016] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present application. It will be apparent, however, that the present application can be practiced without

[0017] In the drawings, specific arrangements or orders of illustrative elements (such as those representing devices, modules, instruction blocks, and data elements) are shown for ease of description. However, it will be appreciated by those skilled in the art that the specific order or arrangement of the illustrative elements as shown in the drawings is not intended to imply a specific processing order or sequence, or a requirement of a particular processing sequence or order. Additionally, the inclusion of an illustrative element in a drawing does not imply that such element is required in all embodiments, nor that the features represented by such element cannot be included in or combined with other elements in some embodiments.

[0018] Further, in the drawings, connecting elements, such as lines or arrows or the like, are used to illustrate connections, relationships or associations between two or more other illustrative elements, and the absence of such connecting elements is not intended to imply that no connection, relationship or association exists with respect to such elements. In other words, the absence of a connection, relationship or association between some elements is not meant to imply that some elements do not affect other elements, or that some elements are not related to other elements in some manner. Further, the use of multiple connecting elements to illustrate connections, relationships or associations between two or more elements is not meant to imply that a single connection, relationship or association cannot exist with respect to some elements, or that more than one connection, relationship or association cannot exist with respect to some elements.

[0019] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description of embodiments, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0020] Several of the described features can each be used independently of one another or in any combination thereof. However, none of the individual features alone can solve any of the above discussed problems, or can only solve one of the above discussed problems. Some of the above discussed problems can not be fully solved by any of the features described herein. Although a title is provided, information related to a particular title can also be found elsewhere in the specification. Embodiments are described herein according to the following outline:

[0021] 1. OVERALL SUMMARY

[0022] 2. SYSTEM SUMMARY

[0023] 3. AUTONOMOUS VEHICLE ARCHITECTURE

[0024] 4. AUTONOMOUS VEHICLE INPUT

[0025] 5. Towing system for assisting a vehicle

[0026] 6. Exemplary embodiments

[0027] OVERALL SUMMARY

[0028] Adding temporarily mountable sensors to at least one point on a trailer attached to a vehicle (e.g., an autonomous vehicle and / or a semi-autonomous vehicle, etc.) provides supplemental coverage of the environment surrounding the vehicle and more accurate positioning data. Typically, the trailer obscures a portion of the environment surrounding the vehicle, particularly the ability to sense objects behind or to the side of the vehicle. Further, as the combination vehicle maneuvers and the trailer position changes relative to the AV, the obscuration changes. Thus, the additional sensors enable the vehicle to sense the environment that would otherwise be obscured.

[0029] Some advantages of these techniques include generating and providing data associated with the precise size and shape of the combination vehicle and trailer for use in control and path planning implemented using the AV computer system. The sensors provide accurate trailer position information relative to the AV, which helps to reduce position error in the trailer swing during maneuvers. Increasing trailer positioning accuracy has benefits in terms of safety and control of the combination vehicle during movement choreography. The temporary sensors also provide supplemental computational capability, processing the received data into trailer positioning estimates before providing it to the AV sensor suite. Additionally, using temporary fixtures to dynamically position the sensors when combined with the vehicle sensor suite allows flexibility in the dimensions of the towed object. The portable sensor suite is adaptable to any towed object, which provides flexibility and economy to the system. Additional safety applications can include relative positioning error position detection such as tail swing or boom runout, object detection, or poor load distribution, etc.

[0030] SYSTEM SUMMARY

[0031] Figure 1 An example of an autonomous vehicle 100 with autonomous capabilities is shown.

[0032] As used herein, the term "autonomous capabilities" refers to a function, feature, or facility that enables a vehicle to operate, in part or in whole, without real-time human intervention, including but not limited to fully autonomous vehicles, highly autonomous vehicles, and conditional autonomous vehicles.

[0033] As used herein, an autonomous vehicle (AV) is a vehicle with autonomous capabilities.

[0034] As used herein, a “vehicle” includes a mode of transportation of goods or people. For example, a car, a bus, a train, an airplane, a drone, a truck, a boat, a ship, a submarine, a spaceship, etc. A self-driving car is an example of a vehicle.

[0035] As used herein, a “trajectory” refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In embodiments, the first spatiotemporal location is referred to as an initial location or a starting location, and the second spatiotemporal location is referred to as a destination, a final location, a target, a target location, or a target location. In some examples, a trajectory is composed of at least one segment (e.g., a number of segments of a road), and each segment is composed of at least one block (e.g., a portion of a lane or an intersection). In embodiments, a spatiotemporal location corresponds to a real-world location. For example, a spatiotemporal location is a pickup or drop-off location for people or goods to get on or off a vehicle.

[0036] As used herein, a “sensor(s)” includes at least one hardware component that detects information about the environment surrounding the sensor. Some hardware components can include sensing components (e.g., image sensors, biometric sensors), transmitting and / or receiving components (e.g., laser or radio frequency wave emitters and receivers), electronic components such as analog-to-digital converters, data storage devices such as RAM and / or non-volatile memory, software or firmware components, and data processing components such as application-specific integrated circuits, microprocessors, and / or microcontrollers.

[0037] As used herein, a “scene description” is a data structure (e.g., a list) or data stream that includes at least one classified or labeled object detected by at least one sensor on an AV vehicle or provided by a source external to the AV.

[0038] As used herein, a “road” is a physical area that can be traversed by a vehicle, and can correspond to a named thoroughfare (e.g., a city street, an interstate highway, etc.) or can correspond to an unnamed thoroughfare (e.g., a driveway within a house or office building, a section of a parking lot, a section of an empty parking lot, a dirt path in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, sport utility vehicles (SUVs), etc.) are capable of traversing a variety of physical areas that are not specifically designed for vehicle travel, a “road” can be any physical area that has not been formally defined as a thoroughfare by a municipality or other government or administrative body.

[0039] As used herein, a “lane” is a portion of a roadway that can be traversed by a vehicle. Sometimes a lane is identified based on lane markings. For example, a lane can correspond to most or all of the space between lane markings, or to only some of the space (e.g., less than 50%) between lane markings. For example, a roadway with lane markings far apart can accommodate two or more vehicles between the markings such that one vehicle can overtake another without crossing a lane marking, and thus can be interpreted as having a lane that is narrower than the space between lane markings, or as having two lanes between lanes. Lanes can also be interpreted in the absence of lane markings. For example, a lane can be interpreted based on physical features of the environment, such as rocks and trees along a path in a rural area, or natural obstacles to avoid in an undeveloped area. Lanes can also be interpreted independent of lane markings or physical features. For example, a lane can be interpreted based on an arbitrary path in an area that lacks obstacles that would otherwise lack features to be interpreted as lane boundaries. In an example scenario, an AV can interpret a lane through an unobstructed portion of a plaza or empty parking lot. In another example scenario, an AV can interpret a lane through a wide (e.g., wide enough for two or more lanes) roadway without lane markings. In such a scenario, the AV can communicate information about the lane to other AVs so that the other AVs can use the same lane information to coordinate path planning among themselves.

[0040] The term “over-the-air (OTA) client” includes any AV or any electronic device (e.g., computer, controller, IoT device, electronic control unit (ECU)) embedded in, coupled to, or in communication with an AV.

[0041] The term “over-the-air (OTA) update” means any update, change, deletion, or addition of software, firmware, data, or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies, including but not limited to: cellular mobile communication (e.g., 2G, 3G, 4G, 5G), radio wireless area networks (e.g., WiFi), and / or satellite internet.

[0042] The term “edge node” means at least one edge device coupled to a network that provides a portal for communication with AVs and can communicate with other edge nodes and cloud-based computing platforms to schedule and deliver OTA updates to OTA clients.

[0043] The term "edge device" means a device that implements an edge node and provides physical wireless access points (APs) to an enterprise or service provider (e.g., VERIZON, AT&T) core network. Examples of edge devices include, but are not limited to, computers, controllers, transmitters, routers, routing switches, integrated access devices (IADs), multiplexers, metropolitan area network (MAN) and wide area network (WAN) access devices.

[0044] "one or more" includes a function performed by one element, a function performed by more than one element, e.g., in a distributed manner, a function performed by one element in combination with a function performed by more than one element, a function performed by more than one element in combination with a function performed by one element, or any combination of the above.

[0045] It will also be understood that, although the terms "first," "second," etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

[0046] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes," "including," "has," "having," "has" and / or "having," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0047] As used herein, the term "if' can be construed to mean "when" or "if when" depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be construed to mean "if when it is determined" or "if when [the stated condition or event] is detected," depending on the context.

[0048] As used herein, an AV system refers to an array of AVs and hardware, software, stored data, and real-time generated data that support operation of the AVs. In embodiments, the AV system is incorporated within the AVs. In embodiments, the AV system is distributed across several locations. For example, some software of the AV system is implemented in a cloud computing environment.

[0049] In general, this document describes techniques applicable to any vehicle with at least one autonomous capability, including fully autonomous vehicles, highly autonomous vehicles, and conditionally autonomous vehicles, such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, incorporated by reference in its entirety for more detailed information on levels of vehicle autonomy). The techniques described in this document are also applicable to partially autonomous vehicles and driver-assist vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles). In embodiments, at least one Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle system can automatically perform certain vehicle operations (e.g., steering, braking, and use of a map) under certain operating conditions based on processing of sensor inputs. The techniques described in this document can benefit vehicles at any level ranging from fully autonomous vehicles to human-operated vehicles.

[0050] Autonomous vehicles have advantages over vehicles that require a human driver. One advantage is safety. For example, in 2016, the United States experienced 6 million car accidents, 2.4 million injuries, 40,000 deaths, and 13 million vehicle crashes, with an estimated societal cost of $910 billion. From 1965 to 2015, the United States traffic fatality rate has decreased from about six to about one per 100 million miles traveled, in part due to additional safety measures deployed in vehicles. For example, an additional half-second warning of an impending crash is believed to mitigate 60% of rear-end collisions. However, passive safety features (e.g., seat belts, airbags) can reach their limit in improving this number. Thus, active safety measures such as automated control of vehicles are a possible next step in improving these statistics. Since a human driver is believed to be responsible for 95% of critical pre-crash events in crashes, automated driving systems can achieve better safety outcomes, for example, by reliably identifying and avoiding critical situations better than a human; making better decisions, obeying traffic laws, and predicting future events better than a human; and controlling vehicles more reliably than a human.

[0051] Reference Figure 1The AV system 120 causes the vehicle 100 to operate along a trajectory 198 through the environment 190 to a destination 199 (sometimes referred to as a final location) while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists, and other obstacles) and obeying road rules (e.g., operating rules or driving preferences).

[0052] In embodiments, the AV system 120 includes devices 101 equipped to receive and operate on operational commands from the computer processor 146. We use the term "operational command" to mean an executable instruction (or set of instructions) that causes the vehicle to perform an action (e.g., a driving maneuver). Operational commands can include, but are not limited to, instructions for the vehicle to start moving forward, stop moving forward, start moving backward, stop moving backward, accelerate, decelerate, make a left turn, and make a right turn. In embodiments, the computer processor 146 is similar to the processor 204 described below with reference to FIG. 2. Examples of devices 101 include a steering controller 102, a brake 103, a gear, an accelerator pedal or other acceleration control mechanism, a windshield wiper, a side door lock, a window control, and a turn indicator. Figure 2

[0053] In embodiments, the AV system 120 includes sensors 121 for measuring or inferring properties of the state or condition of the vehicle 100, such as the AV's position, linear and angular velocity and linear and angular acceleration, and heading (e.g., the direction of the front end of the vehicle 100). Examples of sensors 121 are a GPS, an inertial measurement unit (IMU) that measures both linear acceleration and angular rate of the vehicle, a wheel rate sensor for measuring or estimating wheel slip, a wheel brake pressure or brake torque sensor, an engine torque or wheel torque sensor, and a steering angle and angular rate sensor.

[0054] In embodiments, the sensors 121 also include sensors for sensing or measuring properties of the AV's environment. For example, monocular or stereo video cameras 122 in the visible, infrared, or thermal (or both) light spectrum, LiDAR 123, RADAR, ultrasonic sensors, time-of-flight (TOF) depth sensors, rate sensors, temperature sensors, humidity sensors, and precipitation sensors.

[0055] In embodiments, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with the computer processor 146 or data collected by the sensors 121. In embodiments, the data storage unit 142 is similar to the data storage unit 242 described below with reference to FIG. 2. Figure 2 ​The described ROM 208 or storage 210 are similar. In embodiments, the memory 144 is similar to the main memory 206 described below. In embodiments, the data storage unit 142 and the memory 144 store historical, real-time, and / or predictive information about the environment 190. In embodiments, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In embodiments, data related to the environment 190 is transmitted from the remote database 134 to the vehicle 100 through a communication channel.

[0056] In embodiments, the AV system 120 includes communication devices 140 for transmitting properties of other vehicles' states and conditions, such as position, linear and angular velocity, linear and angular acceleration, and linear and angular heading, measured or inferred to the vehicle 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices and devices for wireless communication through point-to-point or ad hoc networks or both. In embodiments, the communication devices 140 communicate across the electromagnetic spectrum, including radio and optical communications, or other media (e.g., air and acoustic media). The combination of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communication (and, in some embodiments, at least one other type of communication) is sometimes referred to as vehicle-to-everything (V2X) communication. V2X communication is generally in compliance with at least one communication standard for communication with and between autonomous vehicles.

[0057] In embodiments, the communication devices 140 include a communication interface. For example, a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near field, infrared, or radio interface. The communication interface transmits data from the remote database 134 to the AV system 120. In embodiments, the remote database 134 is embedded in a cloud computing environment. The communication devices 140 transmit data collected from the sensors 121 or other data related to the operation of the vehicle 100 to the remote database 134. In embodiments, the communication devices 140 transmit information related to teleoperation to the vehicle 100. In some embodiments, the AV 100 communicates with other remote (e.g., "cloud") servers 136.

[0058] In embodiments, the remote database 134 also stores and transmits digital data (e.g., stores data such as road and street locations). This data is stored in the memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 through a communication channel.

[0059] In embodiments, the remote database 134 stores and transmits historical information (e.g., speed and acceleration profiles) related to driving attributes of vehicles that have previously traveled along the trajectory 198 at similar times of day. In one implementation, such data can be stored on the memory 144 on the vehicle 100, or transmitted from the remote database 134 to the vehicle 100 over a communication channel.

[0060] The computer processor 146 located on the vehicle 100 generates control actions algorithmically based on both real-time sensor data and a priori information, allowing the AV system 120 to perform its autonomous driving capabilities.

[0061] In embodiments, the AV system 120 includes computer peripherals 132 coupled to the computer processor 146 for providing information and reminders to a user (e.g., a passenger or a remote user) of the vehicle 100 and receiving input from the user. In embodiments, the peripherals 132 are similar to the display 212, input device 214, and cursor control 216 discussed below with respect to the computing device 200. The coupling is wireless or wired. Any two or more of the interface devices can be integrated into a single device. Figure 2

[0062] In embodiments, the AV system 120 receives and enforces a privacy level of a passenger (e.g., specified by the passenger or stored in a profile associated with the passenger). The privacy level of the passenger determines how specific information associated with the passenger (e.g., passenger comfort data, biometric data, etc.) is permitted to be used, stored in the passenger profile, and / or stored on the cloud server 136 and associated with the passenger profile. In embodiments, the privacy level specifies specific information associated with the passenger that is deleted upon completion of a ride. In embodiments, the privacy level specifies specific information associated with the passenger and identifies at least one entity that is authorized to access the information. Examples of the specified entity that is authorized to access the information can include other AVs, third-party AV systems, or any entity that can potentially access the information.

[0063] The privacy level of the passenger can be specified at least one level of granularity. In embodiments, the privacy level identifies specific information that is to be stored or shared. In embodiments, the privacy level applies to all information associated with the passenger, such that the passenger can specify that their personal information is not stored or shared. The specification of entities that are permitted to access specific information can also be specified at different levels of granularity. Different sets of entities that are permitted to access specific information can include, for example, other AVs, the cloud server 136, specific third-party AV systems, etc.

[0064] ​In embodiments, the AV system 120 or the cloud server 136 determines whether certain information associated with a passenger can be accessed by the AV 100 or other entities. For example, a third party AV system attempting to access a passenger input related to a particular spatiotemporal location must obtain authorization, e.g., from the AV system 120 or the cloud server 136, to access information associated with the passenger. For example, the AV system 120 uses the passenger's specified privacy level to determine whether the passenger input related to the spatiotemporal location can be presented to the third party AV system, the AV 100, or other AVs. This enables the passenger's privacy level to specify which other entities are allowed to receive data related to the passenger's actions or other data associated with the passenger.

[0065] Figure 2 A computer system 200 is shown. In implementations, the computer system 200 is a special-purpose computing device. The special-purpose computing device is either hard-wired to perform the techniques, or includes digital electronic devices such as at least one special-purpose integrated circuit (ASIC) or field programmable gate array (FPGA) that is persistently programmed to perform the techniques, or includes at least one general purpose hardware processor programmed to perform the techniques method. Such special-purpose computing device(s) can also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. In the implementations, the special-purpose computing device is a

[0066] In embodiments, the computer system 200 includes a bus 202 or other communication mechanism for communicating information, and a processor 204 coupled with bus 202 for processing information. The processor 204 is, for example, a general-purpose microprocessor. The computer system 200 also includes a main memory 206, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 202 for storing information and instructions to be executed by processor 204. In one implementation, the main memory 206 is used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 204. The computer system 200 can further include a

[0067] In embodiments, the computer system 200 also includes a read only memory (ROM) 208 or other static storage device coupled to the bus 202 for storing static information and instructions for the processor 204. A storage device 210, such as a magnetic disk, optical disk, solid-state drive, or a three-dimensional cross-point memory, is provided and coupled to the bus 202 for storing information and instructions.

[0068] In embodiments, the computer system 200 is coupled via the bus 202 to a display 212, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light emitting diode (LED) display, or organic light emitting diode (OLED) display for displaying information to a computer user. An input device 214, including alphanumeric and other keys, is coupled to the bus 202 for communicating information and command selections to the processor 204. Another type of user input device is a cursor control 216, such as a mouse, a trackball, a touch display, or cursor direction keys for communicating direction information and command selections to the processor 204 and for controlling cursor movement on the display 212. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.

[0069] According to one embodiment, the techniques herein are performed by the computer system 200 in response to the processor 204 executing an at least one sequence of instructions contained in an at least one memory. These instructions can be read into the main memory 206 from another storage medium, such as the storage device 210. Execution of the sequences of instructions contained in the main memory 206 causes the processor 204 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry can be used in place of or in combination with software instructions.

[0070] The term “storage media” as used herein refers to any non-transitory media that store data and / or instructions that cause a machine to operate in a specific fashion. Such storage media include non-volatile media and / or volatile media. Non-volatile media include, for example, optical disks, magnetic disks, solid-state drives, or three-dimensional cross-point memory such as the storage device 210. Volatile media include dynamic memory, such as the main memory 206. Common forms of storage media include, for example, a floppy disk, a flexible disk, a hard disk, a solid- state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, an NV-RAM, or any other memory chip or cartridge.

[0071] Storage media differs from, and is not to be confused with, transmission media. Transmission media participate in carrying data between storage media. For example, coaxial cables, copper wires and fiber optic cables are transmission media that can be used to carry digital data between computer systems coupled by a bus 202. Transmission media can also take the form of, or include, acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications.

[0072] In embodiments, various forms of media are involved in carrying one or more sequences of one or more instructions to the processor 204 for execution. For example, the instructions can initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and sends the instructions over a telephone line using a modem. A local modem in the computer system 200 receives the data on the telephone line and uses an infrared transmitter to convert the data to an infrared signal. An infrared detector in the computer system 200 receives the data carried in the infrared signal and places the data on the bus 202. The bus 202 carries the data to the main memory 206, from which the processor 204 retrieves and executes the instructions. The instructions received by the main memory 206 can optionally be stored on storage device 210 either before or after execution by the processor 204.

[0073] The computer system 200 also includes a communication interface 218 coupled to bus 202. Communication interface 218 provides a two-way data communication coupling to a network link 220 that is connected to a local network 222. For example, communication interface 218 is a integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 218 is a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links are also implemented in embodiments. In any such implementation, communication interface 218 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0074] Network link 220 typically provides data communication through at least one network to other data devices. For example, network link 220 provides a connection through local network 222 to a host computer 224 or to cloud data centers or devices operated by an Internet Service Provider (ISP) 226. ISP 226 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 228. Local network 222 and Internet 228 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 220 and through communication interface 218, which carry the digital data to and from computer system 200, are example forms of transmission media.

[0075] Computer system 200 communicates messages over the network(s), network link 220, and communication interface 218. In embodiments, computer system 300 receives code to be used to process. The received code is executed by processor 204 as it is received, and / or stored in storage device 210, or other non-volatile storage for later execution.

[0076] Autonomous vehicle architecture

[0077] Figure 3 An example architecture 300 is shown for an autonomous vehicle (e.g., the vehicle 100 shown. Figure 1 The architecture 300 includes a perception module 302 (sometimes referred to as perception circuitry), a planning module 304 (sometimes referred to as planning circuitry), a control module 306 (sometimes referred to as control circuitry), a localization module 308 (sometimes referred to as localization circuitry), and a database module 310 (sometimes referred to as database circuitry). The modules each play a role in the operation of the vehicle 100. Collectively, the modules 302, 304, 306, 308, and 310 can be referred to as processing circuitry. Figure 1 The AV system 120 shown. In some embodiments, any of the modules 302, 304, 306, 308, and 310 is a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., at least one microprocessor, microcontroller, application-specific integrated circuit [ASIC], hardware memory device, other type of integrated circuit, other type of computer hardware, or a combination of any or all of these). Each of the modules 302, 304, 306, 308, and 310 is sometimes referred to as processing circuitry (e.g., computer hardware, computer software, or a combination of both). A combination of any or all of the modules 302, 304, 306, 308, and 310 is also an example of processing circuitry.

[0078] In use, the planning module 304 receives data representing a destination 312, and determines data representing a trajectory 314 (sometimes referred to as a route) that the vehicle 100 can travel in order to reach (e.g., arrive at) the destination 312. In order for the planning module 304 to determine the data representing the trajectory 314, the planning module 304 receives data from the perception module 302, the localization module 308, and the database module 310.

[0079] The perception module 302 identifies nearby physical objects using, for example, at least one sensor 121 also shown in Figure 1 The objects are classified (e.g., grouped into types such as pedestrians, bicycles, cars, traffic signs, etc.), and a scene description including the classified objects 316 is provided to the planning module 304.

[0080] The planning module 304 also receives data representing the location 318 of the AV from the positioning module 308. The positioning module 308 determines the AV location by calculating location using data from sensor 121 and data (e.g., geographic data) from database module 310. For example, the positioning module 308 uses data from GNSS (Global Navigation Satellite System) sensors and geographic data to calculate the longitude and latitude of the AV. In embodiments, the data used by the positioning module 308 includes high-accuracy maps with lane geometry properties, maps describing road network connectivity properties, maps describing lane physical properties (such as traffic speed, traffic volume, number of vehicle and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or combinations thereof), and maps describing spatial locations of road features (such as intersections, traffic signs, or various types of other traffic signals). In embodiments, high-accuracy maps are constructed by automatically or manually annotating data onto low-accuracy maps.

[0081] The control module 306 receives data representing trajectory 314 and data representing AV position 318, and operates the AV control functions 320a-320c (e.g., steering, throttle, braking, ignition) in a manner that will cause the vehicle 100 to travel along trajectory 314 to reach destination 312. For example, if trajectory 314 includes a left turn, the control module 306 will operate the control functions 320a-320c in such a way that the steering angle of the steering function will cause the vehicle 100 to turn left, and the throttle and brake will cause the vehicle 100 to pause and wait for passing pedestrians or vehicles before making the turn.

[0082] Autonomous Vehicle Input

[0083] Figure 4 The sensing module 302 is shown. Figure 3 The inputs used are 402a-402d (e.g., Figure 1 Examples of sensor 121 and outputs 404a-404d (e.g., sensor data) are shown. One input 402a is a LiDAR (light detection and ranging) system (e.g., Figure 1 The LiDAR system shown is 123. LiDAR is a technique that uses light (e.g., a beam of light such as infrared light) to obtain data related to physical objects in its line of sight. The LiDAR system produces LiDAR data as output 404a. For example, LiDAR data is a collection of 3D or 2D points (also called point clouds) used to construct a representation of environment 190.

[0084] Another input 402b is a RADAR (Radar) system. RADAR is a technology that uses radio waves to obtain data about nearby physical objects. RADAR can obtain data about objects that are not in the line of sight of the LiDAR system. The RADAR system produces RADAR data as output 404b. For example, the RADAR data is at least one radio frequency electromagnetic signal (e.g., a radar return) that is used to construct a representation of the environment 190.

[0085] Another input 402c is a camera system. The camera system uses at least one camera (e.g., a digital camera that uses a photosensor such as a charge-coupled device [CCD]) to acquire information about nearby physical objects. The camera system produces camera data as output 404c. The camera data is often in the form of image data (e.g., data in an image data format such as RAW, JPEG, PNG, etc.). In some examples, the camera system has multiple independent cameras, e.g., for the purpose of stereoscopic imagery (stereo vision), which enables the camera system to perceive depth. Although the objects perceived by the camera system are described here as being "nearby," this is relative to the AV. In some embodiments, the camera system is configured to "see" objects that are far away (e.g., up to 1 kilometer or more in front of the AV). Thus, in some embodiments, the camera system has features such as sensors and lenses that are optimized for perceiving objects that are far away.

[0086] Another input 402d is a traffic light detection (TLD) system. The TLD system uses at least one camera to obtain information about traffic lights, street signs, and other physical objects that provide visual navigational information. The TLD system produces TLD data as output 404d. The TLD data is often in the form of image data (e.g., data in an image data format such as RAW, JPEG, PNG, etc.). The TLD system differs from a system that contains a camera in that the TLD system uses a camera with a wide field of view (e.g., using a wide-angle lens or a fisheye lens) to obtain information about as many physical objects that provide visual navigational information as possible, so that the vehicle 100 has access to all relevant navigational information provided by those objects. For example, the TLD system has a field of view of about 120 degrees or more.

[0087] In some embodiments, the outputs 404a-404d are combined using sensor fusion techniques. Thus, the individual outputs 404a-404d are provided to other systems of the vehicle 100 (e.g., to the perception system 106, the planning system 108, and / or the control system 110) as well as to the environment representation system 102. Figure 3The combined outputs can be provided to other systems in the form of a single combined output or multiple combined outputs of the same type (e.g., using the same combination technique or combining the same outputs or both) or a single combined output or multiple combined outputs of different types (e.g., using different respective combination techniques or combining different respective outputs or both), as illustrated by the planning module 304), or can be provided to other systems in the form of a single combined output or multiple combined outputs of the same type (e.g., using the same combination technique or combining the same outputs or both) or a single combined output or multiple combined outputs of different types (e.g., using different respective combination techniques or combining different respective outputs or both). In some embodiments, an early fusion technique is used. Early fusion techniques are characterized by combining the outputs prior to applying at least one data processing step to the combined outputs. In some embodiments, a late fusion technique is used. Late fusion techniques are characterized by combining the outputs after applying at least one data processing step to the individual outputs.

[0088] Towing system for assisted vehicles

[0089] The towing system is used to attach an assisted vehicle (e.g., a towed vehicle) to a host vehicle (e.g., an autonomous vehicle [AV]). The towed vehicle connected to the AV obstructs the sensor line of sight through which the AV sensors receive information about the surrounding environment. Accordingly, the towing system includes at least one sensing device (sometimes referred to as a sensor) reversibly attached to a surface or attachment point on the towed vehicle, thereby providing additional data to the AV and enhancing the AV’s perception of the surrounding environment. The AV utilizes the additional data in trajectory computation and path planning to improve the positioning and path planning accuracy of the combined vehicle system and overall safety.

[0090] Figure 5 is a schematic diagram of components of the towing system 500, including at least one sensing device 502 (individually referred to as a sensing device 502 and collectively referred to as a plurality of sensing devices 502), at least one processor 504, a memory 506, a power source 508, and a communication array 510. The sensing device 502 can include Figure 4 any of the input systems described in the perception module 302 (e.g., inputs 402a-402d, LiDAR, RADAR, or a camera) or any sensor type with which the perception module 302 is configured to communicate (e.g., receive outputs from). Additional examples of sensing devices 502 include an accelerometer, a magnetometer, a night vision camera (e.g., an infrared camera), a GPS receiver, an ultrasonic sensor, a TOF sensor, a temperature sensor, a humidity sensor, and / or a precipitation sensor, among others. In some embodiments, the towing system 500 does not include a processor 504. In such examples, the towing system 500 can provide signals received from the at least one sensing device 502 to the AV for processing using a local processor (such as the processor 204 Figure 2 ) and the like).

[0091] The sensing device 502 also includes at least one attachment device for temporarily attaching the sensing device 502 to the towed vehicle 615. The attachment device can include a mechanism or combination of mechanisms that can be temporarily attached to a surface or docking location. Examples of attachment devices include magnets, hitches, couplers, suction cups, and the like. The attachment device provides temporary connection of the sensing device 502 to the towed vehicle 615 at any location (e.g., a location selected by a user).

[0092] In embodiments, the attachment device provides temporary connection between multiple sensing devices 502 so that a sensing device 502 can be detached from the towed vehicle 615 and a different sensing device 502 can be attached (e.g., replaced) to the towed vehicle 615 using the same attachment device. For example, the attachment device can include a female port and the input system can include a male plug that establishes temporary connection to the attachment device when the male plug is inserted into the female port, which can then be attached to the towed vehicle 615. In this embodiment, the installed attachment device can remain on the towed vehicle 615 and the input system is installed or replaced. In embodiments, the towed vehicle 615 includes at least one semi-permanent attachment, such as a hitch or coupler to which a removable input system is attached.

[0093] Referring again to Figure 5 , the processor 504 processes instructions for execution within the towing system 500, including instructions stored in the memory 506. The memory 506 stores information within the towing system 500. In embodiments, the memory 506 is one or more volatile memory units. In embodiments, the memory 506 is one or more non-volatile memory units. The memory 506 can also be other forms of computer- readable media, such as magnetic or optical disks, and the like.

[0094] The power source 508 includes, for example, a battery, a battery pack, and / or a socket adapter, an AC-to-DC converter, a DC-to-AC converter, a power regulator, a capacitor bank, and / or one or more interfaces for providing power to the towing system 500. In embodiments, the towing system 500 receives primary or supplemental power from an AV through a temporary electrical connection (e.g., a wired connection).

[0095] The communication array 510 is a radio that provides any technically appropriate communication interface, including but not limited to multiple communication interfaces such as Wi-Fi, Bluetooth, or copper wire connections. In embodiments, the communication array 510 is a radio that communicates on millimeter wavelength frequencies (e.g., mm-wave radios) that correspond to a wavelength range between 10 mm (e.g., 30 GHz) and 1 mm (e.g., 300 GHz). In embodiments, the communication array 510 is a 5G device. In examples, the communication array 510 is more than one communication interface.

[0096] The processor 504 processes the input received from the sensing devices 502 into output received by the communication interface 218 of the AV computer system 200 from the communication array 510. The perception module 302 of the AV architecture 300 uses the output of the traction system 500 to receive data associated with reflected signals (e.g., point clouds, radar echoes, light signals) from the environment and identify nearby objects (e.g., vehicles, people, street signs). In some embodiments, the AV localization module 308 also uses the output of the traction system 500 as supplemental information to determine the location of the towed vehicle relative to the environment or relative to the AV. In embodiments, the processor 504 receives the data associated with the reflected signals and computes a localization estimate of the traction system 500 that the communication array 510 provides to the AV. The AV localization module 308 utilizes the localization estimate of the traction system 500 in determining a localization estimate of the AV.

[0097] Generally, the sensing devices 502 can be attached to any surface or point (e.g., roof, door, side, and / or window, etc.) that is capable of maintaining a temporary connection with the sensing devices 502 and also provide a clear line of sight for reflected signals from the surrounding environment.

[0098] In examples, the traction system 500 includes multiple sensing devices 502 in a single housing and connects to the towed vehicle using a single attachment mechanism. In a second example, the multiple sensing devices 502 of the traction system 500 include respective processors 504, memories 506, power sources 508, and communication arrays 510 for independent operation when in communication with the traction system 500.

[0099] In embodiments, the traction system 500 determines at least one distance measurement and / or relative position to determine at least one dimension of the towed vehicle. For example, the traction system 500 determines a relative position of at least one sensing device 502 relative to the AV. The relative position includes spatial information, such as a horizontal position or a vertical position relative to at least one sensor of the AV, etc.

[0100] The processor 504 of the towing system 500 computes a distance between at least one sensing device 502 and the AV based on at least the relative position of the sensing device 502. For example, a RADAR sensing device receives reflected signals and the processor 504 makes a time-of-flight computation based on the reflected signals to determine the relative position and distance to the AV. As a second example, a LiDAR sensing device 502 of the towing system 500 determines a point cloud of the surrounding environment, or at least two camera sensing devices 502 receive images from the surrounding environment, and the processor 504 of the towing system 500 makes a computation based on the received stereo optical information to determine the dimensions of the towed vehicle.

[0101] The towing system 500 receives reflected signals from the sensing devices 502 and processes them individually into outputs to provide to the AV. In an embodiment, the processor 504 combines the reflected signal outputs into a data structure representing a merged reflected signal (e.g., a composite of the individual reflected signals) and provides the data structure representing the merged reflected signal to the AV (e.g., to one or more systems included in the AV). In another embodiment, the processor 504 combines the reflected signal outputs into a data structure representing a merged reflected signal and determines a position estimate of the towed vehicle based on the merged reflected signal and provides the position estimate to the AV.

[0102] The towing system 500 computes at least one dimension of the towed vehicle based on the determined relative position and distance to the AV. In an example, the towing system 500 includes at least one sensing device 502 capable of perceiving an entire dimension (e.g., length) of the towed vehicle. The at least one sensing device 502 receives reflected signals from the towed vehicle and the processor 504 determines the entire dimension from the reflected signals. In some embodiments, the towing system 500 includes at least two sensing devices 502, each capable of perceiving a partial dimension of the towed vehicle. The at least two sensing devices 502 receive reflected signals corresponding to the partial dimensions and provide the signals to the processor 504, which then determines the entire dimension from the processor 504.

[0103] The dimension includes a spatial dimension value (e.g., feet, meters) corresponding to a height, length, or width of the towed vehicle. In another embodiment, the towing system 500 includes a table (e.g., database, lookup table) in the memory 506 that includes at least one dimension value corresponding to a towed vehicle classification (e.g., shipping container, tractor-trailer, boat, articulated bus). The processor 504 of the towing system 500 computes at least one dimension value of the towed vehicle and determines the towed vehicle classification from the table stored in the memory 506 and provides the towed vehicle classification to the AV.

[0104] In some embodiments, the towing system 500 calculates the distance from the towed vehicle to a second location. In a first example, the towing system 500 is attached to the towed vehicle at a relative position and distance relative to the AV, and the second location is a nearby second vehicle. At least one sensing device 502 of the towing system 500 (e.g., a camera, LiDAR receiver) receives reflected signals from the nearby second vehicle, and a processor 504 determines a second distance to the second vehicle. The towing system 500 provides this second distance to the AV.

[0105] In the second example, the second vehicle includes a communication array (e.g., a GPS transceiver, Wi-Fi, or mm-wave radio) capable of communicating with the traction system 500. The second vehicle establishes a communication connection with the communication array 510 and provides a positioning estimate of the second vehicle to the traction system 500. For example, the traction system 500 then provides the positioning estimate of the second vehicle to the AV for further processing in path planning and control.

[0106] In the third example, the second location can be a fixed location, such as a trailer dock, parking spot, or bay. At least one sensing device 502 of the traction system 500 receives a reflected signal corresponding to the fixed location. In some embodiments, the fixed location may include at least one reference marker (e.g., a reflective marker, high-visibility paint, or symbol) corresponding to the fixed location. Optionally, the fixed location may include a communication array (such as a GPS transceiver, Wi-Fi, or mm-wave radio beacon) for establishing a communication connection with the traction system 500. The traction system 500 receives or determines a positioning estimate from the fixed location communication array based on the received signal and provides the positioning estimate to the AV.

[0107] Exemplary embodiments

[0108] Figure 6A The diagram shows multiple integrated sensors 612a-612c for sensing the surrounding environment (e.g., Figure 1 The example AV 610 (e.g., with sensor 121 shown) Figure 1 (The AV 6100 shown is the same as or similar to the AV 100 shown). Sensors 612a-612c provide sensor data to the sensing module 302 of the AV 610, which in turn provides the sensor data to a request module (such as a positioning module 308) in the AV 610. The depicted AV 610 is a tractor-trailer truck with a fifth wheel hook-up device for connection to a towed semi-trailer. Other examples of the AV 610 may include passenger vehicles, motorized tugs, or other commercial or industrial trucks.

[0109] Example sensors 612a-612c include a LiDAR system 612a, a RADAR system 612b, and a camera system 612c. Each sensor 612a-612c receives reflected signals (e.g., LiDAR point clouds, RADAR echoes, optical signals) from objects (such as vehicles or pedestrians) depending on the line-of-sight map to nearby objects. For example, LiDAR system 612a receives reflected laser signal 613a, RADAR system 612b receives reflected radio signals, and camera system 612c receives reflected light signal 613c. Using these reflected signals (e.g., 613a, 613c), the perception module 302 of AV 610 establishes an estimate of the surrounding environment.

[0110] Figure 6B Showing includes Figure 6A Example AV 610 and example combined vehicle 600 connected to AV 610 via temporary connections (such as hitchs or fifth wheels) to a towed vehicle 615. As depicted, the towed vehicle 615 is a semi-trailer, but other examples of towed vehicles include two-wheeled trailers, refrigerated trailers, flatbed trailers, camping trailers, recreational vehicles, automated racking trailers, grain hopper trailers, livestock trailers, towed boats and / or towed second vehicles, etc. In embodiments, connections may include electrical connections for supplying power to and / or receiving electrical signals from the towed vehicle 615 and / or the towing system 620.

[0111] The forward-facing sensors of AV 610 (such as RADAR system 612b) maintain their respective lines of sight to receive reflected signals from nearby objects. However, the rearward-facing sensors responsible for determining the environment behind AV 610 experience reduced visibility and receive reflected signals from the surface of the towed vehicle 615 rather than from objects in the environment. For example, reflected signals 613a, 613c are reflected from the surface of the towed vehicle 615, partially or completely obscuring the surrounding environment. This creates areas lacking environmental information for AV 610 (e.g., blind spots). Removable sensor outputs provide AV 610 with additional information about the surrounding environment and may include information related to at least a portion of the blind spots.

[0112] like Figure 6BAs shown, the towed vehicle 615 includes a tow system 620 that includes three sensing devices 622a-622c. The tow system 620 includes a LiDAR sensing device 622a, a RADAR sensing device 622b, and a camera sensing device 622c attached to the rear end of the towed vehicle 615. The LiDAR sensing device 622a is attached to an upper surface, the camera sensing device 622c is attached to a side surface proximate the upper surface, and the RADAR sensing device 622b is attached to a rear surface of the towed vehicle 615.

[0113] The positioning of the sensing devices 622a-622c on the towed vehicle 615 establishes clear lines of sight such that the sensing devices 622a-622c receive reflected signals from the surrounding environment, shown by reflected signals 623a and 623c. The sensing devices 622a-622c receive the reflected signals and produce outputs corresponding to the sensing device type (e.g., LiDAR data, RADAR data, or camera data).

[0114] The sensing devices 622a-622c receive reflected signals from the rear surface of the towed vehicle 615 when attached proximate the rear surface. In an embodiment, the tow system 500 utilizes the processor 504 to detect that at least one component of the towed vehicle 615 is experiencing a partial or complete failure of operation (e.g., a malfunction). For example, the camera sensing device 622c detects light emitted from a working light bulb (e.g., a taillight) proximate the rear surface of the towed vehicle 615, but does not detect light emitted from a paired working light bulb opposite the functioning light bulb. The tow system 500 sends a light malfunction notification to the AV indicating the malfunctioning function, and the AV can display the notification to the user.

[0115] In another example, a malfunctioning tire (e.g., flat, damaged, or low pressure) on a side of the towed vehicle 615 causes high amplitude random motion on the side surface. This motion is detected by a first sensing device 502 attached to the same side surface, but not by a second sensing device 502 attached to the opposite side surface. The tow system 500 sends a tire malfunction notification to the AV for display to the user. In another example, the malfunctioning tire causes debris to appear within the field of view of at least one sensing device 502. The tow system 500 detects the debris, and either alone or in combination with the detected random motion signal, sends a tire malfunction notification to the AV for display to the user.

[0116] Figure 6C A top view of an example combination vehicle 600 including an AV 610 and a towed vehicle 615. Figure 6B A top view of an example combination vehicle 600 including an AV 610 and a towed vehicle 615. Figure 6CA towing system 620 is shown that includes an additional camera sensing device 622d attached to the surface opposite the camera sensing device 622c. Reflected signals 623a, 623c, and 623d are shown corresponding to unobstructed lines of sight to nearby objects and the surrounding environment.

[0117] The towed vehicle 615 complicates the determination of the trajectory 314 of the AV 610 by adding an additional dimension to the AV 610 articulating about the attachment point of the AV 610 and the towed vehicle 615. In the case where the AV 610 determines and drives along a trajectory 314 on which the towed vehicle 615 is being pulled behind the AV 610 (e.g., driving forward), the towing system 620 provides output to the AV 610 related to the relative position of the AV 610 to the surrounding environment relative to the towed vehicle 615 while the AV 610 operating control functions 320a-320c to drive along the trajectory 314.

[0118] In embodiments, the AV 610 determines a trajectory 314 on which the towed vehicle 615 is pushed by the AV 610 (e.g., driving in reverse). In such embodiments, the towing system 620 includes at least one additional processor (e.g., a second processor) communicatively coupled to the processor 504 that operates control functions 320a-320c in response to output received from the connected sensing devices. In embodiments, the second processor of the towing system 620 (or the first processor 504) sends control signals to independently operate control functions 320a-320c to drive along the trajectory 314.

[0119] Figure 7 An AV 700 is depicted with an integrated LiDAR sensor 712a that receives obstructed reflected signals from a towed vehicle 715. A towing system 720 is attached to the towed vehicle 715 that includes a LiDAR sensing device 722a on an upper surface, a RADAR sensing device 722b on a rear surface, and two camera sensing devices 722c and 722d arranged in stereo on either side surface. In Figure 7 In the example of FIG. 7B, the direction of travel is shown (e.g., reverse motion). The towing system 720 receives unobstructed reflected signals from the environment representing relative position and velocity in the direction of travel and processes the received input into output. The AV 700 receives the output from the towing system 720 and computes a trajectory 314. The AV 700 operates control functions 320a-320c to drive on the trajectory 314. In embodiments, the towing system 720 computes the trajectory 314 and communicates instructions to the AV 700 to control functions 320a-320c to drive on the trajectory 314.

[0120] Figure 8 is a flowchart representing a process 800 for providing information corresponding to a secondary vehicle to a reversibly attached primary vehicle. In embodiments, the secondary vehicle is the secondary vehicle 615 shown in FIG. 6, the primary vehicle is the AV 610 or Figure 1 the AV 100 in FIG. 6, and the process 800 is performed by a processor such as the processor 504 shown in FIG. 5 or Figure 5 the perception module 402 shown in FIG. 4. Figure 4

[0121] The towing system processor receives data (e.g., reflection signals) corresponding to an environment of the secondary vehicle from at least one sensing device (802). The data corresponding to the environment includes at least one reflection signal (such as a LiDAR point cloud, a RADAR echo, or an optical signal) received by at least one sensing device of the towing system. In embodiments, the at least one sensing device includes a sensing device of the towing system, e.g., a LiDAR, a RADAR, a camera, or any sensing device described herein. Figure 5

[0122] The towing system processor determines at least one dimension of the secondary vehicle and a relative position of the at least one sensing device with respect to the primary vehicle based on the data received from the reflection signals from the environment of the secondary vehicle (804). In some embodiments, the towing system uses the reflection signals from the environment to determine the at least one dimension and the relative position. In some embodiments, the towing system receives the reflection signals from the environment and generates a merged reflection signal based on the reflection signals.

[0123] In some embodiments, the towing system processor determines a vehicle classification based on the determined dimension and the relative position of the secondary vehicle with respect to the primary vehicle. In some embodiments, the towing system processor determines a sensor position at which the at least one sensing device is releasably attached based on the determined dimension and the relative position of the secondary vehicle with respect to the primary vehicle.

[0124] The system processor provides the at least one dimension and the relative position of the at least one sensing device with respect to the primary vehicle (806). In some embodiments, the towing system includes a communication array for providing information to the primary vehicle, such as a communication array 510 that communicates on millimeter wavelength frequencies (e.g., mmWave radios, 5G devices), etc. In some embodiments, the information provided to the primary vehicle is a localization estimate based on the merged reflection signal in 804. In some embodiments, the information provided to the primary vehicle is the vehicle classification or the sensor position determined in 804.

[0125] ​​In the foregoing description, embodiments of the application have been described with reference to numerous specific details that can vary with embodiments. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the application, and what is intended by the applicants to be the scope of the application, is the literal and equivalent scope of the claims issued from this application in its broadest form, including any subsequent amendments. Any definitions expressly set forth herein for terms and phrases used herein should be understood as applying to the terms and phrases as used throughout the claims. In addition, where the specification states a use before an element or a use after an element, these phrases refer to making use of the element, transporting the element, discarding the element, or similar actions involving the element. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in the introductory clauses of the claims.

Claims

1. A system for a vehicle, comprising: At least one first sensor includes at least one attachment configured to be removably attached to an auxiliary vehicle, the auxiliary vehicle being configured to be attached to a main vehicle, the main vehicle including at least one second sensor for sensing the environment; At least one computer-readable medium storing computer-executable instructions; At least one first processor, communicatively coupled to the at least one first sensor and configured to execute the computer-executable instructions, the execution performing operations including: Data associated with reflected signals corresponding to the two-dimensional or three-dimensional environment of the auxiliary vehicle is received from the at least one first sensor; The at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle are determined based on data captured from the environment of the auxiliary vehicle by at least one first sensor removably attached to the auxiliary vehicle, rather than on data captured by the at least one second sensor. as well as The at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle are provided to the main vehicle.

2. The system according to claim 1, further comprising: At least one second processor communicatively coupled to the at least one first processor, wherein the second processor is configured to: The main vehicle is operated based on data received from the at least one first sensor.

3. The system according to claim 1 or 2 further includes a communication array.

4. The system according to claim 3, wherein, The communication array is configured to transmit and receive data in a frequency range between 30 GHz and 300 GHz.

5. The system according to claim 4, wherein, Providing the relative position of the at least one dimension and the at least one first sensor to the main vehicle includes: The communication array is used to provide the relative positions of the at least one dimension and the at least one first sensor to the main vehicle.

6. The system according to claim 1 or 2, wherein, The execution also includes performing the following operations: Detecting objects based on the environment of the auxiliary vehicle.

7. The system according to claim 1 or 2, wherein, The execution also includes performing the following operations: Detect at least one fault in the auxiliary vehicle.

8. The system according to claim 1 or 2, further comprising: A power source configured to provide power to the at least one first sensor, the at least one computer-readable medium, and the at least one first processor.

9. The system according to claim 1 or 2, wherein, Receiving data corresponding to the environment includes: Data associated with the reflected signal is received from each of the at least one first sensor; A merged reflected signal is generated based on the reflected signals received from each of the at least one first sensor; A location estimate is determined based on the merged reflected signals; and The positioning estimate is provided to the main vehicle.

10. The system according to claim 1 or 2, wherein, The determination also includes: The vehicle classification of the auxiliary vehicle is determined based on at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle. The sensor position is determined based on at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle, wherein the at least one first sensor is releasably attached to the sensor position; and The vehicle classification and the sensor location are provided to the main vehicle.

11. A method for a vehicle, comprising: The auxiliary vehicle receives data from at least one first sensor configured to be removably attached to an auxiliary vehicle, which is associated with reflected signals corresponding to a two-dimensional or three-dimensional environment of the auxiliary vehicle, the auxiliary vehicle being configured to be attached to a main vehicle, the main vehicle including at least one second sensor for sensing the environment. The at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle are determined based on data captured from the environment of the auxiliary vehicle by at least one first sensor removably attached to the auxiliary vehicle, rather than on data captured by the at least one second sensor. as well as The at least one dimension of the auxiliary vehicle and the relative position of the at least one first sensor with respect to the main vehicle are provided to the main vehicle.

12. A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a first device, the at least one program comprising instructions that, when executed by the at least one processor, cause the first device to perform the method according to claim 11.

13. An auxiliary transport vehicle, comprising: At least one processor; as well as At least one memory having instructions stored thereon, which, when executed by the at least one processor, cause the at least one processor to perform the method according to claim 11.

14. A computer program product comprising at least one program for execution by at least one processor of a first device, the at least one program comprising instructions that, when executed by the at least one processor, cause the first device to perform the method according to claim 11.

Images