Method and system for camera arrangements for vehicular object detection and avoidance

HK40137309APending Publication Date: 2026-09-11WAYMO LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK42026126354
Authority / Receiving Office
HK · HK
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-12-20
Filing Date
2026-07-20
Publication Date
2026-09-11

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Example embodiments relate to a camera arrangement for vehicle object detection and avoidance. An example system includes a vehicle and at least one camera of a first camera type attached to the vehicle. The system also includes a plurality of cameras of a second camera type attached to the vehicle. Further, the system includes a plurality of cameras of a third camera type attached to the vehicle. Further, the system includes a computing device communicatively coupled to the at least one camera of the first camera type, the plurality of cameras of the second camera type, and the plurality of cameras of the third camera type. The computing device is configured to identify an object located within a first distance range from the vehicle, an object located within a second distance range from the vehicle, and an object located within a third distance range from the vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511920591.9 (22) Application Date 2025.12.18 (30) Priority Data 18 / 989,446 2024.12.20 US (71) Applicant: W.M. W.M. LLC Address: California, USA (72) Inventors: L. Ian S. Shamah J. Ditmer N. Daria (74) Patent Agency: Beijing Liu Shen Law Firm 11105 Patent Attorney: Jin Yujie (51) Int.Cl. B60R 1 / 22 (2022.01) B60R 11 / 04 (2006.01) (54) Invention Title: Method and System for Camera Arrangement for Vehicle Object Detection and Avoidance (57) Abstract: Examples: Embodiments relate to a camera arrangement for vehicle object detection and avoidance. An example system includes a vehicle and at least one camera of a first camera type attached to the vehicle. The system also includes multiple cameras of a second camera type attached to the vehicle. Furthermore, the system includes multiple cameras of a third camera type attached to the vehicle. Additionally, the system includes a computing device communicatively coupled to the at least one camera of the first camera type, the multiple cameras of the second camera type, and the multiple cameras of the third camera type. The computing device is configured to identify objects located within a first distance range from the vehicle, objects located within a second distance range from the vehicle, and objects located within a third distance range from the vehicle. Claims 5 pages, Description 34 pages, Drawings 45 pages, CN 122253769 A 2026.06.23 CN 1 22 25 37 69 A 1. A system comprising: a vehicle; at least one camera of a first camera type attached to the vehicle, wherein each camera of the first camera type has a first field of view; multiple cameras of a second camera type attached to the vehicle, wherein each camera of the second camera type has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle; A plurality of cameras of a third camera type are attached to the vehicle, wherein each camera of the third camera type has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° relative to the vehicle in terms of yaw, wherein the first field of view spans a smaller angle relative to the vehicle than the third field of view, and wherein the third field of view spans a smaller angle relative to the vehicle than the second field of view; and a computing device communicatively coupled to the at least one camera of the first camera type, the plurality of cameras of the second camera type, and the plurality of cameras of the third camera type, wherein the computing device is configured to:Based on one or more first images captured by at least one camera of a first camera type, objects located within a first distance range from the vehicle are identified; based on one or more second images captured by a plurality of cameras of a second camera type, objects located within a second distance range from the vehicle are identified; and based on one or more third images captured by a plurality of cameras of a third camera type, objects located within a third distance range from the vehicle are identified, wherein the third distance range includes distances from the vehicle that are farther than those included in the second distance range, and wherein the first distance range includes distances from the vehicle that are farther than those included in the third distance range. 2. The system of claim 1, wherein a predetermined camera among the plurality of cameras of the second camera type: includes a neutral density filter through which an image is captured; is oriented relative to the carrier to capture an image from a pitch angle greater than 20° relative to the carrier; and is configured to capture a pair of image frames, wherein a first image frame in the pair of image frames is captured at a first exposure time, wherein the first exposure time is set by an automatic exposure setting, wherein a second image frame in the pair of image frames is captured at a second exposure time, and wherein the second exposure time is longer than 1 / 60 of a second. 3. The system of claim 2, further comprising a light detection and ranging (LiDAR) device attached to the vehicle and configured to generate a point cloud indicating distances to objects in the environment surrounding the vehicle, wherein identifying the objects located within a second distance range from the vehicle based on one or more second images captured by the plurality of cameras of the second camera type comprises: aligning a first image frame with the point cloud; overlapping the aligned first and second image frames to generate an overlapping image; and performing object recognition using the overlapping image and the point cloud. 4. The system of claim 1, wherein the plurality of cameras of the second camera type comprises: a camera oriented in a forward direction relative to the vehicle's direction of travel; a camera oriented in a rearward direction relative to the vehicle's direction of travel; and two cameras each oriented in one or more lateral directions relative to the vehicle's direction of travel. 5. The system of claim 1, wherein the at least one camera of the first camera type comprises two cameras each oriented in a forward direction relative to the direction of travel of the vehicle. 6. The system of claim 1, wherein the at least one camera of the first camera type comprises three cameras each oriented in a forward direction relative to the direction of travel of the vehicle, wherein the plurality of cameras of the second camera type comprises:A camera oriented in the forward direction relative to the vehicle's direction of travel; a camera oriented in the rearward direction relative to the vehicle's direction of travel; and two cameras each oriented in one or more lateral directions relative to the vehicle's direction of travel, wherein the plurality of cameras of the third camera type comprises: a camera oriented in the forward direction relative to the vehicle's direction of travel; two cameras each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and two cameras each oriented in one or more lateral directions relative to the vehicle's direction of travel. 7. The system of claim 6, wherein at least one of the at least one camera of the first camera type is attached adjacent to the vehicle to at least one of the plurality of cameras of the third camera type. 8. The system of claim 1, wherein the at least one camera of the first camera type comprises four cameras each at least partially oriented in a forward direction relative to the direction of travel of the vehicle, wherein the plurality of cameras of the second camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; two cameras each oriented in a rearward direction relative to the direction of travel of the vehicle; and two cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle, and wherein the plurality of cameras of the third camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; two cameras each at least partially oriented in the rearward direction relative to the direction of travel of the vehicle; and two cameras each at least partially oriented in one or more lateral directions relative to the direction of travel of the vehicle. 9. The system of claim 8, wherein at least three cameras of the first camera type are each attached to the vehicle adjacent to at least three cameras of the third camera type, and wherein at least two cameras of the second camera type are each attached to the vehicle adjacent to at least two cameras of the third camera type. 10. The system of claim 1, wherein the at least one camera of the first camera type comprises three cameras each at least partially oriented in a forward direction relative to the direction of travel of the vehicle; wherein the plurality of cameras of the second camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; a camera oriented in a rearward direction relative to the direction of travel of the vehicle; and two cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle; andThe plurality of cameras of the third camera type include: a camera oriented in the forward direction relative to the vehicle's direction of travel; two cameras each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and three cameras each at least partially oriented in one or more lateral directions relative to the vehicle's direction of travel. 11. The system of claim 10, wherein at least one camera of the first camera type is attached to the vehicle adjacent to at least one camera of the third camera type, and wherein at least two cameras of the second camera type are each attached to the vehicle adjacent to at least two cameras of the third camera type. 12. The system of claim 1, wherein the at least one camera of the first camera type comprises two cameras each oriented in a forward direction relative to the direction of travel of the vehicle, wherein the plurality of cameras of the second camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; a camera oriented in a rearward direction relative to the direction of travel of the vehicle; and two cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle, and wherein the plurality of cameras of the third camera type comprises: two cameras each at least partially oriented in the forward direction relative to the direction of travel of the vehicle; two cameras each at least partially oriented in the rearward direction relative to the direction of travel of the vehicle; and four cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle. 13. The system of claim 12, further comprising a plurality of cameras of a fourth camera type, wherein the plurality of cameras of the fourth camera type includes two cameras attached to the vehicle, wherein each camera of the fourth camera type has a fourth field of view, wherein the fourth field of view spans a larger angle in yaw relative to the vehicle than a first field of view, wherein the fourth field of view spans a smaller angle in yaw relative to the vehicle than a third field of view, wherein the computing device is further configured to identify objects located within a fourth distance range based on one or more images captured by the plurality of cameras of the fourth camera type, wherein the fourth distance range includes distances from the vehicle that are farther than distances included in a third distance range, and wherein the first distance range includes distances from the vehicle that are farther than distances included in the fourth distance range. 14. The system of claim 12, wherein at least one of the plurality of cameras of the second camera type: includes a neutral density filter through which the image is captured. (Claims 3 / 5, page 4, CN 122253769 A)The system is oriented relative to the vehicle to capture images from a pitch angle greater than 5° relative to the vehicle and to have a field of view that spans a larger angle in yaw relative to the vehicle than at least one other camera of the second camera type. 15. The system of claim 13, wherein at least two of the at least one camera of the first camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the fourth camera type, wherein at least two of the at least one camera of the first camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the third camera type, wherein at least two of the plurality of cameras of the fourth camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the third camera type, and wherein at least two of the plurality of cameras of the third camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the second camera type. 16. The system of claim 1, wherein the at least one camera of the first camera type comprises two cameras each oriented in a forward direction relative to the direction of travel of the vehicle, wherein the plurality of cameras of the second camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; a camera oriented in a rearward direction relative to the direction of travel of the vehicle; and two cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle, and wherein the plurality of cameras of the third camera type comprises: a camera oriented in the forward direction relative to the direction of travel of the vehicle; and four cameras each oriented in one or more lateral directions relative to the direction of travel of the vehicle. 17. The system of claim 16, wherein at least one of the plurality of cameras of the third camera type: comprises a neutral density filter through which an image is captured; and is oriented relative to the vehicle to capture an image from a pitch angle greater than 20° relative to the vehicle. 18. The system of claim 16, further comprising a plurality of cameras of a fourth camera type, wherein the plurality of cameras of the fourth camera type includes four cameras attached to the vehicle, wherein each camera of the fourth camera type has a fourth field of view, wherein the fourth field of view spans a larger angle in yaw relative to the vehicle than a first field of view, wherein the fourth field of view spans a smaller angle in yaw relative to the vehicle than a third field of view, wherein the computing device is further configured to identify objects located within a fourth distance range based on one or more images captured by the plurality of cameras of the fourth camera type, wherein the fourth distance range includes distances from the vehicle that are farther than distances included in a third distance range, andThe first distance range includes distances farther from the vehicle than those included in the fourth distance range. 19. A method comprising: receiving, by a computing device, one or more first images captured by at least one camera of a first camera type, wherein, each camera of the first camera type is attached to a vehicle and has a first field of view; receiving, by the computing device, one or more second images captured by a plurality of cameras of a second camera type, wherein each camera of the second camera type is attached to the vehicle and has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle; and receiving, by the computing device, one or more third images captured by a plurality of cameras of a third camera type, wherein each camera of the third camera type is attached to the vehicle and has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle, wherein the first field of view spans fewer angles in yaw relative to the vehicle than the third field of view, and wherein the third field of view spans fewer angles in yaw relative to the vehicle than the second field of view; The computing device identifies objects located within a first distance range from the vehicle based on one or more first images; the computing device identifies objects located within a second distance range from the vehicle based on one or more second images; the computing device identifies objects located within a third distance range from the vehicle based on one or more third images, wherein the third distance range includes distances farther from the vehicle than distances included in the second distance range, and wherein the first distance range includes distances farther from the vehicle than distances included in the third distance range; and the computing device determines driving decisions for the vehicle based on the objects located within the first distance range, the objects located within the second distance range, or the objects located within the third distance range. 20. A non-transitory computer-readable medium storing program instructions thereon, the program instructions, when executed by a processor, causing the processor to perform operations, the operations comprising: receiving one or more first images captured by at least one camera of a first camera type, wherein each camera of the first camera type is attached to a vehicle and has a first field of view; receiving one or more second images captured by a plurality of cameras of a second camera type, wherein each camera of the second camera type is attached to the vehicle and has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle; and receiving one or more third images captured by a plurality of cameras of a third camera type, wherein the third camera type...Each camera is attached to the vehicle and has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle, wherein the first field of view spans fewer angles in yaw relative to the vehicle than the third field of view, and wherein the third field of view spans fewer angles in yaw relative to the vehicle than the second field of view; Based on the one or more first images, objects located within a first distance range from the vehicle are identified; Based on the one or more second images, objects located within a second distance range from the vehicle are identified; Based on the one or more third images, objects located within a third distance range from the vehicle are identified, wherein the third distance range includes distances farther from the vehicle than distances included in the second distance range, and wherein the first distance range includes distances farther from the vehicle than distances included in the third distance range; and Based on the objects located within the first distance range from the vehicle, the objects located within the second distance range from the vehicle, or the objects located within the third distance range from the vehicle, driving decisions for the vehicle are determined. Claims 5 / 5 Page 6 CN 122253769 A Method and System for Camera Arrangement for Vehicle Object Detection and Avoidance Technical Field

[0001] This disclosure relates to methods and systems for camera arrangement for vehicle object detection and avoidance. Background Art

[0002] Unless otherwise stated herein, the description in this section is not prior art to the claims of this application and is not considered prior art by virtue of its inclusion in this section.

[0003] Cameras and image sensors are devices that can be used to capture images of a scene. Some cameras (e.g., film cameras) capture images on film chemically. Other cameras (e.g., digital cameras) capture image data electrically (e.g., using charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensors). Images captured by cameras can be analyzed to determine their content. For example, a processor can execute machine learning algorithms to identify objects in a scene based on a previously classified library of objects, including the shape, color, size, etc. of the objects (e.g., such machine learning algorithms can be applied to computer vision in robotics or other applications).

[0004] Cameras can have various features that distinguish one camera from another. For example, a camera and / or the images captured by it can be identified by values ​​such as aperture size, f-number, exposure time, shutter speed, depth of field, focal length, International Organization for Standardization (ISO) sensitivity (or gain), pixel size, sensor resolution, exposure distance, etc. These features can be based on the lens, image sensor, and / or other aspects of the camera. Furthermore, these features can also be adjusted within a single camera (e.g., the aperture of the lens on the camera can be adjusted between photos). Summary of the Invention

[0005] This disclosure relates to techniques that can be used to detect and identify objects, such as objects around a vehicle. Such detection and identification of objects can utilize data, such as images, collected from different types of sensors and / or cameras. These sensors and / or cameras may be located on the vehicle. The choice of camera and / or sensor type and the position of the camera and / or sensor on the vehicle can affect the information captured by the camera and / or sensor.

[0006] In one aspect, a system is provided. The system includes a vehicle. The system also includes at least one camera of a first camera type attached to the vehicle. Each camera of the first camera type has a first field of view. Additionally, the system includes a plurality of cameras of a second camera type attached to the vehicle. Each camera of the second camera type has a second field of view, and the second field of view spans at least 170° in yaw relative to the vehicle. Furthermore, the system includes a plurality of cameras of a third camera type attached to the vehicle. Each camera of the third camera type has a third field of view. The combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle. The first field of view spans a smaller angle in yaw relative to the vehicle than the third field of view, and the third field of view spans a smaller angle in yaw relative to the vehicle than the second field of view. Furthermore, the system includes a computing device communicatively coupled to at least one camera of a first camera type, a plurality of cameras of a second camera type, and a plurality of cameras of a third camera type. The computing device is configured to identify objects located within a first distance range from the vehicle based on one or more first images captured by the at least one camera of the first camera type. The computing device is also configured to identify objects located within a second distance range from the vehicle based on one or more second images captured by the plurality of cameras of the second camera type. Furthermore, the computing device is configured to identify objects located within a third distance range from the vehicle based on one or more third images captured by the plurality of cameras of the third camera type. The third distance range includes a distance from the vehicle that is farther than the distance included in the second distance range, and the first distance range includes a distance from the vehicle that is farther than the distance included in the third distance range.

[0007] In another aspect, a method is provided. The method includes receiving one or more first images captured by at least one camera of a first camera type by a computing device. Each camera of the first camera type is attached to the vehicle and has a first field of view. Furthermore, the method includes receiving one or more second images captured by a plurality of cameras of the second camera type via a computing device. Each camera of the second camera type is attached to the vehicle and has a second field of view. The second field of view spans at least 170° in yaw relative to the vehicle. Furthermore, the method includes receiving one or more second images captured by a plurality of cameras of the third camera type via a computing device.One or more third images captured by a camera. Each camera of the third camera type is attached to the vehicle and has a third field of view, and the combined field of view of the multiple cameras of the third camera type spans 360° relative to the vehicle in terms of yaw. The first field of view spans less angle relative to the vehicle in terms of yaw than the third field of view, and the third field of view spans less angle relative to the vehicle in terms of yaw than the second field of view. Furthermore, the method includes identifying objects located within a first distance range from the vehicle by a computing device based on one or more first images. The method also includes identifying objects located within a second distance range from the vehicle by a computing device based on one or more second images. Additionally, the method includes identifying objects located within a third distance range from the vehicle by a computing device based on one or more third images. The third distance range includes distances farther from the vehicle than those included in the second distance range, and the first distance range includes distances farther from the vehicle than those included in the third distance range. Furthermore, the method includes determining driving decisions for the vehicle by a computing device based on objects located within the first distance range, objects located within the second distance range, or objects located within the third distance range.

[0008] In another aspect, a non-transient computer-readable medium having instructions stored thereon is provided. The program instructions, when executed by a processor, cause the processor to perform operations. The operations include receiving one or more first images captured by at least one camera of a first camera type. Each camera of the first camera type is attached to a vehicle and has a first field of view. The operations also include receiving one or more second images captured by a plurality of cameras of a second camera type. Each camera of the second camera type is attached to the vehicle and has a second field of view, and the second field of view yaws at least 170° relative to the vehicle. Furthermore, the operations include receiving one or more third images captured by a plurality of cameras of a third camera type. Each camera of the third camera type is attached to the vehicle and has a third field of view, and the combined field of view of the plurality of cameras of the third camera type yaws 360° relative to the vehicle. The first field of view yaws less than the third field of view, and the third field of view yaws less than the second field of view. Furthermore, the operations include identifying objects located within a first distance range from the vehicle based on one or more first images. The operations also include identifying objects located within a second distance range from the vehicle based on one or more second images. Additionally, the operation includes identifying objects located within a third distance range from the vehicle based on one or more third images. The third distance range includes distances farther from the vehicle than those included in a second distance range, and the first distance range includes distances farther from the vehicle than those included in the third distance range. Furthermore, the operation includes identifying objects located within a first distance range from the vehicle, and objects located at a distance from the vehicle...Objects within a second distance range or objects within a third distance range from the vehicle are used to determine driving decisions for the vehicle.

[0009] These and other aspects, advantages, and alternatives will become apparent to those skilled in the art from the following detailed description and, where appropriate, from the accompanying drawings. Brief Description of the Drawings

[0010] FIG1 is a functional block diagram illustrating a vehicle according to an exemplary embodiment. Specification 2 / 34 pages 8 CN 122253769 A

[0011] FIG2A is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0012] FIG2B is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0013] FIG2C is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0014] FIG2D is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0015] FIG2E is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0016] FIG2F is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0017] FIG2G is an illustration of the physical configuration of a vehicle according to an exemplary embodiment.

[0018] FIG2H is a diagram of the physical configuration of a vehicle according to an example embodiment.

[0019] FIG2I is a diagram of the physical configuration of a vehicle according to an example embodiment.

[0020] FIG2J is a diagram of the field of view of various sensors according to an example embodiment.

[0021] FIG2K is a diagram of beam steering for sensors according to an example embodiment.

[0022] FIG3 is a conceptual diagram of wireless communication between various computing systems associated with an autonomous or semi-autonomous vehicle according to an example embodiment.

[0023] FIG4A is a block diagram of a system including a lidar device according to an example embodiment.

[0024] FIG4B is a block diagram of a lidar device according to an example embodiment.

[0025] FIG5A is a diagram of a camera of a first camera type and an associated first field of view according to an example embodiment.

[0026] FIG5B is a diagram of a camera of a second camera type and an associated second field of view according to an example embodiment.

[0027] FIG5C is a diagram of a camera of a third camera type and an associated third field of view according to an example embodiment.

[0028] FIG5D is an illustration of a fourth camera type and an associated fourth field of view according to an example embodiment.

[0029] FIG6A is an illustration of a camera system according to an example embodiment.

[0030] FIG6B is a simplified illustration of the camera system of FIG6A according to an example embodiment, showing a camera of a first camera type.

[0031] FIG6C is a simplified illustration of the camera system of FIG6A according to an example embodiment, showing a camera of a second camera type.

[0032] FIG6D is a simplified illustration of the camera system of FIG6A according to an example embodiment, showing a camera of a third camera type.

[0033] Figure 7A is an illustration of a camera system according to an example embodiment.

[0034] Figure 7B is a simplified illustration of the camera system of Figure 7A according to an example embodiment, showing a camera of a first camera type.

[0035] Figure 7C is a simplified illustration of the camera system of Figure 7A according to an example embodiment, showing a camera of a second camera type.

[0036] Figure 7D is a simplified illustration of the camera system of Figure 7A according to an example embodiment, showing a camera of a third camera type.

[0037] Figure 8A is an illustration of a camera system according to an example embodiment.

[0038] Figure 8B is a simplified illustration of the camera system of Figure 8A according to an example embodiment, showing a camera of a first camera type.

[0039] Figure 8C is a simplified illustration of the camera system of Figure 8A according to an example embodiment, showing a camera of a second camera type.

[0040] Figure 8D is a simplified illustration of the camera system of Figure 8A according to an example embodiment, showing a camera of a third camera type.

[0041] Figure 9A is an illustration of a camera system according to an example embodiment.

[0042] FIG9B is a simplified illustration of the camera system of FIG9A according to an exemplary embodiment, showing a camera of a first camera type of the camera system.

[0043] FIG9C is a simplified illustration of the camera system of FIG9A according to an exemplary embodiment, showing a camera of a second camera type of the camera system.

[0044] FIG9D is a simplified illustration of the camera system of FIG9A according to an exemplary embodiment, showing a camera of a third camera type of the camera system.

[0045] FIG9E is a simplified illustration of the camera system of FIG9A according to an exemplary embodiment, showing a camera of a fourth camera type of the camera system.

[0046] FIG10A is an illustration of a camera system according to an exemplary embodiment.

[0047] FIG10B is a simplified illustration of the camera system of FIG10A according to an exemplary embodiment, showing a camera of a first camera type of the camera system.

[0048] FIG10C is a simplified illustration of the camera system of FIG10A according to an exemplary embodiment, showing a camera of a second camera type of the camera system.

[0049] FIG10D is a simplified illustration of the camera system of FIG10A according to an exemplary embodiment, showing a camera of a third camera type of the camera system.

[0050] FIG10E is a simplified illustration of the camera system of FIG10A according to an exemplary embodiment, showing a camera of a fourth camera type in the camera system.

[0051] FIG11A is an illustration of a camera system according to an exemplary embodiment.

[0052] FIG11B is a simplified illustration of the camera system of FIG11A according to an exemplary embodiment, showing a camera of a first camera type in the camera system.

[0053] Figure 11C is a simplified illustration of the camera system of Figure 11A according to an exemplary embodiment, showing a camera of a second camera type in the camera system.

[0054] Figure 11D is a simplified illustration of the camera system of Figure 11A according to an exemplary embodiment, showing a camera of a third camera type in the camera system.

[0055] Figure 11E is a simplified illustration of the camera system of Figure 11A according to an exemplary embodiment, showing a camera of a fourth camera type in the camera system.

[0056] Figure 12 is an illustration of an object recognition technology according to an exemplary embodiment.

[0057] Figure 13 is an illustration of a third camera type camera according to a first modification of an exemplary embodiment.

[0058] Figure 14 is a flowchart illustration of a method according to an exemplary embodiment. Detailed Description

[0059] Exemplary methods and systems are considered herein. Any exemplary embodiments or features described herein are not necessarily to be construed as preferred or advantageous over other embodiments or features. Furthermore, the exemplary embodiments described herein are not intended to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a wide variety of different configurations, all of which are considered herein. Additionally, the specific arrangements shown in the figures should not be considered limiting. It should be understood that other embodiments may include more or fewer of each element shown in the given figures. Additionally, some of the shown elements may be combined or omitted. Furthermore, exemplary embodiments may include elements not shown in the figures. Specification 4 / 34 pages 10 CN 122253769 A

[0060] The lidar device described herein may include one or more light emitters and one or more detectors for detecting light emitted by the one or more light emitters and reflected by one or more objects in the environment surrounding the lidar device. As an example, the surrounding environment may include an internal or external environment, such as the interior or exterior of a building. Additionally or alternatively, the surrounding environment may include the interior of a vehicle. Further, the surrounding environment may include the area around a road and / or the vicinity of a road. Examples of objects in the surrounding environment include, but are not limited to, other vehicles, traffic signs, pedestrians, cyclists, road surfaces, buildings, and terrain. Furthermore, one or more light emitters may emit light into the local environment of the lidar itself. For example, light emitted from one or more light emitters may interact with the lidar housing and / or surfaces or structures coupled to the lidar. In some cases, lidar can be mounted on a vehicle, in which case one or more light emitters can be configured to emit light that interacts with objects near the vehicle. Furthermore, the light emitters may include fiber optic amplifiers, laser diodes, light-emitting diodes (LEDs), etc.

[0061] Information (e.g., images) can be collected from multiple cameras and / or sensors. This information can be combined and used...Machine vision on a vehicle (e.g., a vehicle operating in an autonomous or semi-autonomous mode). For example, a camera system can use information collected from these cameras and other sensors to provide alerts to a user or make other decisions about the vehicle.

[0062] The arrangement of cameras within a camera system for machine vision on a vehicle (e.g., a vehicle operating in an autonomous or semi-autonomous mode) can be complex. Such a system may require the integration of multiple cameras, sensors, and processors to provide real-time information about the environment surrounding the vehicle. Cameras and other sensors can be of different types, including: cameras and other sensors that capture images at high resolution, cameras and other sensors configured to detect objects at short distances from the vehicle, and cameras and other sensors configured to detect objects at long distances from the vehicle.

[0063] Thus, when designing a camera system, multiple design parameters may exist that are considered. One set of such design parameters is the type of cameras and / or sensors used in the camera system and the location of these cameras and / or sensors. When determining the type of cameras and / or sensors to be used, and when determining the location of those cameras and / or sensors on the vehicle, multiple potentially competing objects may exist. One such object could be improved safety, for example, the ability of a vehicle to avoid or warn passengers of potential hazards. Another object could be improved efficiency, for example, the ability to make more complex decisions based on the same amount of information. This could include decisions that take into account conditions that may lead to a deterioration in the quality of sensor data (e.g., different weather conditions). Another object could be reduced overall costs of producing, training, using, and / or maintaining a camera system. In some embodiments described herein, the selection of the type of camera and / or sensor and their location on the vehicle can simultaneously advance these objects. In some embodiments described herein, the selection of the type of camera and / or sensor and their location on the vehicle can involve prioritizing one object over one or more of other objects.

[0064] Multiple different types of cameras can be attached to the vehicle and used for computer vision (e.g., to capture images for object detection and / or classification and avoidance). For example, three remote cameras can be attached to the roof of the vehicle and oriented in the forward direction (e.g., the main direction of travel of the vehicle). Additionally, such a system can include four cameras configured to detect objects located near the vehicle. Each of these cameras can have a field of view of at least 170° (e.g., a fisheye camera). Furthermore, such cameras can be attached to the front bumper or side fender bumper of the vehicle (e.g., above the front or rear wheel wells of the vehicle) and / or can be oriented in a forward, lateral, or rearward direction relative to the vehicle's direction of travel.

[0065] Some embodiments may also include five cameras, which, when combined, are capable of detecting objects located at a distance from the center of the vehicle.Objects at a distance and at any yaw angle relative to the vehicle (e.g., any angle between 0° and 360°). The five cameras may be attached to the vehicle and / or to the roof of the vehicle above one or more front wheels (e.g., one of the adjacent 170° field-of-view cameras). In addition, the five cameras capable of observing the vehicle in 360° can each be oriented in one or more lateral, forward, and / or rearward directions relative to the vehicle's direction of travel. Given this camera arrangement, the system can simultaneously capture images that can be used to identify objects that are: (1) at close range relative to the vehicle; (2) at medium range and within a full 360° yaw relative to the vehicle; and (3) at far range relative to the vehicle and in a forward direction relative to the vehicle's direction of travel.

[0066] In some embodiments described herein, images captured from one or more cameras may be combined with data collected by other sensors (e.g., light detection and ranging (LiDAR) devices). The combination of information from multiple sources can enable improved detection of specific types of objects, such as traffic lights and headlights. For example, a lidar device can be attached to a vehicle and configured to generate a point cloud indicating the distance to objects in the environment surrounding the vehicle.

[0067] In some embodiments, one of the cameras configured to detect objects located close to the vehicle may include a neutral density filter through which the image is captured. Such a camera may be oriented relative to the vehicle to capture images from a pitch angle greater than 20° relative to the vehicle. Furthermore, the camera may be configured to capture a pair of images. The first of the two images may be captured with an exposure time set by the camera's automatic exposure settings. The second of the two images may be captured with an exposure time longer than a predetermined exposure time, which is long enough to ensure the capture of flickering light sources (e.g., a predetermined exposure time of 1 / 60 or 1 / 50 of a second) or an exposure time longer than the camera's automatic exposure settings. The two images may be aligned with and overlaid with the point cloud generated by the lidar device to produce an overlapping image. Then, detection of objects near the car can be performed based on the overlapping images.

[0068] The following description and figures will illustrate the features of various exemplary embodiments. The embodiments provided are by way of example and not limitation. Therefore, the figures are not necessarily drawn to scale.

[0069] Exemplary systems within the scope of this disclosure will now be described in more detail. The exemplary systems can be implemented in or take the form of a car. In addition, exemplary systems can also be implemented in or take the form of various vehicles, such as cars, trucks (e.g., pickup trucks, vans, tractors and tractor trailers), motorcycles, buses, airplanes, helicoptersMachines, drones, lawnmowers, bulldozers, boats, submarines, all-terrain vehicles, snowmobiles, airplanes, recreational vehicles, amusement park vehicles, farm equipment or vehicles, construction equipment or vehicles, warehouse equipment or vehicles, factory equipment or vehicles, trams, golf carts, trains, handcarts, sidewalk transport vehicles, and robotic equipment. Other vehicles are also possible. Furthermore, in some embodiments, the example system may not include a vehicle.

[0070] Referring now to the accompanying drawings, FIG1 is a functional block diagram illustrating an example vehicle 100, which can be configured to operate fully or partially in an autonomous mode. More specifically, vehicle 100 can operate in an autonomous mode without human interaction by receiving control commands from a computing system. As part of operating in autonomous mode, vehicle 100 can use sensors to detect and possibly identify objects in the surrounding environment for safe navigation. Furthermore, the example vehicle 100 can operate in a partially autonomous (i.e., semi-autonomous) mode, wherein some functions of the vehicle 100 are controlled by a human driver of the vehicle 100, while some functions of the vehicle 100 are controlled by a computing system. For example, the vehicle 100 may also include a subsystem that enables the driver to control the operation of the vehicle 100 (such as steering, acceleration, and braking), while the computing system performs assistance functions, such as lane departure warning / lane keeping assist or adaptive cruise control, based on other objects in the surrounding environment (e.g., the vehicle).

[0071] As described herein, in partially autonomous driving mode, even if the vehicle assists one or more driving operations (e.g., steering, braking, and / or acceleration to perform lane centering, adaptive cruise control, advanced driver assistance systems (ADAS), and emergency braking), the human driver should be aware of the vehicle's surroundings and supervise the assisted driving operations. Here, even if the vehicle may perform all driving tasks in certain situations, the human driver is expected to be responsible for control as needed.

[0072] Although various systems and methods are described below in conjunction with autonomous vehicles for the sake of brevity, these or similar systems and methods can be used in various driver assistance systems that do not rise to the level of fully autonomous driving systems (i.e., partially autonomous driving systems). In the United States, the Society of Automotive Engineers (SAE) has defined different levels of automated driving operation, [Page 6 / 34, CN 122253769 A], to indicate the degree or extent of vehicle control in driving, although different organizations in the United States or other countries may classify levels differently. More specifically, the disclosed systems and methods can be used in SAE Level 2 driver assistance systems that implement steering, braking, acceleration, lane centering, adaptive cruise control, and other driver support. The disclosed systems and methods can be used in SAE Level 3 driver assistance systems capable of autonomous driving under limited (e.g., highway) conditions. Similarly, the disclosed systems and methods can be used with SAE...In vehicles with Level 4 autonomous driving systems, SAE Level 4 autonomous driving systems operate autonomously in most normal driving situations and require only occasional attention from a human operator. In all such systems, accurate lane estimation can be performed automatically without driver input or control (e.g., when the vehicle is in motion), resulting in improved reliability of vehicle positioning and navigation, as well as overall safety improvements for autonomous, semi-autonomous, and other driver assistance systems. As previously mentioned, other organizations in the United States or other countries may classify levels of autonomous driving operations differently than SAE does. Without limitation, the systems and methods disclosed herein can be used for driver assistance systems defined by the autonomous driving operation levels of these other organizations.

[0073] As shown in FIG1, vehicle 100 may include various subsystems such as propulsion system 102, sensor system 104, control system 106, one or more peripheral devices 108, power supply 110, computer system 112 (which may also be referred to as computing system) with data storage device 114, and user interface 116. In other examples, vehicle 100 may include more or fewer subsystems, each subsystem may include multiple elements. The subsystems and components of vehicle 100 can be interconnected in various ways. Additionally, in embodiments, the functionality of vehicle 100 described herein can be divided into additional functional or physical components, or combined into fewer functional or physical components. For example, control system 106 and computer system 112 can be combined into a single system that operates vehicle 100 according to various operations.

[0074] Propulsion system 102 may include one or more components operable to provide powered motion to vehicle 100, and may include engine / motor 118, energy source 119, transmission 120, and wheels / tires 121, as well as other possible components. For example, engine / motor 118 may be configured to convert energy source 119 into mechanical energy, and may correspond to one or a combination of internal combustion engines, electric motors, steam engines, or Stirling engines, as well as other possible options. For example, in some embodiments, propulsion system 102 may include multiple types of engines and / or motors, such as gasoline engines and electric motors.

[0075] Energy source 119 refers to an energy source that can provide power, in whole or in part, to one or more systems of vehicle 100 (e.g., engine / motor 118). For example, energy source 119 may correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other electrical sources. In some embodiments, energy source 119 may include a combination of a fuel tank, battery, capacitor, and / or flywheel.

[0076] Transmission 120 can transmit mechanical power from engine / motor 118 to wheels / tires 121 and / or vehicle 100.Other possible systems. Thus, the transmission 120 may include a gearbox, clutch, differential, and drive shaft, as well as other possible components. The drive shaft may include a wheel axle connected to one or more wheels / tires 121.

[0077] In the example embodiment, the wheels / tires 121 of the vehicle 100 may have various configurations. For example, the vehicle 100 may exist as a unicycle, bicycle / motorcycle, tricycle, or four-wheeled car / truck, as well as other possible configurations. Thus, the wheels / tires 121 may be connected to the vehicle 100 in various ways and may exist in different materials, such as metal and rubber.

[0078] The sensor system 104 may include various types of sensors, such as a Global Positioning System (GPS) 122, an Inertial Measurement Unit (IMU) 124, radar 126, lidar 128, camera 130, steering sensor 123, and throttle / brake sensor 125, as well as other possible sensors. In some embodiments, sensor system 104 may further include sensors configured to monitor internal systems of vehicle 100 (e.g., O2 monitor, fuel gauge, engine oil temperature, and brake wear).

[0079] GPS 122 may include a transceiver operable to provide information about the position of vehicle 100 relative to the Earth. IMU 124 may be configured to use one or more accelerometers and / or gyroscopes and may sense changes in the position and orientation of vehicle 100 based on inertial acceleration. For example, IMU 124 may detect pitch and yaw of vehicle 100 when vehicle 100 is stationary or in motion.

[0080] Radar 126 may represent one or more systems configured to use radio signals to sense objects (including the velocity and direction of travel) in the surrounding environment of vehicle 100. Thus, radar 126 may include an antenna configured to transmit and receive radio signals. In some embodiments, radar 126 may correspond to an installable radar configured to obtain measurements of the surrounding environment of vehicle 100.

[0081] LiDAR 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components, and may operate in a coherent mode (e.g., using heterodyne detection) or an incoherent detection mode (i.e., time-of-flight mode). In some embodiments, one or more detectors of lidar 128 may include one or more photodetectors, which may be particularly sensitive detectors (e.g., avalanche photodiodes). In some examples, such photodetectors may be able to detect single photons (e.g., single-photon avalanche diodes (SPADs)). Furthermore, such photodetectors may be arranged (e.g., via series electrical connections) in an array (e.g., as in a silicon photomultiplier tube (SiPM)). In some examplesFor example, one or more photodetectors are Geiger-mode operating devices, and the lidar includes sub-components designed for such Geiger-mode operation.

[0082] Camera 130 may include one or more devices (e.g., still cameras, video cameras, thermal imaging cameras, stereo cameras, and night vision cameras) configured to capture images of the surrounding environment of vehicle 100.

[0083] Steering sensor 123 may sense the steering angle of vehicle 100, which may involve measuring the angle of the steering wheel or measuring an electrical signal representing the angle of the steering wheel. In some embodiments, steering sensor 123 may measure the angle of the wheels of vehicle 100, for example, detecting the angle of the wheels relative to the forward axle of vehicle 100. Steering sensor 123 may also be configured to measure a combination (or subset) of the steering wheel angle, an electrical signal representing the steering wheel angle, and the angles of the wheels of vehicle 100.

[0084] Throttle / brake sensor 125 may detect the position of the throttle or brake of vehicle 100. For example, the throttle / brake sensor 125 can measure the angles of both the accelerator pedal (throttle) and the brake pedal, or it can measure electrical signals that can represent, for example, the angles of the accelerator pedal (throttle) and / or the brake pedal. The throttle / brake sensor 125 can also measure the angle of the throttle body of the vehicle 100, which may include part of a modulated physical mechanism (e.g., a butterfly valve and carburetor) that provides energy to the engine / motor 118 via the power source 119. Additionally, the throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of the vehicle 100, or a combination (or subset) of the angles of the accelerator pedal (throttle) and the brake pedal, electrical signals representing the angles of the accelerator pedal (throttle) and the brake pedal, the angle of the throttle body, and the pressure exerted by at least one brake pad on the rotor of the vehicle 100. In other embodiments, the throttle / brake sensor 125 may be configured to measure the pressure applied to a pedal (e.g., the throttle or brake pedal) of the vehicle.

[0085] The control system 106 may include components configured to assist navigation of the vehicle 100, such as a steering unit 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / path system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 may be operable to adjust the heading of the vehicle 100, and the throttle 134 may control the operating speed of the engine / motor 118 to control the acceleration of the vehicle 100. The braking unit 136 may decelerate the vehicle 100, which may involve using friction to decelerate the wheels / tires 121. In some embodiments, the braking unit 136 may convert the kinetic energy of the wheels / tires 121 into electrical current for subsequent use by one or more systems of the vehicle 100.

[0086] The sensor fusion algorithm 138 may include a Kalman filter, a Bayesian network, or other algorithms that can process data from sensors.Other algorithms for the data of system 104. In some embodiments, sensor fusion algorithm 138 may provide evaluations based on incoming sensor data, such as evaluations of individual objects and / or features, evaluations of specific situations, and / or evaluations of the potential impact of a given situation on the specification page 8 / 34 14 CN 122253769 A.

[0087] Computer vision system 140 may include hardware and software (e.g., a general-purpose processor such as a central processing unit (CPU), a dedicated processor such as a graphics processing unit (GPU) or tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), volatile memory, non-volatile memory, or one or more machine learning models) operable to process and analyze images to attempt to determine moving objects (e.g., other vehicles, pedestrians, cyclists, or animals) and non-moving objects (e.g., traffic lights, road boundaries, speed bumps, or potholes). Thus, computer vision system 140 may use object recognition, structure of motion (SFM), video tracking, and other algorithms used in computer vision, such as object recognition, mapping the environment, tracking objects, estimating the velocity of objects, etc.

[0088] The navigation / path system 142 can determine the driving path of the vehicle 100, which may involve dynamically adjusting navigation during operation. Thus, the navigation / path system 142 can navigate the vehicle 100 using data from sensor fusion algorithm 138, GPS 122, maps, and other sources. The obstacle avoidance system 144 can assess potential obstacles based on sensor data and enable the vehicle 100's system to avoid or otherwise traverse potential obstacles.

[0089] As shown in FIG1, the vehicle 100 may also include peripheral devices 108, such as a wireless communication system 146, a touchscreen 148, an internal microphone 150, and / or a speaker 152. The peripheral devices 108 can provide controls or other elements for a user to interact with the user interface 116. For example, the touchscreen 148 can provide information to the user of the vehicle 100. The user interface 116 can also accept input from the user via the touchscreen 148. The peripheral devices 108 can also enable the vehicle 100 to communicate with devices such as other vehicle equipment.

[0090] The wireless communication system 146 can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication (such as Code Division Multiple Access (CDMA), Evolved Data Optimized (EVDO), Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS)) or cellular communication (such as 4G Global Microwave Access Interoperability (WiMAX) or Long Term Evolution (LTE) or 5G). Alternatively, the wireless communication system 146 can use WIFI® or other possible...The connection communicates with a wireless local area network (WLAN). The wireless communication system 146 can also communicate directly with the device using, for example, an infrared link, Bluetooth, or ZigBee. In the context of this disclosure, other wireless protocols, such as various vehicle communication systems, are possible. For example, the wireless communication system 146 may include one or more dedicated short-range communication (DSRC) devices, which may include public and / or private data communication between the vehicle and / or roadside stations.

[0091] The vehicle 100 may include a power supply 110 for powering components. In some embodiments, the power supply 110 may include a rechargeable lithium-ion or lead-acid battery. For example, the power supply 110 may include one or more batteries configured to provide power. The vehicle 100 may also use other types of power supplies. In an example embodiment, the power supply 110 and the energy source 119 may be integrated into a single energy source.

[0092] The vehicle 100 may also include a computer system 112 to perform operations, such as those described herein. Thus, computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored in a non-transitory computer-readable medium (e.g., data storage device 114). In some embodiments, computer system 112 may represent multiple computing devices that can be used to control various components or subsystems of vehicle 100 in a distributed manner.

[0093] In some embodiments, data storage device 114 may include instructions 115 (e.g., program logic) executable by processor 113 to perform various functions of vehicle 100, including those functions described above in conjunction with FIG. 1. Data storage device 114 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of propulsion system 102, sensor system 104, control system 106, and peripheral device 108.

[0094] In addition to instructions 115, data storage device 114 may store data such as road maps, route information, etc. Information such as this specification, page 9 / 34, CN 122253769 A, can be used by the vehicle 100 and computer system 112 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0095] The vehicle 100 may include a user interface 116 for providing information to or receiving input from the user of the vehicle 100. The user interface 116 may control or implement control over the content and / or layout of interactive images that may be displayed on the touchscreen 148. Furthermore, the user interface 116 may include one or more input / output devices within the group of peripheral devices 108, such as a wireless communication system 146, a touchscreen 148, a microphone 150, and a speaker 152.

[0096] Computer system 112 can control the functions of vehicle 100 based on inputs received from various subsystems (e.g., propulsion system 102, sensor system 104, or control system 106) and from user interface 116. For example, computer system 112 can utilize inputs from sensor system 104 to estimate outputs generated by propulsion system 102 and control system 106. According to embodiments, computer system 112 can operate to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 can disable some or all functions of vehicle 100 based on signals received from sensor system 104.

[0097] Components of vehicle 100 can be configured to operate in a manner interconnected with other components, either internally or externally to their respective systems. For example, in an example embodiment, camera 130 can capture multiple images that may represent information about the state of the surrounding environment of vehicle 100 operating in autonomous or semi-autonomous mode. The state of the surrounding environment may include parameters of the road on which the vehicle is operating. For example, computer vision system 140 may be able to identify slope (gradient) or other features based on multiple images of the road. Furthermore, a combination of GPS 122 and features identified by computer vision system 140 may be used together with map data stored in data storage device 114 to determine specific road parameters. Additionally, radar 126 and / or lidar 128 and / or some other environmental mapping, ranging, and / or positioning sensor systems may also provide information about the vehicle's surroundings.

[0098] In other words, a combination of various sensors (which may be referred to as input indication and output indication sensors) and computer system 112 may interact to provide indications of inputs for controlling the vehicle or indications of the vehicle's surroundings.

[0099] In some embodiments, computer system 112 may make determinations about various objects based on data provided by systems other than radio systems. For example, vehicle 100 may have lasers or other optical sensors configured to sense objects in the vehicle's field of view. Computer system 112 may use the outputs from various sensors to determine information about objects in the vehicle's field of view and may determine distance and orientation information to various objects. Computer system 112 can also determine whether an object is desired or undesirable based on outputs from various sensors.

[0100] Although FIG. 1 shows various components of vehicle 100 (i.e., wireless communication system 146, computer system 112, data storage device 114, and user interface 116) as being integrated into vehicle 100, one or more of these components may be installed or associated separately from vehicle 100. For example, data storage device 114 may exist partially or entirely separate from vehicle 100. Therefore, vehicle 100 may be provided in the form of device elements that can be positioned separately or together. Devices constituting vehicle 100Components may be communicatively coupled together in a wired and / or wireless manner.

[0101] Figures 2A-2E illustrate an example vehicle 200 (e.g., a fully autonomous or semi-autonomous vehicle) that may include some or all of the functions described in conjunction with vehicle 100 of Figure 1. Although vehicle 200 is shown in Figures 2A-2E as a van with side mirrors for illustrative purposes, this disclosure is not limited thereto. For example, vehicle 200 may represent a truck, car, semi-trailer truck, motorcycle, golf cart, off-road vehicle, agricultural vehicle, or any other vehicle described elsewhere herein (e.g., bus, boat, aircraft, helicopter, drone, lawnmower, bulldozer, submarine, all-terrain vehicle, snowmobile, aircraft, recreational vehicle, amusement park vehicle, farm equipment, construction equipment or vehicle, warehouse equipment or vehicle, factory equipment or vehicle, tram, train, handcart, sidewalk transport vehicle, and robotic equipment). Specification 10 / 34 pages 16 CN 122253769 A

[0102] Example vehicle 200 may include one or more sensor systems 202, 204, 206, 208, 210, 212, 214 and 218. In some embodiments, sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may represent one or more optical systems (e.g., cameras), one or more lidar, one or more radars, one or more inertial sensors, one or more humidity sensors, one or more acoustic sensors (e.g., microphones and sonar devices), or one or more other sensors configured to sense information about the environment around vehicle 200. In other words, any sensor system now known or created hereafter may be coupled to vehicle 200 and / or may be used in conjunction with various operations of vehicle 200. As an example, lidar may be used for autonomous driving or other types of navigation, planning, perception and / or mapping operations of vehicle 200. Additionally, sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may represent combinations of sensors described herein (e.g., one or more lidar and radar; one or more lidar and camera; one or more camera and radar; or one or more lidar, camera, and radar).

[0103] Note that the number, location, and type of sensor systems (e.g., 202 and 204) depicted in Figures 2A-E are intended as non-limiting examples of the location, number, and type of such sensor systems for autonomous or semi-autonomous vehicles. Alternative numbers, locations, types, and configurations of such sensors are possible (e.g., consistent with vehicle size, shape, aerodynamics, fuel economy, aesthetics, or other conditions to reduce costs or adapt to specific environments or application environments). For example, sensor systems(e.g., 202 and 204) can be positioned at various other locations on the vehicle (e.g., at location 216) and can have a field of view corresponding to the interior and / or surrounding environment of the vehicle 200.

[0104] The sensor system 202 can be mounted on top of the vehicle 200 and can include one or more sensors configured to detect information about the environment around the vehicle 200 and output indications of the information. For example, the sensor system 202 can include any combination of cameras, radar, lidar, inertial sensors, humidity sensors, and acoustic sensors (e.g., microphones and sonar devices). The sensor system 202 can include one or more movable mounts operable to adjust the orientation of one or more sensors in the sensor system 202. In one embodiment, the movable mount can include a rotating platform that can scan the sensors to obtain information from every direction around the vehicle 200. In another embodiment, the movable mount of the sensor system 202 can be moved in a scanning manner within a specific angular and / or azimuth and / or elevation range. Sensor system 202 can be mounted on the roof of a vehicle, but other mounting locations are also possible.

[0105] Additionally, the sensors of sensor system 202 can be distributed in different locations and do not need to be juxtaposed in a single location. Furthermore, each sensor of sensor system 202 can be configured to move or scan independently of other sensors of sensor system 202. Additionally or alternatively, multiple sensors can be mounted at one or more of sensor locations 202, 204, 206, 208, 210, 212, 214 and / or 218. For example, there may be two lidar devices mounted at sensor locations and / or there may be one lidar device and one radar device mounted at sensor locations.

[0106] One or more sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may include one or more lidar devices. For example, a lidar device may include multiple light emitter devices arranged within an angular range relative to a given plane (e.g., the x-y plane). For example, one or more of sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may be configured to rotate or pivot about an axis perpendicular to a given plane (e.g., the z-axis) to illuminate the environment around vehicle 200 with light pulses. Information about the surrounding environment can be determined based on various aspects of the detected reflected light pulses (e.g., elapsed time of flight, polarization, and intensity).

[0107] In an example embodiment, sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may be configured to...The vehicle 200 is configured to provide corresponding point cloud information that can be associated with physical objects in the surrounding environment of the vehicle 200. While the vehicle 200 and sensor systems 202, 204, 206, 208, 210, 212, 214, and 218 are shown to include certain features, it should be understood that other types of sensor systems are contemplated within the scope of this disclosure, as described on pages 11 / 34 of this specification (CN 122253769 A). Furthermore, the example vehicle 200 may include any of the components described in conjunction with the vehicle 100 of FIG1. ​​

[0108] In the example configuration, one or more radars may be located on the vehicle 200. Similar to radar 126 described above, one or more radars may include antennas configured to transmit and receive radio waves (e.g., electromagnetic waves with frequencies between 30 Hz and 300 GHz). Such radio waves can be used to determine the distance and / or velocity to one or more objects in the surrounding environment of the vehicle 200. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may include one or more radars. In some examples, one or more radars may be located near the rear of vehicle 200 (e.g., sensor systems 208 and 210) to actively scan the environment near the rear of vehicle 200 for radio-reflective objects. Similarly, one or more radars may be located near the front of vehicle 200 (e.g., sensor systems 212 or 214) to actively scan the environment near the front of vehicle 200. For example, the radars may be located in a position suitable for illuminating an area including the path of travel of vehicle 200 without being obstructed by other features of vehicle 200. For example, the radars may be embedded and / or mounted in or near the front bumper, headlights, hood, and / or engine cover, etc. Furthermore, one or more additional radars may be positioned to actively scan the sides and / or rear of vehicle 200 for radio-reflective objects, for example, by including such devices in or near the rear bumper, side panels, rocker arm panels, and / or chassis, etc.

[0109] The vehicle 200 may include one or more cameras. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may include one or more cameras. The camera may be a photosensitizing instrument, such as a still camera, video camera, thermal imaging camera, stereo camera, night vision camera, etc., configured to capture multiple images of the surrounding environment of the vehicle 200. For this purpose, the camera may be configured to detect visible light and may additionally or alternatively be configured to detect light from other parts of the spectrum, such as infrared or ultraviolet light. The camera may be a two-dimensional detector and may optionally have a three-dimensional spatial sensitivity range. In some embodiments, the camera may include, for example, a range detector, which is...The camera is configured to generate a two-dimensional image indicating the distances from the camera to multiple points in the surrounding environment. For this purpose, the camera can use one or more range detection techniques. For example, the camera can provide range information using structured light technology, in which the vehicle 200 illuminates objects in the surrounding environment with a predetermined light pattern (such as a grid or checkerboard pattern), and the camera is used to detect the reflection of the predetermined light pattern from the surrounding environment. Based on the distortion in the reflected light pattern, the vehicle 200 can determine the distance to points on the object. The predetermined light pattern can include infrared light or radiation of other suitable wavelengths for such measurements. In some examples, the camera can be mounted inside the windshield of the vehicle 200. Specifically, the camera can be positioned to capture images from a forward-facing view relative to the orientation of the vehicle 200. Other mounting positions and viewing angles of the camera can also be used, whether inside or outside the vehicle 200. Furthermore, the camera can have associated optics operable to provide an adjustable field of view. Further still, the camera can be mounted to the vehicle 200 with a movable mounting to change the camera's pointing angle, for example, via a translation / tilt mechanism.

[0110] The vehicle 200 may also include one or more acoustic sensors for sensing the surrounding environment of the vehicle 200 (e.g., one or more of sensor systems 202, 204, 206, 208, 210, 212, 214, 216, 218 may include one or more acoustic sensors). The acoustic sensors may include microphones (e.g., piezoelectric microphones, condenser microphones, ribbon microphones, or microelectromechanical systems (MEMS) microphones) for sensing sound waves (i.e., pressure differences) in a fluid (e.g., air) surrounding the vehicle 200. Such acoustic sensors can be used to identify sounds in the surrounding environment (e.g., sirens, human voices, animal sounds, or alarms), and the control strategy of the vehicle 200 may be based on these sounds. For example, if the acoustic sensors detect an sirens (e.g., a mobile siren or a fire truck siren), the vehicle 200 may decelerate and / or navigate to the edge of the road.

[0111] Although not shown in Figures 2A to 2E, vehicle 200 may include a wireless communication system (e.g., similar to wireless communication system 146 of Figure 1, page 12 / 34 of the specification, CN 122253769 A, and / or other than wireless communication system 146 of Figure 1). The wireless communication system may include a wireless transmitter and receiver, which may be configured to communicate with devices external to or internal to vehicle 200. Specifically, the wireless communication system may include transceivers configured to communicate with other vehicles and / or computing devices, for example, in a vehicle communication system or road station. Examples of such vehicle communication systems include DSRC, radio frequency identification (RFID), and other communication standards proposed for intelligent transportation systems.

[0112] In addition to or in lieu of those shown, vehicle 200 may include one or more other components. Additional components may include electrical or mechanical functions.

[0113] The control system of vehicle 200 may be configured to control vehicle 200 according to a control strategy among a plurality of possible control strategies. The control system may be configured to receive information from sensors coupled to vehicle 200 (on or outside vehicle 200), modify the control strategy (and associated driving behavior) based on that information, and control vehicle 200 according to the modified control strategy. The control system may also be configured to monitor information received from sensors and continuously evaluate driving conditions; and may also be configured to modify the control strategy and driving behavior based on changes in driving conditions. For example, the route taken by the vehicle from one destination to another may be modified based on driving conditions. Additionally or alternatively, speed, acceleration, turning angle, following distance (i.e., distance to the vehicle in front of the current vehicle), lane selection, etc., may be modified in response to changes in driving conditions.

[0114] As described above, in some embodiments, the vehicle 200 may be in the form of a van, but alternative forms are also possible and are considered herein. Thus, Figures 2F-2I show embodiments in which the vehicle 250 is in the form of a semi-truck. For example, Figure 2F shows a front view of the vehicle 250, and Figure 2G shows an isometric view of the vehicle 250. In embodiments where the vehicle 250 is a semi-truck, the vehicle 250 may include a tractor section 260 and a trailer section 270 (shown in Figure 2G). Figures 2H and 2I provide side and top views of the tractor section 260, respectively. Similar to the vehicle 200 illustrated above, the vehicle 250 illustrated in Figures 2F-2I may also include various sensor systems (e.g., similar to sensor systems 202, 206, 208, 210, 212, 214 shown and described with reference to Figures 2A-2E). In some embodiments, although the vehicle 200 of Figures 2A-2E may include only a single copy of some sensor systems (e.g., sensor system 204), the vehicle 250 shown in Figures 2F-2I may include multiple copies of that sensor system (e.g., sensor systems 204A and 204B, as shown).

[0115] While the figures and description throughout may refer to a vehicle of a given form (e.g., a semi-truck 250 or a van 200), it should be understood that the embodiments described herein can be equally applied to various vehicle environments (e.g., with modifications to take into account the shape factor of the vehicle). For example, sensors and / or other components described or shown as part of van 200 may also be used in semi-truck 250 (e.g., for navigation and / or obstacle detection and avoidance).

[0116] Figure 2J shows various sensor fields of view (e.g., associated with the vehicle 250 described above). As described above, vehicle 250 mayIt may contain multiple sensors / sensor units. For example, the positions of various sensors may correspond to the positions of the sensors disclosed in Figures 2F-2I. However, in some cases, the sensors may have other positions. For the sake of simplicity, the sensor position reference numerals are omitted from Figure 2J. For each sensor unit of vehicle 250, Figure 2J shows a representative field of view (e.g., fields of view labeled 252A, 252B, 252C, 252D, 254A, 254B, 256, 258A, 258B, and 258C). The field of view of a sensor may include an angular region on which the sensor can detect an object (e.g., an azimuth region and / or an elevation region).

[0117] Figure 2K shows the beam steering of a sensor for a vehicle (e.g., vehicle 250 shown and described with reference to Figures 2F-2J) according to an example embodiment. In various embodiments, the sensor unit of vehicle 250 may be radar, lidar, sonar, etc. Furthermore, in some embodiments, the sensor may be scanned within the field of view of the sensor during operation. Various scanning angles of the example sensor are shown as region 272, each indicating the angular region on which the sensor is operating. The sensor may periodically or iteratively change the region on which it is operating. In some embodiments, the vehicle specification page 13 / 34 19 CN 122253769 A 250 may use multiple sensors to measure region 272. Additionally, other regions may be included in other examples. For example, one or more sensors may measure aspects of the trailer 270 of the vehicle 250 and / or the area directly in front of the vehicle 250.

[0118] At some angles, the operating region 275 of the sensor may include the rear wheels 276A, 276B of the trailer 270. Thus, the sensor may measure the rear wheels 276A and / or rear wheels 276B during operation. For example, the rear wheels 276A, 276B may reflect lidar signals or radar signals transmitted by the sensor. The sensor may receive reflected signals from the rear wheels 276A, 276. Thus, the data collected by the sensor may include data reflected from the wheels.

[0119] In some cases, such as when the sensor is radar, reflections from the rear wheels 276A, 276B may appear as noise in the received radar signal. Therefore, in cases where the rear wheels 276A, 276B guide the radar signal away from the sensor, the radar can operate with an enhanced signal-to-noise ratio.

[0120] FIG3 is a conceptual illustration of wireless communication between various computing systems associated with an autonomous or semi-autonomous vehicle according to an example embodiment. In particular, wireless communication can occur between the remote computing system 302 and the vehicle 200 via network 304. Wireless communication can also occur between the server computing system 306 and the remote computing system 302, and between the server computing system 306 and the vehicle 200.

[0121] Vehicle 200 may correspond to various types of vehicles capable of transporting passengers or objects between locations, and may take the form of any one or more of the vehicles discussed above. In some instances, vehicle 200 may operate in an autonomous or semi-autonomous mode, which enables the control system to use sensor measurements to safely navigate vehicle 200 between destinations. When operating in autonomous or semi-autonomous mode, vehicle 200 may navigate with or without passengers. As a result, vehicle 200 may pick up and drop off passengers between desired destinations.

[0122] Telecomputing system 302 may represent any type of device associated with remote assistance technology, including but not limited to those described herein. In the example, telecomputing system 302 may represent any type of device configured to (i) receive information related to vehicle 200, (ii) provide an interface through which an operator may sequentially perceive information and input a response related to the information, and (iii) send the response to vehicle 200 or other devices. The remote computing system 302 can take various forms, such as a workstation, desktop computer, laptop computer, tablet computer, mobile phone (e.g., smartphone), and / or server. In some examples, the remote computing system 302 may include multiple computing devices operating together in a network configuration.

[0123] The remote computing system 302 may include one or more subsystems and components similar to or the same as those of the vehicle 200. At least, the remote computing system 302 may include a processor configured to perform the various operations described herein. In some embodiments, the remote computing system 302 may also include a user interface including input / output devices such as a touchscreen and speakers. Other examples are also possible.

[0124] The network 304 represents the infrastructure that enables wireless communication between the remote computing system 302 and the vehicle 200. The network 304 also enables wireless communication between the server computing system 306 and the remote computing system 302, and between the server computing system 306 and the vehicle 200.

[0125] The location of the remote computing system 302 may vary within the examples. For example, the remote computing system 302 may have a remote location relative to the vehicle 200, having wireless communication via network 304. In another example, the remote computing system 302 may correspond to a computing device within the vehicle 200, which is separate from the vehicle 200, but an operator can interact with the computing device while simultaneously being a passenger or driver of the vehicle 200. In some examples, the remote computing system 302 may be a computing device having a touchscreen operable by a passenger of the vehicle 200.

[0126] In some embodiments, the operations performed by the remote computing system 302 described herein may additionally or alternatively...The remote assistance is performed by vehicle 200 (i.e., by any system or subsystem of vehicle 200). In other words, vehicle 200 can be configured to provide a remote assistance mechanism that the driver or passenger of the vehicle can interact with, as described on pages 14 / 34 of the specification, 20 CN 122253769 A.

[0127] Server computing system 306 can be configured to wirelessly communicate with remote computing system 302 and vehicle 200 (or possibly directly with remote computing system 302 and / or vehicle 200) via network 304. Server computing system 306 can represent any computing device configured to receive, store, determine and / or transmit information relating to vehicle 200 and its remote assistance. Thus, server computing system 306 can be configured to perform any operation or part of such operation described herein as being performed by remote computing system 302 and / or vehicle 200. Some embodiments of wireless communication associated with remote assistance may utilize server computing system 306, while others may not.

[0128] The server computing system 306 may include one or more subsystems and components similar to or the same as those of the remote computing system 302 and / or the vehicle 200, such as processors configured to perform the various operations described herein, and wireless communication interfaces for receiving information from and providing information to the remote computing system 302 and the vehicle 200.

[0129] The various systems described above can perform various operations. These operations and related features will now be described.

[0130] Based on the discussion above, a computing system (e.g., the remote computing system 302, the server computing system 306, or the computing system local to the vehicle 200) may be operable to use a camera to capture images of the surrounding environment of an autonomous or semi-autonomous vehicle. Typically, at least one computing system will be able to analyze the images and may control the autonomous or semi-autonomous vehicle.

[0131] In some embodiments, to facilitate autonomous or semi-autonomous operation, the vehicle (e.g., vehicle 200) may receive data representing objects in the environment surrounding the vehicle (also referred to herein as “environmental data”) in various ways. The sensor system on the vehicle can provide environmental data representing objects in the surrounding environment. For example, the vehicle can have various sensors, including cameras, radar, lidar, microphones, radio units, and other sensors. Each of these sensors can transmit environmental data about information received by each respective sensor to a processor in the vehicle.

[0132] In one example, the camera can be configured to capture still images and / or video. In some embodiments, the vehicle can have more than one camera positioned in different orientations. Moreover, in some embodiments, the camera can be moved to capture images and / or video in different directions. The camera can be configured to store the captured images and video to storage.The captured images and / or videos may be environmental data. Additionally, the camera may include an image sensor as described herein.

[0133] In another example, the radar may be configured to transmit electromagnetic signals reflected by various objects near the vehicle, and then capture the electromagnetic signals reflected from the objects. The captured reflected electromagnetic signals may enable the radar (or processing system) to make various determinations about the objects reflecting the electromagnetic signals. For example, the distance to the various reflecting objects and the position of the various reflecting objects may be determined. In some embodiments, the vehicle may have more than one radar in different orientations. The radar may be configured to store the captured information in a memory for later processing by the vehicle's processing system. The information captured by the radar may be environmental data.

[0134] In another example, a lidar may be configured to transmit electromagnetic signals (e.g., infrared light, such as infrared light from a gas or diode laser or other possible light source) reflected by target objects near the vehicle. The lidar may be able to capture the reflected electromagnetic (e.g., infrared light) signals. The captured reflected electromagnetic signals may enable a ranging system (or processing system) to determine the distance to various objects. The lidar can also determine the rate or speed of a target object and store it as environmental data.

[0135] Additionally, in this example, a microphone can be configured to capture audio of the environment surrounding the vehicle. The sounds captured by the microphone can include emergency vehicle sirens and other vehicle sounds. For example, the microphone can capture the sirens of ambulances, fire trucks, or police cars. The processing system can be able to identify the captured audio signals to indicate an emergency vehicle. In another example, the microphone can capture the exhaust sound of another vehicle, such as the sound of a motorcycle. The processing system can be able to identify the captured audio signals to indicate a motorcycle. The data captured by the microphone can form part of the environmental data.

[0136] In yet another example, the radio unit can be configured to transmit electromagnetic signals, which can take the form of Bluetooth 802.11 signals and / or other radio technology signals. The first electromagnetic radiation signal can be transmitted via one or more antennas located in the radio unit. Furthermore, the first electromagnetic radiation signal can be transmitted using one of many different radio signaling modes. However, in some embodiments, it is desirable to transmit the first electromagnetic radiation signal in a signaling pattern requesting a response from a device located near the autonomous or semi-autonomous vehicle. The processing system may be able to detect nearby devices based on the response transmitted back to the radio unit and use the transmitted information as part of the environmental data.

[0137] In some embodiments, the processing system may be able to combine information from various sensors to further determineThe processing system can determine the surrounding environment of the vehicle. For example, the processing system can combine data from both radar information and captured images to determine whether another vehicle or pedestrian is in front of the autonomous or semi-autonomous vehicle. In other embodiments, the processing system can use other combinations of sensor data to make determinations about the surrounding environment.

[0138] When operating in autonomous (or semi-autonomous) mode, the vehicle can control its operation with almost no human input. For example, an operator can input an address into the vehicle, and the vehicle can then be able to travel to a designated destination without further input from a person (e.g., the person does not have to steer or touch the brake / accelerator pedal). In addition, when the vehicle is operating autonomously or semi-autonomously, the sensor system can receive environmental data. The vehicle's processing system can change the control of the vehicle based on the environmental data received from various sensors. In some examples, the vehicle can change its speed in response to environmental data from various sensors. The vehicle can change its speed to avoid obstacles, comply with traffic regulations, etc. When the processing system in the vehicle identifies an object near the vehicle, the vehicle can be able to change its speed or otherwise change its movement.

[0139] When the vehicle detects an object but lacks high confidence in the detection, the vehicle may request a human operator (or a more powerful computer) to perform one or more remote assistance tasks, such as (i) confirming whether the object is actually present in the surrounding environment (e.g., whether there is actually a stop sign or whether there is actually no stop sign), (ii) confirming whether the vehicle's identification of the object is correct, (iii) correcting the identification if it is incorrect, and / or (iv) providing supplementary instructions (or modifying the current instructions) to an automated or semi-automated vehicle. Remote assistance tasks may also include the operator providing instructions to control the vehicle's operation (e.g., instructing the vehicle to stop at the stop sign if the operator determines the object is a stop sign), although in some cases, the vehicle itself may control its own operation based on feedback from the operator related to object identification.

[0140] To facilitate this, the vehicle may analyze environmental data representing objects in the surrounding environment to identify at least one object with a detection confidence level below a threshold. A processor in the vehicle may be configured to detect various objects in the surrounding environment based on environmental data from various sensors. For example, in one embodiment, the processor may be configured to detect objects that may be important for the vehicle's identification. Such objects may include pedestrians, cyclists, street signs, other vehicles, indicator signals on other vehicles, and various other objects detected in the captured environmental data.

[0141] Detection confidence can indicate the likelihood that the identified object is correctly identified or present in the surrounding environment. For example, the processor can perform object detection on objects within image data in the received environmental data.And based on the inability to identify objects with a detection confidence level above a threshold, at least one object with a detection confidence level below a threshold is determined. If the result of object detection or object identification is uncertain, then the detection confidence level may be lower or below a set threshold.

[0142] The vehicle can detect objects in the surrounding environment in various ways depending on the source of the environmental data. In some embodiments, the environmental data may come from a camera and is image or video data. In other embodiments, the environmental data may come from a LiDAR (Light Detection and Control System) as described on page 16 / 34 of the specification, CN 122253769 A. The vehicle can analyze the captured image or video data to identify objects in the image or video data. The method and apparatus can be configured to monitor image and / or video data for the presence of objects in the surrounding environment. In other embodiments, the environmental data may be radar, audio, or other data. The vehicle can be configured to identify objects in the surrounding environment based on radar, audio, or other data.

[0143] In some embodiments, the technique used by the vehicle to detect objects may be based on a set of known data. For example, data related to environmental objects may be stored in a memory located in the vehicle. The vehicle can compare the received data with the stored data to determine objects. In other embodiments, the vehicle can be configured to determine objects based on the context of the data. For example, street signs associated with buildings are typically orange. Therefore, the vehicle can be configured to detect orange objects located near one side of the road as street signs associated with buildings. Additionally, when the vehicle's processing system detects an object in the captured data, it can also calculate a confidence level for each object.

[0144] Furthermore, the vehicle can also have a confidence threshold. The confidence threshold can vary depending on the type of object being detected. For example, the confidence threshold may be lower for objects that may require a rapid response from the vehicle (such as brake lights on another vehicle). However, in other embodiments, the confidence threshold can be the same for all detected objects. When the confidence level associated with a detected object is greater than the confidence threshold, the vehicle can assume that the object is correctly identified and adjust the vehicle's control accordingly based on that assumption.

[0145] When the confidence level associated with a detected object is less than the confidence threshold, the actions taken by the vehicle can vary. In some embodiments, the vehicle can react as if the detected object were present, even though the confidence level is low. In other embodiments, the vehicle can react as if the detected object does not exist.

[0146] When the vehicle detects an object in the surrounding environment, it can also calculate a confidence level associated with the specific detected object. Depending on the embodiment, the confidence level can be calculated in various ways. In one example, when detecting an object in the surrounding environment, the vehicle can compare environmental data with predetermined data about known objects. The environmental data and the predetermined data...The closer the match, the higher the confidence level. In other embodiments, the vehicle may use mathematical analysis of environmental data to determine the confidence level associated with the object.

[0147] In response to determining that the object has a detection confidence level below a threshold, the vehicle may send a request for remote assistance in object identification to a remote computing system. As described above, the remote computing system may take various forms. For example, the remote computing system may be a computing device within the vehicle, separate from the vehicle, but which an operator may interact with while being a passenger or driver of the vehicle, such as a touchscreen interface for displaying remote assistance information. Additionally or alternatively, as another example, the remote computing system may be a remote computer terminal or other device located at a location not near the vehicle.

[0148] The remote assistance request may include environmental data containing the object, such as image data, audio data, etc. The vehicle may send the environmental data to the remote computing system via a network (e.g., network 304) and, in some embodiments, via a server (e.g., server computing system 306). An operator of the remote computing system may, in turn, use the environmental data as the basis for responding to the request.

[0149] In some embodiments, when an object is detected as having a confidence level below a confidence threshold, the object may be given preliminary identification, and the vehicle may be configured to adjust the operation of the vehicle in response to the preliminary identification. Such operational adjustment may take the form of stopping the vehicle, switching the vehicle to a manual control mode, changing the vehicle's speed (e.g., rate and / or direction), and other possible adjustments.

[0150] In other embodiments, even if the vehicle detects an object with a confidence level that reaches or exceeds the threshold, the vehicle may operate according to the detected object (e.g., stop if the object is identified as a stop sign with high confidence), but may be configured to request remote assistance while (or later) the vehicle is operating according to the detected object.

[0151] FIG4A is a block diagram of a system according to an example embodiment. In particular, FIG4A shows a system 400, which includes a system controller 402, a lidar device 410, a plurality of sensors 412, and a plurality of controllable components 414. System controller 402 includes processor 404, memory 406, and instructions 408 stored in memory 406 and executable by processor 404 to perform functions.

[0152] Processor 404 may include one or more processors, such as one or more general-purpose microprocessors (e.g., having a single core or multiple cores) and / or one or more dedicated microprocessors. One or more processors may include, for example, one or more central processing units (CPUs), one or more microcontrollers, one or more graphics processing units (GPUs), or one or more...Tensor processing units (TPUs), one or more ASICs and / or one or more field-programmable gate arrays (FPGAs). Other types of processors, computers, or devices configured to execute software instructions are also considered herein.

[0153] Memory 406 may include computer-readable media, such as non-transitory computer-readable media, which may include, but is not limited to, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile random access memory (e.g., flash memory), solid-state drive (SSD), hard disk drive (HDD), optical disc (CD), digital video optical disc (DVD), digital magnetic tape, read / write (R / W) CD, R / W DVD, etc.

[0154] The lidar device 410 described further below includes a plurality of light emitters configured to emit light (e.g., in the form of light pulses) and one or more photodetectors configured to detect light (e.g., reflected portions of light pulses). The lidar device 410 can generate three-dimensional (3D) point cloud data from the output of the photodetector and provide the 3D point cloud data to the system controller 402. The system controller 402 can then perform operations on the 3D point cloud data to determine characteristics of the surrounding environment (e.g., relative positions of objects within the surrounding environment, edge detection, object detection, and proximity sensing).

[0155] Similarly, the system controller 402 can use the outputs from multiple sensors 412 to determine characteristics of the system 400 and / or the surrounding environment. For example, the sensors 412 may include one or more of GPS, IMU, image capture devices (e.g., cameras), light sensors, thermal sensors, and other sensors that indicate parameters related to the system 400 and / or the surrounding environment. For illustrative purposes, the lidar device 410 is depicted as separate from the sensors 412, and in some examples may be considered as part of or considered as the sensors 412.

[0156] Based on the characteristics of system 400 and / or the surrounding environment determined by system controller 402 based on the outputs from lidar device 410 and sensor 412, system controller 402 can control controllable component 414 to perform one or more actions. For example, system 400 may correspond to a vehicle, in which case controllable component 414 may include the vehicle's braking system, steering system, and / or acceleration system, and system controller 402 may modify aspects of these controllable components based on the characteristics determined from lidar device 410 and / or sensor 412 (e.g., when system controller 402 controls the vehicle in an autonomous or semi-autonomous mode). In this example, lidar device 410 and sensor 412 may also be controlled by system controller 402.

[0157] Figure 4B is a block diagram of a lidar device according to an example embodiment. Specifically, Figure 4B shows a lidar device 410 having a controller 416 configured to control a plurality of light emitters 424 and one or more photodetectors (e.g., a plurality of photodetectors 426, etc.). The lidar device 410 also includes an ignition circuit 428 configured to select and supply power to a respective light emitter among the plurality of light emitters 424, and may include a selector circuit 430 configured to select a respective photodetector among the plurality of photodetectors 426. The controller 416 includes a processor 418, a memory 420, and instructions 422 stored in the memory 420.

[0158] Similar to processor 404, processor 418 may include one or more processors, such as one or more general-purpose microprocessors and / or one or more special-purpose microprocessors. One or more processors may include, for example, one or more CPUs, one or more microcontrollers, one or more GPUs, one or more TPUs, one or more ASICs, and / or one or more FPGAs. This document also considers other types of processors, computers, or devices configured to execute software instructions.

[0159] Similar to memory 406, memory 420 may include computer-readable media, such as non-transitory computer-readable media, such as, but not limited to, ROM, PROM, EPROM, EEPROM, non-volatile random access memory (e.g., flash memory), SSD, HDD, CD, DVD, digital magnetic tape, R / W CD, R / W DVD, etc.

[0160] Instructions 422 are stored on memory 420 and are executable by processor 418 to perform functions associated with control ignition circuit 428 and selector circuit 430 for generating 3D point cloud data, and for processing the 3D point cloud data (or possibly facilitating processing of the 3D point cloud data by another computing device such as system controller 402).

[0161] Controller 416 may determine 3D point cloud data by emitting light pulses using light emitter 424. Emission time is established for each light emitter, and the relative position at the time of emission is also tracked. The LiDAR device 410 is surrounded by various objects that reflect light pulses. For example, when the LiDAR device 410 is in an environment including a road, such objects may include vehicles, signs, pedestrians, road surfaces, or building cones. Some objects may be more reflective than others, such that the intensity of the reflected light can indicate the type of object reflecting the light pulse. Furthermore, the surface of an object may be in a different position relative to the LiDAR device 410, and therefore take more or less time to reflect a portion of the light pulse back to the LiDAR device 410. Therefore, the controller 416 can track the detection time of the reflected light pulse detected by the photodetector and the light...The relative position of the detector at the detection time. By measuring the time difference between the emission time and the detection time, the controller 416 can determine how far the light pulse travels before being received, and thus determine the relative distance to the corresponding object. By tracking the relative position of the emission time and the detection time, the controller 416 can determine the orientation of the light pulse and the reflected light pulse relative to the lidar device 410, thereby determining the relative orientation of the object. By tracking the intensity of the received light pulse, the controller 416 can determine how reflective the object is. Therefore, the 3D point cloud data determined based on this information can indicate the relative position of the detected reflected light pulse (e.g., in a coordinate system such as a Cartesian coordinate system) and the intensity of each reflected light pulse.

[0162] The ignition circuit 428 is used to select the light emitter for emitting the light pulse. The selector circuit 430 is similarly used to sample the output from the photodetector.

[0163] As described above, the camera system may include multiple cameras and / or sensors, each camera and / or sensor having different properties based on differences in one or more components of the camera. For example, each camera may include one or more lenses housed within the lens barrel of the camera. Each camera may also include one or more image sensors. Each image sensor may receive light from the scene via one or more lenses and mirrors housed within the lens barrel. The camera may also include additional components (e.g., shutter button, viewfinder, flash, battery, electronic memory for recording captured images, display screen, or selection button).

[0164] In some embodiments, the lens barrel of one or more cameras may be configured to rotate about its axis to modify the relative position of one or more lenses within the lens barrel, thereby adjusting the camera's field of view and / or zoom. For example, one or more components in the camera may be electronically controlled (e.g., a camera controller may adjust one or more lenses within the lens barrel to modify the camera's zoom, e.g., during an autofocus process).

[0165] The camera may include only image sensors behind one or more lenses (e.g., telecentric lenses). Other arrangements are also possible. In some embodiments, for example, the camera may include one or more optical filters (e.g., polarizing filters, color filters, or neutral density filters) and / or one or more electronic stages and / or motors configured to adjust the position of one or more components of the camera.

[0166] As previously described, in some embodiments, the camera can be used for object detection and avoidance within an autonomous vehicle (e.g., similar to camera 130 illustrated and described with reference to FIG1). FIGS. 5A to 5D show example cameras that can be incorporated (e.g., in various numbers, on pages 19 / 34 of specification 25 CN 122253769 A and / or in various locations) into the camera systems disclosed herein.

[0167] FIG. 5A is a camera of first camera type 502 with an associated first field of view 503 according to an example embodiment.The illustration is shown. In some embodiments, a camera of the first camera type 502 may be mounted on a vehicle such as vehicle 100. In some embodiments, images captured by the camera of the first camera type 502 may be used to identify objects at a first distance range from the vehicle.

[0168] FIG5B is an illustration of a camera of the second camera type 504 having an associated second field of view 505 according to an example embodiment. The camera of the second camera type 504 may be mounted on a vehicle such as vehicle 100. The second field of view 505 may span a larger angle (e.g., an angular direction about the z-axis) in yaw relative to vehicle 100 than the first field of view 503. In some embodiments, the second field of view 505 spans at least 170° in yaw relative to vehicle 100. Images captured by the camera of the second camera type 504 may be used to identify objects at a second distance range from vehicle 100. In some embodiments, the first distance range includes distances from vehicle 100 that are farther than those included in the second distance range.

[0169] FIG5C is an illustration of a camera of third camera type 506 having an associated third field of view 507 according to an example embodiment. The camera of third camera type 506 may be mounted on a vehicle such as vehicle 100. In some embodiments, the combined field of view of a plurality of cameras of third camera type 506 may span 360° in yaw relative to vehicle 100. Compared to third field of view 507, first field of view 503 may span less angle in yaw relative to vehicle 100. Compared to second field of view 505, third field of view 507 may span less angle in yaw relative to vehicle 100. Images captured by the camera of third camera type 506 may be used to identify objects at a third distance range from vehicle 100. In some embodiments, the third distance range may include distances from vehicle 100 that are farther than distances included in a second distance range. The first distance range may include distances from vehicle 100 that are farther than distances included in the third distance range.

[0170] The camera of the first camera type 502, the camera of the second camera type 504, and / or the camera of the third camera type 506 may have camera lenses with focal lengths ranging from about 1.5 mm to about 25 mm. Furthermore, the camera of the first camera type 502, the camera of the second camera type 504, and / or the camera of the third camera type 506 may have a total dynamic range of less than 120 dB. The camera of the first camera type 502 may have an image resolution of about 17 megapixels. The camera of the second camera type 504 may have an image resolution of about 2 megapixels. The camera of the third camera type 506 may have an image resolution of about 8 megapixels.

[0171] FIG5D is a camera of the fourth camera type 508 with an associated fourth field of view 509 according to an example embodiment.The illustration is shown. A camera of fourth camera type 508 can be mounted on a vehicle (such as vehicle 100). A fourth field of view 509 can span a larger angle in yaw relative to vehicle 100 than a first field of view 503. A fourth field of view 509 can span a smaller angle in yaw relative to vehicle 100 than a third field of view 507. Images captured by the camera of fourth camera type 508 can be used to identify objects at a fourth distance range from vehicle 100. In some embodiments, the fourth distance range may include distances from vehicle 100 that are farther than those included in the third distance range. The first distance range may include distances from vehicle 100 that are farther than those included in the fourth distance range.

[0172] FIG. 6A is an illustration of a camera system 600 according to an example embodiment. As shown, multiple cameras can be mounted on vehicle 100. As shown in FIG. 6A, in some example embodiments, three cameras of first camera type 502 may be present. Each camera of first camera type 502 may have an associated first field of view 503. Each of these cameras of the first camera type 502 may be oriented in the forward direction relative to the direction of travel of the vehicle 100. Although the direction of travel shown in FIG. 6A is the negative x-direction, other directions of travel are also possible. For example, the direction of travel may alternatively be the positive x-direction (e.g., the vehicle 100 may travel in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, two, four, five, etc.) and / or other attachment locations of cameras of the first camera type 502 (e.g., on the underside of the vehicle 100 or inside the vehicle specification 20 / 34 page 26 CN 122253769 A 100) are also possible.

[0173] As used herein, the phrase “oriented in the forward direction” means that the central optical axis of the corresponding field of view is oriented approximately forward or at least parallel to the direction of travel of the vehicle. Similarly, as used herein, the phrase “oriented in the rearward direction” means that the central optical axis of the corresponding field of view is oriented approximately rearward or at least parallel to the rearward direction relative to the vehicle's direction of travel. Furthermore, as used herein, the phrases “one or more lateral directions” and “one or more transverse directions” mean that the central optical axis of the corresponding field of view is yawed approximately 90° relative to the vehicle's direction of travel. Additionally, as used herein, the phrase “at least partially oriented” in that direction means that at least a portion of the field of view overlaps with that direction (but not necessarily the central axis of the corresponding field of view). For example, “at least partially oriented in the forward direction” means that at least a portion of the field of view overlaps with the forward direction relative to the vehicle's direction of travel.

[0174] As illustrated in FIG6A, in some embodiments, the camera system 600 may include four of a second camera type 504.Cameras. Each camera of the second camera type 504 may have an associated second field of view 505. One of the cameras of the second camera type 504 may be oriented in a forward direction relative to the direction of travel of the vehicle 100. One of the cameras of the second camera type 504 may be oriented in a rearward direction relative to the direction of travel of the vehicle 100. Two cameras, each of the second camera type 504, may be oriented in one or more lateral directions (e.g., parallel to the y-axis) relative to the direction of travel of the vehicle 100. Other numbers of cameras of the second camera type 504 (e.g., one, two, three, or five) and / or other attachment locations of the cameras of the second camera type 504 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0175] As illustrated in FIG6A, in some embodiments, the camera system 600 may have five cameras of the third camera type 506. Each camera of the third camera type 506 may have an associated third field of view 507. In some embodiments, the camera of the third camera type 506 may be oriented in a forward direction relative to the direction of travel of the vehicle 100. Two cameras, each of the third camera type 506, may be oriented at least partially in a rearward direction relative to the direction of travel of the vehicle 100. Furthermore, two cameras, each of the third camera type 506, may be oriented in one or more lateral directions relative to the direction of travel of the vehicle 100. Other numbers of the third camera type 506 (e.g., one, two, three, four, six, etc.) and / or other attachment locations of the third camera type 506 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0176] The camera of the first camera type 502 may be attached to the vehicle adjacent to the camera of the third camera type 506. As used herein, the phrase “adjacent” means within 10 cm. However, in some embodiments, the cameras may be closer to or further away from each other (e.g., within 1 meter or 1 cm of each other). Additionally, the two cameras of the second camera type 504 can each be attached to the vehicle 100 adjacent to a camera of the third camera type 506. Using fewer cameras in a camera system can increase the speed at which this camera system can detect and / or identify objects (e.g., because the camera system will process less data to make a determination, since fewer cameras are used to capture images). Figure 6A depicts a large amount of information. Therefore, for clarity, Figures 6B-6D are provided to illustrate the camera system 600 of Figure 6A, but all camera types except one are removed from the figures (e.g., to eliminate clutter).

[0177] Figure 6B is a simplified illustration of the camera system 600 of Figure 6A according to an exemplary embodiment, showing the camera system 600.Cameras of the first camera type 502. Each of the cameras of the first camera type 502 can be mounted on the roof of the vehicle 100. This allows the camera system 600 to capture objects at a greater distance from the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible. The two cameras of the first camera type 502 can be oriented at least partially in the forward direction relative to the direction of travel of the vehicle 100. This provides a combined field of view that spans a larger angle in terms of yaw relative to the vehicle 100, thereby facilitating the detection of objects that may enter the forward movement path of the vehicle 100.

[0178] FIG6C is a simplified illustration of the camera system 600 of FIG6A according to an exemplary embodiment, showing cameras of the second camera type 504 of the camera system 600. According to FIG6C, four cameras of the second camera type 504 may be present. As can be seen from Figure 6A, the combined field of view of the cameras of the second camera type 504 can span 360° in yaw relative to the vehicle 100. This facilitates the detection of objects positioned close to the vehicle 100 in the area surrounding the vehicle 100. One of the cameras of the second camera type 504 can be attached to the front bumper of the vehicle 100. One of the cameras of the second camera type 504 can be attached to the rear bumper of the vehicle 100. Two of the cameras of the second camera type 504 can be attached to one or more side bumpers of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). Positioning the cameras of the second camera type 504 in this way facilitates the detection of objects close to the road surface (e.g., less than 1 meter above the road surface) that may intersect the driving direction of the vehicle 100. Other attachment locations (e.g., in the tire wells or near the side mirrors) are also possible.

[0179] Figure 6D is a simplified illustration of the camera system 600 of Figure 6A according to an exemplary embodiment, showing a camera of the third camera type 506 of the camera system 600. As can be seen from Figure 6D, five cameras of the third camera type 506 can be present, and the combined field of view of the cameras of the third camera type 506 can span 360° in yaw relative to the vehicle 100. Three cameras of the third camera type 506 can be attached to the roof of the vehicle 100. Two cameras of the third camera type 506 can each be attached to one or more side bumpers of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). Such a configuration facilitates the detection of objects within a third distance range that may intersect the driving direction of the vehicle 100. Other attachment locations (e.g., in the tire wells or near the side mirrors) are also possible.

[0180] Figure 7A is an illustration of a camera system 700 according to an exemplary embodiment. As can be seen from Figure 7A, five cameras of the third camera type 506 can be present.Four cameras of the first camera type 502 are provided to the vehicle 100, each camera having a first field of view 503. Each of the cameras of the first camera type 502 can be oriented at least partially in a forward direction relative to the travel direction of the vehicle 100. Including multiple cameras of the first camera type 502 facilitates the detection of objects forward relative to the travel direction of the vehicle 100, because each camera of the first camera type 502 is capable of independently detecting objects forward relative to the travel direction of the vehicle 100. Although the travel direction shown in FIG. 7A is the negative x-direction, other travel directions are also possible. For example, the travel direction can alternatively be the positive x-direction (e.g., the vehicle 100 can travel in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, two, four, five, etc.) and / or other attachment locations of the cameras of the first camera type 502 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0181] As seen in FIG7A, there may be five cameras of a second camera type 504 attached to the vehicle 100, each camera having a second field of view 505. One camera of the second camera type 504 may be oriented in a forward direction relative to the driving direction of the vehicle 100. Two cameras of the second camera type 504 may each be oriented in one or more lateral directions relative to the driving direction of the vehicle 100. Two cameras of the second camera type 504 may each be oriented in a rearward direction relative to the driving direction of the vehicle 100. This configuration facilitates the detection of objects within a second distance range rearward relative to the driving direction of the vehicle 100, because the fields of view of the three cameras of the second camera type 504 may overlap in one or more areas behind the vehicle 100, and each camera of the second camera type 504 may be able to independently detect objects rearward relative to the driving direction of the vehicle 100. Other numbers of cameras of the second camera type 504 (e.g., one, two, three, four, or six) and / or other attachment locations of cameras of the second camera type 504 (e.g., on the underside of vehicle 100 or inside vehicle 100) are also possible.

[0182] Also depicted in FIG7A, there may be five cameras of the third camera type 506 attached to vehicle 100, each camera having a third field of view 507. The cameras of the third camera type 506 may be oriented in a forward direction relative to the direction of travel of vehicle 100. Two cameras of the third camera type 506 may each be at least partially oriented in a rearward direction relative to the direction of travel of vehicle 100. Furthermore, two cameras of the third camera type 506 may each be at least partially oriented in one or more lateral directions relative to the direction of travel of vehicle 100. Other numbers of cameras of the third camera type 506Other attachment locations for the number (e.g., one, two, three, four, or six) and / or the camera of the third camera type 506 (e.g., on the underside of the vehicle 100 or inside the vehicle 100) are also possible.

[0183] Furthermore, as depicted in FIG7A, at least three cameras of the first camera type 502 may each be attached to the vehicle 100 adjacent to at least three cameras of the third camera type 506. Furthermore, at least two cameras of the second camera type 504 may each be attached to the vehicle 100 adjacent to at least two cameras of the third camera type 506. FIG7A depicts a great deal of information. Therefore, for clarity, FIGS7B-7D are provided to illustrate the camera system 700 of FIG7A, but all camera types except one camera type have been removed from the figures (e.g., to eliminate clutter).

[0184] FIG7B is a simplified illustration of the camera system 700 of FIG7A according to an exemplary embodiment, showing the camera of the first camera type 502 of the camera system 700. As depicted in FIG7B, the four cameras of the first camera type 502 can each be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0185] FIG7C is a simplified illustration of the camera system 700 of FIG7A according to an exemplary embodiment, showing the cameras of the second camera type 504 of the camera system 700. As depicted in FIG7C, one camera of the second camera type 504 can be attached to the front bumper of the vehicle 100. Furthermore, as depicted in FIG7C, the four cameras of the second camera type 504 can each be attached to the roof of the vehicle 100. In addition, as shown in FIG7C, two cameras of the second camera type 504 can be attached to the vehicle above the rear wheel wells of the vehicle 100. This configuration facilitates the detection of objects within a second distance range relative to the driving direction of vehicle 100, because the fields of view of the three cameras of the second camera type 504 may overlap in one or more areas behind vehicle 100, and each camera of the second camera type 504 may be able to independently detect objects within the second distance range from vehicle 100. Other attachment locations (e.g., above or inside a tire well or near a side mirror) are also possible.

[0186] FIG7D is a simplified illustration of the camera system 700 of FIG7A according to an exemplary embodiment, showing the cameras of the third camera type 506 of the camera system 700. As depicted in FIG7D, the five cameras of the third camera type 506 may be attached to the roof of vehicle 100. Attaching all the cameras of the third camera type 506 to the roof can help detect objects within the third distance range and spanning 360° in yaw relative to vehicle 100, because all the cameras of the third camera type 506 can be at approximately the same distance from the road surface. If all the cameras of the third camera type 506 are at approximately the same distance from the road surface, thenWhen a computing device (e.g., system controller 402) uses images captured by a camera of third camera type 506 to detect objects within a third distance range of vehicle 100, there may be fewer reasons to compensate for the distance difference between the camera of third camera type 506 and the road surface. Other attachment locations (e.g., above or inside a tire well or near a side mirror) are also possible.

[0187] FIG8A is an illustration of a camera system 800 according to an example embodiment. As depicted in FIG8A, three cameras of first camera type 502 may be present, each at least partially oriented in a forward direction relative to the driving direction of vehicle 100. Each camera of first camera type 502 may have an associated first field of view 503. Although the driving direction shown in FIG8A is the negative x-direction, other driving directions are also possible. For example, the driving direction may alternatively be the positive x-direction (e.g., vehicle 100 may be driving in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, two, four, or five) and / or other attachment locations of the cameras of the first camera type 502 (e.g., on the underside of the vehicle 100 or inside the vehicle 100) are also possible.

[0188] Furthermore, as depicted in FIG8A, four cameras of the second camera type 504 may be present, each camera having an associated second field of view 505. One of the cameras of the second camera type 504 may be oriented in a forward direction relative to the driving direction of the vehicle 100. One of the cameras of the second camera type 504 may be oriented in a rearward direction relative to the driving direction of the vehicle 100. Two of the cameras of the second camera type 504 may each be oriented in one or more lateral directions relative to the driving direction of the vehicle 100. Other numbers of cameras of the second camera type 504 (e.g., one, two, three, or five) and / or other attachment locations of cameras of the second camera type 504 (e.g., on the underside of vehicle 100 or inside vehicle 100) are also possible.

[0189] Furthermore, as seen in FIG8A, six cameras of the third camera type 506 may be present, each having an associated third field of view 507. One camera of the third camera type 506 may be oriented in a forward direction relative to the direction of travel of vehicle 100. Two cameras of the third camera type 506 may be oriented at least partially in a rearward direction relative to the direction of travel of vehicle 100. Furthermore, three cameras of the third camera type 506 may each be oriented at least partially in one or more lateral directions relative to the direction of travel of vehicle 100. Such a configuration facilitates the detection of distances from the vehicle located on one side of vehicle 100 (e.g., the right side of vehicle 100, the left side of vehicle 100, the driver's side of vehicle 100, or the passenger side of vehicle 100).Objects within a third distance range of 100. This is because the fields of view of the three cameras of the third camera type 506 may overlap in one or more areas on one side of the vehicle 100, and each camera of the third camera type 506 may be able to independently detect objects within a third distance range of the vehicle 100. Therefore, facilitating the detection of objects within a third distance range of the vehicle 100 can facilitate the vehicle 100 performing turning maneuvers (e.g., turning left or right). Other numbers of cameras of the third camera type 506 (e.g., one, two, three, four, five, or seven) and / or other attachment locations of the cameras of the third camera type 506 (e.g., on the underside of the vehicle 100 or inside the vehicle 100) are also possible.

[0190] Furthermore, as can be seen from FIG. 8A, at least one camera of the first camera type 502 may be attached to the vehicle 100 adjacent to at least one camera of the third camera type 506. Furthermore, at least two cameras of the second camera type 504 may be attached to the vehicle 100 adjacent to at least two cameras of the third camera type 506.

[0191] FIG8B is a simplified illustration of the camera system 800 of FIG8A according to an exemplary embodiment, showing a camera of a first camera type 502 of the camera system 800. As depicted in FIG8B, each of the cameras of the first camera type 502 may be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0192] FIG8C is a simplified illustration of the camera system 800 of FIG8A according to an exemplary embodiment, showing a camera of a second camera type 504 of the camera system 800. One camera of the second camera type 504 may be attached to the front bumper of the vehicle 100. Furthermore, both cameras of the second camera type 504 may each be attached to one or more side bumpers of the vehicle 100 (e.g., above one or more wheel wells of the vehicle 100). Furthermore, one camera of the second camera type 504 may be attached to the roof of the vehicle 100. Other attachment locations (e.g., inside the tire wells or near the side mirrors) are also possible.

[0193] FIG8D is a simplified illustration of the camera system 800 of FIG8A according to an exemplary embodiment, showing cameras of a third camera type 506 of the camera system 800. As depicted in FIG8D, five cameras of the third camera type 506 can be attached to the roof of the vehicle 100. Attaching the five cameras of the third camera type 506 to the roof of the vehicle 100 can help detect objects within a third distance range and spanning 360° in yaw relative to the vehicle 100, because these cameras of the third camera type 506 can be positioned at approximately the same distance from the road surface. If these cameras of the third camera type 506 are at approximately the same distance from the road surface, then when using images captured by the cameras of the third camera type 506 to detect objects within the third distance range of the vehicle 100...When dealing with objects, the computing device (e.g., system controller 402) may not need to compensate for the distance difference between the camera of the third camera type 506 and the road surface. Furthermore, one camera of the second camera type 504 may be attached to the side bumper of the vehicle 100 (e.g., above the left front wheel well of the vehicle 100). Other attachment locations (e.g., inside the tire well or near the side mirror) are also possible.

[0194] FIG9A is an illustration of a camera system 900 according to an example embodiment. As depicted in FIG9A, two cameras of the first camera type 502 may be present, each camera having an associated first field of view 503. Each camera of the first camera type 502 may be oriented in a forward direction relative to the driving direction of the vehicle 100. Although the driving direction shown in FIG9A is the negative x direction, other driving directions are also possible. For example, the driving direction may alternatively be the positive x direction (e.g., the vehicle 100 may be driving in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, three, four, or five) and / or other attachment locations of cameras of the first camera type 502 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0195] Furthermore, four cameras of the second camera type 504 may be present, each camera having an associated second field of view 505. One camera of the second camera type 504 may be oriented in a forward direction relative to the travel direction of the vehicle 100. Furthermore, one camera of the second camera type 504 may be oriented in a rearward direction relative to the travel direction of the vehicle 100. Additionally, two cameras of the second camera type 504 may be oriented relative to each other in one or more lateral directions relative to the travel direction of the vehicle 100. Furthermore, other numbers of cameras of the second camera type 504 (e.g., one, two, three, or five) and / or other attachment locations of cameras of the second camera type 504 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0196] As depicted in FIG9A, eight cameras of third camera type 506 may be present, each camera having an associated third field of view 507. Two cameras of third camera type 506 may be at least partially oriented in a forward direction relative to the travel direction of vehicle 100. Furthermore, two cameras of third camera type 506 may be at least partially oriented in a rearward direction relative to the travel direction of vehicle 100. Additionally, four cameras of third camera type 506 may each be oriented in one or more lateral directions relative to the travel direction of vehicle 100. Furthermore, two cameras of fourth camera type 508 may be present, each camera having an associated fourth field of view 509. These cameras of fourth camera type 508 may be at least partially oriented in a forward direction relative to the travel direction of vehicle 100.In the forward direction relative to the direction of travel of the vehicle 100. Cameras of the first camera type 502 and the fourth camera type 508, which are oriented in the forward direction or at least partially oriented in the forward direction relative to the direction of travel of the vehicle 100, can conveniently detect objects at least partially oriented in the forward direction relative to the direction of travel of the vehicle 100. Other numbers of cameras of the third camera type 506 (e.g., one, two, three, four, five, six, seven, or nine) and / or other attachment locations of the cameras of the third camera type 506 (e.g., on the underside of the vehicle 100 or inside the vehicle 100) are also possible.

[0197] As depicted in FIG9A, two cameras of the first camera type 502 may each be attached to the vehicle 100 adjacent to a camera of the fourth camera type 508. Each camera of the fourth camera type 508 may also be attached to the vehicle 100 adjacent to a camera of the third camera type 506. Furthermore, the two cameras of the second camera type 504 can each be attached to the vehicle 100 adjacent to the two cameras of the third camera type 506.

[0198] FIG9B is a simplified illustration of the camera system 900 of FIG9A according to an exemplary embodiment, showing the camera of the first camera type 502 of the camera system 900. As shown in FIG9B, the two cameras of the first camera type can be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0199] FIG9C is a simplified illustration of the camera system 900 of FIG9A according to an exemplary embodiment, showing the camera of the second camera type 504 of the camera system 900. One camera of the second camera type 504 can be attached to the roof of the vehicle 100. Furthermore, one camera of the second camera type 504 can be attached to the front bumper of the vehicle 100. Furthermore, one camera of the second camera type 504 can be attached to the rear bumper of the vehicle 100. Other attachment locations (e.g., above or inside a tire well or near a side mirror) are also possible.

[0200] As depicted in FIG9C, a first predetermined camera 504A of a plurality of cameras of a second camera type 504 attached to the roof of the vehicle 100 may be present, having an associated first predetermined camera field of view 505A. The first predetermined camera 504A of the plurality of cameras may be a camera of the second camera type 504 including a neutral density filter through which images are captured. The first predetermined camera 504A of the plurality of cameras of the second camera type 504 may be oriented relative to the vehicle 100 in order to capture images from a pitch angle relative to the vehicle 100 (e.g., rotation about the z-axis).For example, the pitch angle is greater than 5° in the forward direction relative to the travel direction of the vehicle 100. Such an orientation facilitates the detection of objects, such as traffic lights, as described in more detail below. Other attachment locations (e.g., above tire wells or near side mirrors) are also possible.

[0201] FIG9D is a simplified illustration of the camera system 900 of FIG9A according to an exemplary embodiment, showing cameras of a third camera type 506 of the camera system 900. As depicted in FIG9D, eight cameras of the third camera type 506 can be attached to the roof of the vehicle 100. Attaching the eight cameras of the third camera type 506 to the roof of the vehicle 100 can help detect objects within a third distance range and spanning 360° in yaw relative to the vehicle 100, because these cameras of the third camera type 506 can be positioned at approximately the same distance from the road surface. If the cameras of these third camera types 506 are at approximately the same distance from the road surface, then when using images captured by the cameras of the third camera type 506 to detect objects within a third distance range of vehicle 100, the computing device (e.g., system controller 402) may not need to compensate for the distance differences between the cameras of the third camera type 506 and the road surface. Other attachment locations (e.g., above or inside a tire well or near a side mirror) are also possible. This configuration facilitates the detection of objects within a third distance range yawed 360° relative to vehicle 100 because the fields of view of two or more cameras of the third camera type 506 can overlap in the area yawed 360° relative to vehicle 100, and each camera of the third camera type 506 can independently detect objects within a third distance range yawed 360° relative to vehicle 100.

[0202] FIG9E is a simplified illustration of the camera system 900 of FIG9A according to an exemplary embodiment, showing the cameras of the fourth camera type 508 of the camera system 900. As depicted in FIG9E, the two cameras of the fourth camera type 508 may be attached to the roof of vehicle 100. Other attachment locations (e.g., above or inside a tire well or near a side mirror) are also possible.

[0203] FIG10A is an illustration of a camera system 1000 according to an example embodiment. As depicted in FIG10A, the camera system 1000 may include two cameras of a first camera type 502, each camera having an associated first field of view 503. Each camera of the first camera type 502 may be oriented in a forward direction relative to the driving direction of the vehicle 100. Although the driving direction shown in FIG10A is the negative x direction, other driving directions are also possible. For example, the driving direction may alternatively be the positive x direction (e.g., the vehicle 100 may be driving in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, three, four or five) and / or other attachment locations of the cameras of the first camera type 502 (e.g., on or inside a tire well or near a side mirror) are possible.It is also possible to have cameras located on the underside or inside the vehicle 100.

[0204] Furthermore, as depicted in FIG10A, the camera system 1000 may include four cameras of a second camera type 504, each camera having an associated second field of view 505. One camera of the second camera type 504 may be oriented in a forward direction relative to the direction of travel of the vehicle 100. Furthermore, one camera of the second camera type 504 may be oriented in a rearward direction relative to the direction of travel of the vehicle 100. Additionally, two cameras of the second camera type 504 may be oriented in one or more lateral directions relative to the direction of travel of the vehicle 100. Furthermore, other numbers of cameras of the second camera type 504 (e.g., one, two, three, or five) and / or other attachment locations of the cameras of the second camera type 504 (e.g., on the underside or inside the vehicle 100) are also possible.

[0205] As depicted in FIG10A, eight cameras of a third camera type 506 may be present, each camera having an associated third field of view 507. In this configuration, two cameras of the third camera type 506 can be at least partially oriented in a forward direction relative to the travel direction of the vehicle 100. Furthermore, two cameras of the third camera type 506 can be at least partially oriented in a rearward direction relative to the travel direction of the vehicle 100. Additionally, all four cameras of the third camera type 506 can be each oriented in one or more lateral directions relative to the travel direction of the vehicle 100, as described on pages 26 / 34 of the specification (CN 122253769 A). This configuration facilitates the detection of objects entering the rear movement path of the vehicle 100 within a third distance range from the vehicle 100, because the fields of view of the three cameras of the third camera type 506 may overlap in one or more areas behind the vehicle 100, and each camera of the third camera type 506 may be able to independently detect objects within a third distance range from the vehicle 100. Other numbers of cameras of the third camera type 506 (e.g., one, two, three, four, five, six, seven, or nine) and / or other attachment locations of the cameras of the third camera type 506 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0206] In some example embodiments, two cameras of the fourth camera type 508 may be present, each having an associated fourth field of view 509. These cameras of the fourth camera type 508 may be at least partially oriented in the forward direction relative to the direction of travel of the vehicle 100. Including cameras of the first camera type 502 and cameras of the fourth camera type 508 that are oriented in the forward direction relative to the direction of travel of the vehicle 100 or at least partially oriented in the forward direction, it is convenient to detect objects that are at least partially oriented in the forward direction relative to the direction of travel of the vehicle 100. The fourth camera type 508Other numbers of cameras (e.g., one, three, or four) and / or other attachment locations of the cameras of the fourth camera type 508 (e.g., on the underside of the vehicle 100 or inside the vehicle 100) are also possible.

[0207] Furthermore, as depicted in FIG10A, at least two cameras of the first camera type 502 may each be attached to the vehicle 100 adjacent to a camera of the fourth camera type 508. Furthermore, at least two cameras of the first camera type 502 may each be attached to the vehicle 100 adjacent to a camera of the third camera type 506. Furthermore, at least two cameras of the fourth camera type 508 may each be attached to the vehicle adjacent to a camera of the third camera type 506. Additionally, at least two cameras of the third camera type 506 may each be attached to the vehicle adjacent to a camera of the second camera type 504.

[0208] FIG10B is a simplified illustration of the camera system 1000 of FIG10A according to an exemplary embodiment, showing the cameras of the first camera type 502 of the camera system 1000. As depicted in FIG10B, the two cameras of the first camera type can each be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0209] FIG10C is a simplified illustration of the camera system 1000 of FIG10A according to an exemplary embodiment, showing a camera of the second camera type 502 of the camera system 1000. As depicted in FIG10C, one camera of the second camera type 504 can be attached to the front bumper of the vehicle 100. In addition, one camera of the second camera type 504 can be attached to the rear bumper of the vehicle 100. Furthermore, the two cameras of the second camera type 504 can each be attached to the side bumper of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). This configuration facilitates the detection of objects within a second distance range that may intersect the direction of travel of the vehicle 100, because the fields of view of the three cameras of the second camera type 504 may overlap in one or more areas in front of the vehicle 100, and each camera of the second camera type 504 may be able to independently detect objects within the second distance range from the vehicle 100. Other attachment locations (e.g., in tire wells or near side mirrors) are also possible.

[0210] As further depicted in FIG10C, the camera system 1000 may include a second predetermined camera 504B of a plurality of cameras of the second camera type 504 attached to the roof of the vehicle 100, having an associated second predetermined camera field of view 505B. The second predetermined camera 504B of the plurality of cameras of the second camera type 504 may be a camera of the second camera type 504 that includes a neutral density filter through which images are captured. Furthermore, the second predetermined camera 504B of the plurality of cameras of the second camera type 504 can be oriented relative to the vehicle 100 so as to be oriented from a pitch angle relative to the vehicle 100 (e.g., around).The image is captured by rotation along the z-axis, with the pitch angle greater than 20° in the forward direction relative to the driving direction of the vehicle 100. Such an orientation facilitates the detection of objects, such as traffic lights, as described in more detail below. Other attachment locations (e.g., above the tire wells or near the side mirrors) are also possible.

[0211] FIG10D is a simplified illustration of the camera system 1000 of FIG10A according to an exemplary embodiment, showing the camera of the third camera type 506 of the camera system specification 27 / 34 pages 33 CN 122253769 A 1000. As depicted in FIG10D, the six cameras of the third camera type 506 can be attached to the roof of the vehicle 100. In addition, as shown in FIG10D, two cameras of the third camera type 506 can each be attached to one or more side bumpers of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). Other attachment locations (e.g., inside the tire wells or near the side mirrors) are also possible.

[0212] FIG10E is a simplified illustration of the camera system 1000 of FIG10A according to an exemplary embodiment, showing a camera of a fourth camera type 508 of the camera system 1000. As depicted in FIG10E, the two cameras of the fourth camera type 508 may be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0213] FIG11A is an illustration of a camera system 1100 according to an exemplary embodiment. As depicted in FIG11A, the camera system 1100 may include two cameras of a first camera type 502, each camera having an associated first field of view 503. The two cameras of the first camera type 502 may each be oriented in a forward direction relative to the driving direction of the vehicle 100. Although the driving direction shown in FIG11A is the negative x-direction, other driving directions are also possible. For example, the driving direction may alternatively be the positive x-direction (e.g., the vehicle 100 may be traveling in the opposite direction). Furthermore, other numbers of cameras of the first camera type 502 (e.g., one, three, four, or five) and / or other attachment locations of cameras of the first camera type 502 (e.g., on the underside of vehicle 100 or inside vehicle 100) are also possible.

[0214] Furthermore, four cameras of the second camera type 504 may be present, each camera having an associated second field of view 505. One camera of the second camera type 504 may be oriented in a forward direction relative to the direction of travel of vehicle 100. Furthermore, one camera of the second camera type 504 may be oriented in a rearward direction relative to the direction of travel of vehicle 100. Additionally, two cameras of the second camera type 504 may each be oriented in one or more lateral directions relative to the direction of travel of vehicle 100. Furthermore, other numbers of cameras of the second camera type 504 (e.g., one, two, three, or five) and / orOther attachment locations for the cameras of the second camera type 504 (e.g., on the underside of vehicle 100 or inside vehicle 100) are also possible.

[0215] As depicted in FIG11A, the camera system 1100 may include five cameras of the third camera type 506, each having an associated third field of view 507. One camera of the third camera type 506 may be oriented in a forward direction relative to the direction of travel of vehicle 100. In addition, four cameras of the third camera type 506 may be oriented in one or more lateral directions relative to the direction of travel of vehicle 100. Other numbers of cameras of the third camera type 506 (e.g., one, two, three, four or six) and / or other attachment locations for the cameras of the third camera type 506 (e.g., on the underside of vehicle 100 or inside vehicle 100) are also possible.

[0216] Furthermore, as depicted in FIG11A, four cameras of the fourth camera type 508 (each having an associated fourth field of view 509) may be attached to vehicle 100. The two cameras of the fourth camera type 508 may be oriented at least partially in the forward direction relative to the travel direction of the vehicle 100. Furthermore, the two cameras of the fourth camera type 508 may be oriented in the rearward direction relative to the travel direction of the vehicle 100. This configuration facilitates the detection of objects that may enter the forward or rearward movement path of the vehicle 100 within a fourth distance range. Other numbers of cameras of the fourth camera type 508 (e.g., one, two, three, or five) and / or other attachment locations of the cameras of the fourth camera type 508 (e.g., under the vehicle 100 or inside the vehicle 100) are also possible.

[0217] As depicted in FIG11A, the two cameras of the second camera type 504 may each be attached to the vehicle 100 adjacent to one or more cameras of the third camera type 506. Furthermore, the two cameras of the first camera type 502 may be attached to the vehicle 100 adjacent to one or more cameras of the fourth camera type 508.

[0218] FIG11B is a simplified illustration of the camera system 1100 of FIG11A according to an exemplary embodiment, showing a camera of the first camera type 502 of the camera system specification 28 / 34 pages 34 CN 122253769 A 1100. As depicted in FIG11B, both cameras of the first camera type 502 may be attached to the roof of the vehicle 100. Other attachment locations (e.g., above or inside the tire wells or near the side mirrors) are also possible.

[0219] FIG11C is a simplified illustration of the camera system 1100 of FIG11A according to an exemplary embodiment, showing a camera of the second camera type 504 of the camera system 1100. As depicted in FIG11C, one camera of the second camera type 504 may be attached.The camera of the second camera type 504 can be attached to the front bumper of the vehicle 100. Additionally, one camera of the second camera type 504 can be attached to the rear bumper of the vehicle 100. Furthermore, both cameras of the second camera type 504 can each be attached to the side bumper of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). Other attachment locations (e.g., in the tire wells or near the side mirrors) are also possible.

[0220] FIG11D is a simplified illustration of the camera system 1100 of FIG11A according to an exemplary embodiment, showing a camera of the third camera type 506 of the camera system 1100. As depicted in FIG11D, both cameras of the third camera type 506 can be attached to the roof of the vehicle 100. Furthermore, both cameras of the third camera type 506 can each be attached to the side bumper of the vehicle 100 (e.g., above the front wheel wells of the vehicle 100). Other attachment locations (e.g., in the tire wells or near the side mirrors) are also possible.

[0221] As further depicted in FIG11D, the camera system 1100 may include a predetermined camera 506A of a plurality of cameras of third camera type 506 attached to the roof of the vehicle 100, having an associated third predetermined camera field of view 507A. The predetermined camera 506A of the plurality of cameras of third camera type 506 may be a camera of third camera type 506 including a neutral density filter through which images are captured. Furthermore, the predetermined camera 506A of the plurality of cameras of third camera type 506 may be oriented relative to the vehicle 100 in order to capture images from a pitch angle (e.g., rotation about the z-axis) relative to the vehicle 100, which pitch angle is greater than 20° in the forward direction relative to the driving direction of the vehicle 100. Such orientation can facilitate the detection of objects, such as traffic lights, as described in more detail below. Other attachment locations (e.g., above a tire well or near a side mirror) are also possible.

[0222] FIG11E is a simplified illustration of the camera system 1100 of FIG11A according to an exemplary embodiment, showing a camera of a fourth camera type 508 of the camera system 1100. As depicted in FIG11E, the four cameras of the fourth camera type 508 may be attached to the roof of the vehicle 100. Other attachment locations (e.g., above the tire wells or near the side mirrors) are also possible.

[0223] As previously mentioned, information from multiple sensors and / or cameras can be collected and combined. FIG12 is an illustration of object recognition technology 1200 according to an exemplary embodiment. In some embodiments, a lidar device may be attached to the vehicle. For example, a lidar device may be attached to the vehicle 100. The lidar device may generate a point cloud 1202 indicating the distance to objects in the environment surrounding the vehicle 100. At least some of these objects may also be included in image frames 1204 captured by a camera (such as a camera of a second camera type 504) that may be attached to the vehicle 100. The lidar device and the camera mayTo occupy different positions on the vehicle 100. Therefore, the field of view of the camera and the field of view of the lidar device may not be perfectly aligned. However, portions of the field of view of the camera and the field of view of the lidar device can overlap each other. In this way, the computing device can align the image frame 1204 with the point cloud 1202 to produce aligned image frames. Doing so can enhance the detection and / or recognition of objects 1206 within the overlapping portion 1208 of the image frame 1204 and the point cloud 1202, since the lidar device and the camera can each include information having complementary attributes about the object. In various embodiments, the information about the identified object can be used by multiple systems of the vehicle 100. For example, the information about the identified object can be used by the obstacle avoidance system 144 to assess whether the identified object is a potential obstacle.

[0224] FIG13 is an illustration of a camera of a predetermined camera 506A among a plurality of cameras of a third camera type 506 according to an example embodiment. As shown in Figure 13, the camera of the predetermined camera 506A among the plurality of cameras of the third camera type 506 can be oriented relative to the vehicle 100 to capture images from a pitch angle greater than 20° relative to the vehicle 100 (e.g., rotation about the z-axis). This facilitates the detection of objects located above the vehicle 100 in the surrounding environment, such as traffic lights 1302. The orientation of the predetermined camera 506A among the plurality of cameras of the third camera type 506 relative to the vehicle 100 can be adjusted to detect objects closer to or farther from the road surface. For example, to facilitate the detection of objects closer to the road surface, the orientation of the predetermined camera 506A among the plurality of cameras of the third camera type 506 can be adjusted relative to the vehicle 100 to capture images from a pitch angle greater than 10° or 5° relative to the vehicle 100. As another example, to facilitate the detection of objects further away from the road surface, the orientation of a predetermined camera 506A among the plurality of cameras of the third camera type 506 relative to the vehicle 100 can be adjusted to capture an image from a pitch angle greater than 25° or 20° relative to the vehicle 100. This facilitates the detection of objects closer to or further away from the road surface. The value of the pitch angle relative to the vehicle 100 is provided only as an example. Other pitch angles are also possible.

[0225] In some example embodiments, a predetermined camera among the plurality of cameras of the second camera type 504 may include a neutral density filter through which an image is captured. In such an embodiment, the predetermined camera among the plurality of cameras of the second camera type 504 may be oriented relative to the vehicle 100 to capture an image from a pitch angle greater than 20° relative to the vehicle. Furthermore, the predetermined camera among the plurality of cameras of the second camera type 504 may be configured to capture a pair of image frames, wherein the first image frame in the pair of image frames is captured at a first exposure time, wherein the first exposure time is set by an automatic exposure.The second image frame in the pair of image frames is captured at a second exposure time, and the second exposure time is longer than 1 / 60 of a second. The second exposure time can be longer than 1 / 50 of a second. The second exposure time can be shorter than 1 / 60 of a second. Using such a camera in a camera system can reduce the maintenance associated with such a camera system by reducing the total number of cameras in the camera system (e.g., because a single camera can present multiple imaging modalities).

[0226] In some example embodiments, a lidar device can be attached to vehicle 100 and configured to generate a point cloud indicating the distance to objects in the environment surrounding vehicle 100. In such an embodiment, identifying objects located within a second distance range from vehicle 100 based on one or more second images captured by a plurality of cameras of second camera type 504 may include aligning a first image frame with a point cloud, overlapping the aligned first image frame and the second image frame to generate an overlapping image, and performing object recognition using the overlapping image and the point cloud.

[0227] In some example embodiments, the plurality of cameras of the second camera type 504 may include a camera oriented in a forward direction relative to the travel direction of the vehicle 100, a camera oriented in a rearward direction relative to the travel direction of the vehicle 100, and two cameras oriented in one or more lateral directions relative to the travel direction of the vehicle 100.

[0228] In some example embodiments, at least one camera of the first camera type 502 may include two cameras, each oriented in a forward direction relative to the travel direction of the vehicle 100.

[0229] In some example embodiments, at least one camera of the first camera type 502 may include three cameras, each oriented in a forward direction relative to the travel direction of the vehicle 100. The plurality of cameras of the second camera type 504 may include: a camera oriented in a forward direction relative to the travel direction of the vehicle 100, a camera oriented in a rearward direction relative to the travel direction of the vehicle 100; and two cameras, each oriented in one or more lateral directions relative to the travel direction of the vehicle 100. Furthermore, the plurality of cameras of the third camera type 506 may include: a camera oriented in a forward direction relative to the travel direction of the vehicle 100, two cameras oriented at least partially in a rearward direction relative to the travel direction of the vehicle 100, and two cameras oriented in one or more lateral directions relative to the travel direction of the vehicle 100.

[0230] In some example embodiments, at least one of the at least one camera of the first camera type 502 may be attached to the vehicle 100, adjacent to at least one of the plurality of cameras of the third camera type 506.

[0231] In some example embodiments, at least one camera of the first camera type 502 may include four cameras, each camera being at least partially oriented in a forward direction relative to the travel direction of the vehicle 100. In such an embodiment, the second camera...The plurality of cameras of camera type 504 may include a camera oriented in a forward direction relative to the travel direction of vehicle 100, two cameras oriented in a rearward direction relative to the travel direction of vehicle 100, and two cameras oriented in one or more lateral directions relative to the travel direction of vehicle 100. Furthermore, in such embodiments, the plurality of cameras of third camera type 506 may include a camera oriented in a forward direction relative to the travel direction of vehicle 100, two cameras at least partially oriented in a rearward direction relative to the travel direction of vehicle 100, and two cameras at least partially oriented in one or more lateral directions relative to the travel direction of vehicle 100.

[0232] In some example embodiments, at least three cameras of at least one camera of first camera type 502 may be respectively attached to vehicle 100, adjacent to at least three cameras of the plurality of cameras of third camera type 506. In such embodiments, at least two of the multiple cameras of the second camera type 504 may be attached to the vehicle 100, adjacent to at least two of the multiple cameras of the third camera type 506.

[0233] In some example embodiments, at least one camera of the first camera type 502 may include three cameras, each at least partially oriented in a forward direction relative to the travel direction of the vehicle 100. In such embodiments, the multiple cameras of the second camera type 504 may include a camera oriented in a forward direction relative to the travel direction of the vehicle 100, a camera oriented in a rearward direction relative to the travel direction of the vehicle 100, and two cameras oriented in one or more lateral directions relative to the travel direction of the vehicle 100. Furthermore, in such embodiments, the multiple cameras of the third camera type 506 may include a camera oriented in a forward direction relative to the travel direction of the vehicle 100, two cameras at least partially oriented in a rearward direction relative to the travel direction of the vehicle 100, and three cameras at least partially oriented in one or more lateral directions relative to the travel direction of the vehicle 100.

[0234] In some example embodiments, at least one camera of at least one camera of the first camera type 502 may be attached to the vehicle 100, adjacent to at least one camera of the plurality of cameras of the third camera type 506, and at least two cameras of the plurality of cameras of the second camera type 504 may each be attached to the vehicle 100, adjacent to at least two cameras of the plurality of cameras of the third camera type 506.

[0235] In some example embodiments, at least one camera of the first camera type 502 may include two cameras, each oriented in a forward direction relative to the driving direction of the vehicle 100. In such an embodiment, the second camera type 504...Multiple cameras may include a camera oriented in a forward direction relative to the travel direction of the vehicle 100, a camera oriented in a rearward direction relative to the travel direction of the vehicle 100, and two cameras oriented in one or more lateral directions relative to the travel direction of the vehicle 100. Furthermore, in such embodiments, a plurality of cameras of a third camera type 506 may include two cameras, each at least partially oriented in a forward direction relative to the travel direction of the vehicle 100, two cameras, each at least partially oriented in a rearward direction relative to the travel direction of the vehicle 100, and four cameras, each oriented in one or more lateral directions relative to the travel direction of the vehicle 100.

[0236] In some example embodiments, the camera system may include a plurality of cameras of a fourth camera type 508. The plurality of cameras of the fourth camera type 508 may include two cameras attached to the vehicle 100. Each camera of the fourth camera type 508 may have a fourth field of view 509, and the fourth field of view 509 may span a larger yaw angle relative to the vehicle 100 than a first field of view 503. Furthermore, the fourth field of view 509 can span a smaller yaw angle relative to the vehicle 100 than the third field of view 507. In such an embodiment, the computing device can be configured to identify objects within a fourth distance range based on one or more images captured by a plurality of cameras of the fourth camera type 508. The fourth distance range may include distances from the vehicle that are farther than those included in the third distance range, and the first distance range may include distances from the vehicle that are farther than those included in the fourth distance range.

[0237] In some example embodiments, at least one of the plurality of cameras of the second camera type 504 may include a neutral density filter through which images are captured. Such a camera may be oriented relative to the vehicle 100, 31 / 34 pages 37 CN 122253769 A, to capture images from a pitch angle greater than 5° relative to the vehicle. Furthermore, such a camera may have a field of view that spans a larger angle in terms of yaw relative to the vehicle 100 compared to at least one other camera of the second camera type 504.

[0238] In some example embodiments, at least two cameras of at least one camera of the first camera type 502 may be attached to the vehicle 100, adjacent to at least two cameras of the plurality of cameras of the fourth camera type 508. In such an embodiment, at least two cameras of at least one camera of the first camera type 502 may be attached to the vehicle 100, adjacent to at least two cameras of the plurality of cameras of the third camera type 506. Furthermore, in such an embodiment, at least two cameras of the plurality of cameras of the fourth camera type 508 may be attached to the vehicle 100, adjacent to at least two cameras of the third camera type 506.At least two cameras of type 506 are adjacent to each other. Furthermore, in such embodiments, at least two cameras of type 506 may each be attached to vehicle 100, adjacent to at least two cameras of type 502.

[0239] In some example embodiments, at least one camera of type 502 may include two cameras, each oriented in a forward direction relative to the direction of travel of vehicle 100. In such embodiments, the multiple cameras of type 504 may include a camera oriented in a forward direction relative to the direction of travel of vehicle 100, a camera oriented in a backward direction relative to the direction of travel of vehicle 100, and two cameras oriented in one or more lateral directions relative to the direction of travel of vehicle 100. In such embodiments, the multiple cameras of type 506 may include one camera oriented in a forward direction relative to the direction of travel of vehicle 100 and four cameras oriented in one or more lateral directions relative to the direction of travel of vehicle 100.

[0240] In some example embodiments, at least one of the cameras of the third camera type 506 may include a neutral density filter through which images are captured and oriented relative to the vehicle 100 to capture images from a pitch angle greater than 20° relative to the vehicle.

[0241] In some example embodiments, the camera system may also include a plurality of cameras of a fourth camera type 508. In such an embodiment, the plurality of cameras of the fourth camera type 508 may include four cameras attached to the vehicle 100. Each camera of the fourth camera type 508 may have a fourth field of view 509, and the fourth field of view 509 may span a larger angle in yaw relative to the vehicle 100 than the first field of view 503. Furthermore, the fourth field of view 509 may span a smaller angle in yaw relative to the vehicle than the third field of view 507. Additionally, the computing device may be configured to identify objects located within a fourth distance range based on one or more images captured by the plurality of cameras of the fourth camera type 508. Such a fourth distance range may include distances farther from the vehicle than those included in the third distance range, and the first distance range may include distances farther from the vehicle than those included in the fourth distance range.

[0242] A single device may have features of at least one camera of a first camera type 502, at least one camera of a second camera type 504, at least one camera of a third camera type 506, at least one camera of a fourth camera type 508, and / or a lidar device. For example, a single device may capture images for identifying objects within a second distance range from the vehicle 100 and generate a point cloud 1202 indicating the distance to objects in the environment surrounding the vehicle 100. As another example, a single deviceThe device can capture images for identifying objects within both a first distance range and a second distance range from the vehicle 100. Other types of sensors may be used in addition to or in place of a camera or lidar device within a single device. For example, a radar sensor may be incorporated into a single device.

[0243] FIG14 is a flowchart of method 1400 according to an example embodiment. Method 1400 can be performed using images captured by camera systems 600, 700, 800, 900, 1000, or 1100. In alternative embodiments, alternative camera systems may be used to perform method 1400.

[0244] At block 1402, method 1400 may include receiving by a computing device one or more first images captured by at least one camera of a first camera type (see page 32 / 34 of specification 38 CN 122253769 A), wherein each camera of the first camera type is attached to the vehicle and has a first field of view.

[0245] At block 1404, method 1400 may include receiving, by a computing device, one or more second images captured by a plurality of cameras of a second camera type, wherein each camera of the second camera type is attached to the vehicle and has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle.

[0246] At block 1406, method 1400 may include receiving, by a computing device, one or more third images captured by a plurality of cameras of a third camera type, wherein each camera of the third camera type is attached to the vehicle and has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle, wherein the first field of view spans fewer angles in yaw relative to the vehicle than the third field of view, and wherein the third field of view spans fewer angles in yaw relative to the vehicle than the second field of view.

[0247] At block 1408, method 1400 may include identifying, by a computing device, an object located within a first distance range from the vehicle based on one or more first images.

[0248] At block 1410, method 1400 may include the computing device identifying objects located within a second distance range from the vehicle based on one or more second images.

[0249] At block 1412, method 1400 may include the computing device identifying objects located within a third distance range from the vehicle based on one or more third images, wherein the third distance range includes distances farther from the vehicle than distances included in the second distance range, and wherein the first distance range includes distances farther from the vehicle than distances included in the third distance range.

[0250] At block 1414, method 1400 may include the computing device determining driving decisions for the vehicle based on objects located within a first distance range from the vehicle, objects located within a second distance range from the vehicle, or objects located within a third distance range from the vehicle.

[0251] In some embodiments, method 1400 may further include causing a vehicle (e.g., vehicle 100) to perform driving maneuvers based on driving decisions by a computing device.

[0252] This disclosure is not limited to the specific embodiments described herein, which are intended as illustrations of various aspects. Those skilled in the art will understand that many modifications and variations can be made without departing from its spirit and scope. In addition to those listed herein, those skilled in the art will understand from the foregoing description functionally equivalent methods and apparatus within the scope of this disclosure. These modifications and variations are intended to fall within the scope of the appended claims.

[0253] The foregoing detailed description, with reference to the accompanying drawings, describes various features and functions of the disclosed systems, devices, and methods. In the drawings, similar symbols generally identify similar components unless the context otherwise requires. The exemplary embodiments described herein and in the drawings are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter set forth herein. It will be readily understood that aspects of this disclosure, as generally described herein and illustrated in the drawings, can be arranged, replaced, combined, separated, and designed in a wide variety of different configurations, all of which are expressly contemplated herein.

[0254] With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the accompanying drawings and as discussed herein, each step, block, operation, and / or communication may represent the processing and / or transmission of information according to exemplary embodiments. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functionality involved. Furthermore, more or fewer blocks and / or operations may be used with any of the message flow diagrams, scenarios, and flowcharts discussed herein, and these message flow diagrams, scenarios, and flowcharts may be combined with each other in part or in whole.

[0255] Steps, blocks, or operations representing information processing may correspond to circuitry that can be configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, steps or blocks representing information processing may correspond to modules, segments, or portions of program code (including associated data). Program code may include one or more instructions executable by a processor for implementing specific logical operations or actions in a method or technique. Program code and / or associated data may be stored on any type of computer-readable medium, such as a storage device including RAM, a disk drive, a solid-state drive, or another storage medium.

[0256] Furthermore, steps, blocks, or operations representing one or more information transfers may correspond to transfers within the same physical device.Information transfer between software and / or hardware modules. However, other information transfers may occur between software and / or hardware modules in different physical devices.

[0257] The specific arrangement shown in the figures should not be considered limiting. It should be understood that other embodiments may include more or fewer of each element shown in the given figures. Furthermore, some of the elements shown may be combined or omitted. In addition, exemplary embodiments may include elements not shown in the figures.

[0258] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and not limiting, and the true scope is indicated by the appended claims. Instruction manual page 34 / 34, page 40, CN 122253769 A, Figure 1; Instruction manual figure 1 / 45, page 41, CN 122253769 A, Figure 2A, Figure 2B; Instruction manual figure 2 / 45, page 42, CN 122253769 A, Figure 2C, Figure 2D; Instruction manual figure 3 / 45, page 43, CN 122253769 A, Figure 2E; Instruction manual figure 4 / 45, page 44, CN 122253769 A, Figure 2F; Instruction manual figure 5 / 45, page 45, CN 122253769 A, Figure 2G; Instruction manual figure 6 / 45, page 46, CN 122253769 A, Figure 2H; Instruction manual figure 7 / 45, page 47, CN 122253769 A, Figure 2I; Instruction manual figure 8 / 45, page 48, CN 122253769 A, Figure 2J; Instruction manual figure 9 / 45, page 49, CN 122253769 A Figure 2K Instruction Manual Drawing, Page 10 / 45, 50 CN 122253769 A Figure 3 Instruction Manual Drawing, Page 11 / 45, 51 CN 122253769 A Figure 4A Instruction Manual Drawing, Page 12 / 45, 52 CN 122253769 A Figure 4B Instruction Manual Drawing, Page 13 / 45, 53 CN 122253769 A Figure 5A Figure 5B Instruction Manual Drawing, Page 14 / 45, 54 CN 122253769 A Figure 5C Figure 5D Instruction Manual Drawing, Page 15 / 45, 55 CN 122253769 A Figure 6A Instruction Manual Drawing, Page 16 / 45, 56 CN 122253769 A Figure 6B Instruction Manual Drawing, Page 17 / 45, 57 CN 122253769 A Figure 6C DescriptionIllustrations: Page 18 / 45, Figure 58, CN 122253769 A, Figure 6D; Page 19 / 45, Figure 59, CN 122253769 A, Figure 7A; Page 20 / 45, Figure 60, CN 122253769 A, Figure 7B; Page 21 / 45, Figure 61, CN 122253769 A, Figure 7C; Page 22 / 45, Figure 62, CN 122253769 A, Figure 7D; Page 23 / 45, Figure 63, CN 122253769 A, Figure 8A; Page 24 / 45, Figure 64, CN 122253769 A, Figure 8B; Page 25 / 45, Figure 65, CN 122253769 A, Figure 8C; Page 26 / 45, Figure 66, CN 122253769 A, Figure 8D; Page 27 / 45, Figure 67. CN 122253769 A Figure 9A, Instruction Manual Illustration, Page 28 / 45, 68 CN 122253769 A Figure 9B, Instruction Manual Illustration, Page 29 / 45, 69 CN 122253769 A Figure 9C, Instruction Manual Illustration, Page 30 / 45, 70 CN 122253769 A Figure 9D, Instruction Manual Illustration, Page 31 / 45, 71 CN 122253769 A Figure 9E, Instruction Manual Illustration, Page 32 / 45, 72 CN 122253769 A Figure 10A, Instruction Manual Illustration, Page 33 / 45, 73 CN 122253769 A Figure 10B, Instruction Manual Illustration, Page 34 / 45, 74 CN 122253769 A Figure 10C, Instruction Manual Illustration, Page 35 / 45, 75 CN 122253769 A Figure 10D, Instruction Manual Illustration, Page 36 / 45, 76 CN 122253769 A Figure 10E Instruction Manual Appendix 37 / 45 Page 77 CN 122253769 A Figure 11A Instruction Manual Appendix 38 / 45 Page 78 CN 122253769 A Figure 11B Instruction Manual Appendix 39 / 45 Page 79 CN 122253769 A Figure 11C Instruction Manual Appendix 40 / 45 Page 80 CN 122253769 A Figure 11D Instruction Manual Appendix 41 / 45 Page 81 CN 122253769 A Figure 11E Instruction Manual AppendixFigure 42 / 45, page 82, CN 122253769 A; Figure 12, Appendix to the Specification; Figure 43 / 45, page 83, CN 122253769 A; Figure 13, Appendix to the Specification; Figure 44 / 45, page 84, CN 122253769 A; Figure 14, Appendix to the Specification; Figure 45 / 45, page 85, CN 122253769 A; Abstract

Claims

1. A system comprising: Vehicle; At least one camera of a first camera type is attached to the vehicle, wherein each camera of the first camera type has a first field of view; Multiple cameras of the second camera type are attached to the vehicle, wherein each camera of the second camera type has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle. A plurality of cameras of a third camera type are attached to the vehicle, wherein each camera of the third camera type has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° relative to the vehicle in terms of yaw, wherein the first field of view spans a smaller angle relative to the vehicle in terms of yaw than the third field of view, and wherein the third field of view spans a smaller angle relative to the vehicle in terms of yaw than the second field of view; and A computing device communicatively coupled to at least one camera of a first camera type, a plurality of cameras of a second camera type, and a plurality of cameras of a third camera type, wherein the computing device is configured to: Based on one or more first images captured by at least one camera of the first camera type, objects located within a first distance range from the vehicle are identified; Based on one or more second images captured by the plurality of cameras of the second camera type, objects located within a second distance range from the vehicle are identified; and Based on one or more third images captured by the plurality of cameras of the third camera type, objects located within a third distance range from the vehicle are identified, wherein the third distance range includes distances from the vehicle that are farther than distances included in a second distance range, and wherein the first distance range includes distances from the vehicle that are farther than distances included in the third distance range.

2. The system according to claim 1, wherein, The predetermined camera among the plurality of cameras of the second camera type: Includes a neutral density filter, through which the image is captured; The vehicle is oriented relative to the vehicle in order to capture images from a pitch angle greater than 20° relative to the vehicle. and A pair of image frames is configured to capture an image frame, wherein the first image frame in the pair is captured at a first exposure time, wherein the first exposure time is set by an automatic exposure setting, wherein the second image frame in the pair is captured at a second exposure time, and wherein the second exposure time is longer than 1 / 60 of a second.

3. The system of claim 2 further includes a light detection and ranging (LiDAR) device, said light detection and ranging (LiDAR) device being attached to the vehicle and configured to generate a point cloud indicating the distance to objects in the environment surrounding the vehicle. in, Identifying the object located within a second distance range from the vehicle based on one or more second images captured by the plurality of cameras of the second camera type includes: Align the first image frame with the point cloud; Overlap the aligned first and second image frames to generate an overlapping image; and Object recognition is performed using the overlapping image and the point cloud.

4. The system according to claim 1, wherein, The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the direction of travel of the vehicle; A camera oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the direction of travel of the vehicle.

5. The system according to claim 1, wherein, The first camera type comprises two cameras, each oriented in a forward direction relative to the direction of travel of the vehicle.

6. The system according to claim 1, in, The first camera type comprises at least three cameras, each oriented in a forward direction relative to the direction of travel of the vehicle. The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; A camera oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel, and The plurality of cameras of the third camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; Two cameras, each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel.

7. The system according to claim 6, wherein, At least one of the at least one cameras of the first camera type is attached adjacent to at least one of the plurality of cameras of the third camera type to the vehicle.

8. The system according to claim 1, in, The first camera type comprises four cameras, each at least partially oriented in a forward direction relative to the direction of travel of the vehicle. The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; Two cameras, each oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel, and The plurality of cameras of the third camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; Two cameras, each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and Two cameras, each at least partially oriented in one or more lateral directions relative to the vehicle's direction of travel.

9. The system according to claim 8, in, At least three cameras of the first camera type are each attached adjacent to at least three cameras of the third camera type to the vehicle, and In this embodiment, at least two of the plurality of cameras of the second camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the third camera type.

10. The system according to claim 1, in, The first camera type includes three cameras, each at least partially oriented in a forward direction relative to the direction of travel of the vehicle; The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; A camera oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the direction of travel of the vehicle; and The plurality of cameras of the third camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; Two cameras, each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and Three cameras, each at least partially oriented in one or more lateral directions relative to the vehicle's direction of travel.

11. The system according to claim 10, in, At least one camera of the first camera type is attached adjacent to at least one camera of the third camera type to the vehicle, and In this embodiment, at least two of the plurality of cameras of the second camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the third camera type.

12. The system according to claim 1, in, The first camera type comprises at least two cameras, each oriented in a forward direction relative to the direction of travel of the vehicle. The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; A camera oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel, and The plurality of cameras of the third camera type include: Two cameras, each at least partially oriented in the forward direction relative to the vehicle's direction of travel; Two cameras, each at least partially oriented in the rearward direction relative to the vehicle's direction of travel; and Four cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel.

13. The system of claim 12, further comprising a plurality of cameras of a fourth camera type, in, The plurality of cameras of the fourth camera type includes two cameras attached to the vehicle. In this type of camera, each camera has a fourth field of view. The fourth field of view spans a larger angle in terms of yaw relative to the vehicle than the first field of view. The fourth field of view spans fewer angles in yaw relative to the vehicle than the third field of view. The computing device is further configured to identify objects located within a fourth distance range based on one or more images captured by the plurality of cameras of the fourth camera type. The fourth distance range includes distances farther from the vehicle than those included in the third distance range, and The first distance range includes distances farther from the vehicle than those included in the fourth distance range.

14. The system according to claim 12, wherein, At least one of the plurality of cameras of the second camera type: Includes a neutral density filter, through which the image is captured. Oriented relative to the vehicle in order to capture images from a pitch angle greater than 5° relative to the vehicle, and It has a field of view that spans a larger angle in terms of yaw relative to the vehicle than at least one other camera of the second camera type.

15. The system according to claim 13, in, At least two cameras of the first camera type are each attached adjacent to at least two cameras of the fourth camera type to the vehicle. In this embodiment, at least two cameras of the first camera type are each attached adjacent to at least two cameras of the third camera type of the plurality of cameras to the vehicle. In this embodiment, at least two cameras of the fourth camera type are each attached to the vehicle adjacent to at least two cameras of the third camera type, and In this embodiment, at least two of the plurality of cameras of the third camera type are each attached to the vehicle adjacent to at least two of the plurality of cameras of the second camera type.

16. The system according to claim 1, in, The first camera type comprises at least two cameras, each oriented in a forward direction relative to the direction of travel of the vehicle. The plurality of cameras of the second camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; A camera oriented in a rearward direction relative to the vehicle's direction of travel; and Two cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel, and The plurality of cameras of the third camera type include: A camera oriented in the forward direction relative to the vehicle's direction of travel; and Four cameras, each oriented in one or more lateral directions relative to the vehicle's direction of travel.

17. The system according to claim 16, wherein, At least one of the plurality of cameras of the third camera type: Includes a neutral density filter, through which the image is captured; and The vehicle is oriented relative to the vehicle in order to capture images from a pitch angle greater than 20° relative to the vehicle.

18. The system of claim 16, further comprising a plurality of cameras of a fourth camera type, in, The plurality of cameras of the fourth camera type includes four cameras attached to the vehicle. In this type of camera, each camera has a fourth field of view. The fourth field of view spans a larger angle in terms of yaw relative to the vehicle than the first field of view. The fourth field of view spans fewer angles in yaw relative to the vehicle than the third field of view. The computing device is further configured to identify objects located within a fourth distance range based on one or more images captured by the plurality of cameras of the fourth camera type. The fourth distance range includes distances farther from the vehicle than those included in the third distance range, and The first distance range includes distances farther from the vehicle than those included in the fourth distance range.

19. A method comprising: A computing device receives one or more first images captured by at least one camera of a first camera type, wherein each camera of the first camera type is attached to the vehicle and has a first field of view; The computing device receives one or more second images captured by a plurality of cameras of a second camera type, wherein each camera of the second camera type is attached to the vehicle and has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle; The computing device receives one or more third images captured by a plurality of cameras of a third camera type, wherein each camera of the third camera type is attached to the vehicle and has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle, wherein the first field of view spans less angle in yaw relative to the vehicle than the third field of view, and wherein the third field of view spans less angle in yaw relative to the vehicle than the second field of view; The computing device identifies objects located within a first distance range from the vehicle based on the one or more first images; The computing device identifies objects located within a second distance range from the vehicle based on the one or more second images; The computing device identifies objects located within a third distance range from the vehicle based on the one or more third images, wherein the third distance range includes distances farther from the vehicle than distances included in a second distance range, and wherein a first distance range includes distances farther from the vehicle than distances included in the third distance range; and The computing device determines driving decisions for the vehicle based on the object located within a first distance range from the vehicle, the object located within a second distance range from the vehicle, or the object located within a third distance range from the vehicle.

20. A non-transitory computer-readable medium having stored thereon program instructions that, when executed by a processor, cause the processor to perform operations, the operations including: Receive one or more first images captured by at least one camera of a first camera type, wherein each camera of the first camera type is attached to the vehicle and has a first field of view; Receive one or more second images captured by a plurality of cameras of a second camera type, wherein each camera of the second camera type is attached to the vehicle and has a second field of view, and wherein the second field of view spans at least 170° in yaw relative to the vehicle; Receive one or more third images captured by a plurality of cameras of a third camera type, wherein each camera of the third camera type is attached to the vehicle and has a third field of view, wherein the combined field of view of the plurality of cameras of the third camera type spans 360° in yaw relative to the vehicle, wherein the first field of view spans less angle in yaw relative to the vehicle than the third field of view, and wherein the third field of view spans less angle in yaw relative to the vehicle than the second field of view; Based on the one or more first images, identify objects located within a first distance range from the vehicle; Based on the one or more second images, identify objects located within a second distance range from the vehicle; Based on the one or more third images, objects located within a third distance range from the vehicle are identified, wherein the third distance range includes distances farther from the vehicle than distances included in a second distance range, and wherein a first distance range includes distances farther from the vehicle than distances included in the third distance range; and A driving decision for the vehicle is determined based on the object located within a first distance range from the vehicle, the object located within a second distance range from the vehicle, or the object located within a third distance range from the vehicle.