Scratch-resistant fisheye lens with built-in heater

Abstract

A wide-angle optical system having a protection and a heating layer is disclosed. The wide-angle optical system may include a fisheye lens, a protection layer disposed on an outer surface of the fisheye lens, and a heating layer disposed on an inner surface of the protection layer facing the fisheye lens. The heating layer may include a transparent conductive layer configured to provide heat to the protection layer in response to receiving an electric current from a power supply.

Description

Technology Field

[0001] This application claims priority to U.S. provisional patent application 63 / 626443, filed on January 29, 2024, the whole contents of which are incorporated herein by reference, and the title of the invention "SCRATCH RESISTANT FISHEYE LENS WITH INTEGRATED HEATER".

[0002] The present invention relates to a scratch-resistant fisheye lens with a built-in heater. Brief explanation of the drawing

[0003] FIG. 1 is an exemplary environment in which a vehicle including one or more components of an autonomous driving system can be implemented. Figure 2 is a diagram of one or more systems of a vehicle including an autonomous driving system. FIG. 3 is a diagram of components of one or more exemplary devices and / or one or more exemplary systems of FIG. 1 and FIG. 2. Figure 4a is a diagram of specific components of an exemplary autonomous driving system. Figure 4b is a diagram of specific components of an exemplary autonomous driving system. Figure 4c is a diagram of specific components of an exemplary autonomous driving system. Figure 4d is a diagram of specific components of an exemplary autonomous driving system. FIG. 5a schematically illustrates a wide-angle optical imaging system. FIG. 5b schematically illustrates a side cross-sectional view of a wide-angle optical imaging system comprising curved protective and heating layers disposed on an object lens of the wide-angle optical imaging system having a curved convex surface having substantially the same curvature as the curved protective and heating layers. FIGS. 6a and 6b schematically illustrate a side cross-sectional view (6a) and a front view (6b) of a wide-angle optical imaging system comprising layers disposed on an object lens of a wide-angle optical imaging system having a curved convex surface having a curvature substantially different from the curvature of a curved protective and heating layer. FIGS. 7a and 7b schematically illustrate a side cross-sectional view (7a) and a front view (7b) of a wide-angle optical imaging system including a protective and heating layer connected to an object lens by a spacer. Specific details for implementing the invention

[0004] In the following description, numerous specific details are presented for illustrative purposes to provide a complete understanding of the present disclosure. However, it will be apparent that the embodiments described by the present disclosure can be practiced without these specific details. In some instances, well-known structures and devices are illustrated in the form of block diagrams to avoid unnecessarily obscuring aspects of the present disclosure.

[0005] Specific arrangements or sequences of schematic elements, such as those representing systems, devices, modules, instruction blocks, data elements, etc., are illustrated in the drawings for convenience of explanation. However, a person skilled in the art will understand that a specific order or arrangement of schematic elements in the drawings does not imply that a specific processing order or sequence of processes, or a separation of processes, is required unless explicitly stated otherwise. Furthermore, the inclusion of schematic elements in the drawings does not imply that such elements are required in all embodiments, or that features represented by such elements may not be included in or combined with other elements in some embodiments.

[0006] Furthermore, where connection elements, such as solid or dashed lines or arrows, are used in the drawings to illustrate a connection, relationship, or association between two or more different schematic elements, the absence of any such connection elements does not imply that a connection, relationship, or association cannot exist. In other words, to avoid obscuring the present disclosure, some connections, relationships, or associations between elements are not illustrated in the drawings. Additionally, for ease of illustration, a single connection element may be used to represent multiple connections, relationships, or associations between elements. For example, if a connection element represents the communication of signals, data, or instructions (e.g., "software instructions"), it should be understood by a person skilled in the art that such an element may represent one or more signal paths (e.g., buses) that may be necessary to influence the communication.

[0007] Terms such as first, second, third, etc. are used to describe various components, but these elements should not be limited by these terms. Terms such as first, second, third, etc. are used solely to distinguish one element from another. For example, the first contact may be referred to as the second contact, and similarly, the second contact may be referred to as the first contact without departing from the scope of the described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0008] The technical terms used in the description of the various described embodiments herein are included solely to describe specific embodiments and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include plural forms as well and may be used interchangeably with “one or more” or “at least one” unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and encompasses any and all possible combinations of one or more of the associated enumerated items. It will be further understood that when the terms “includes,” “including,” “comprises,” and / or “comprising” are used in this description, they specify the presence of the mentioned features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0009] As used herein, the terms “communication” and “to communicate” refer to at least one of the reception, receipt, transmission, delivery, provision, etc. of information (or, for example, information expressed by data, signals, messages, instructions, commands, etc.). One unit (for example, a device, a system, a component of a device or system, a combination thereof, etc.) communicating with another unit means that one unit can receive information directly or indirectly from another unit and / or transmit information to the other unit (for example, transmit). This may essentially refer to a direct or indirect wired and / or wireless connection. Additionally, two units may communicate with each other even if the transmitted information may be modified, processed, relayed, and / or routed between the first unit and the second unit. For example, the first unit may communicate with the second unit even if the first unit passively receives information and does not actively transmit information to the second unit. As another example, if at least one intermediate unit (e.g., a third unit located between the first unit and the second unit) processes information received from the first unit and transmits the processed information to the second unit, the first unit may communicate with the second unit. In some embodiments, the message may refer to a network packet containing data (e.g., a data packet, etc.).

[0010] As used herein, the term “in case” is optionally interpreted, depending on the context, to mean “when” or “at” or “in response to a decision”, “in response to detecting”, etc. Similarly, the phrase “when determined” or “when [the mentioned condition or event] is detected” is optionally interpreted, depending on the context, to mean “as soon as it is determined”, “in response to the determination”, “as soon as [the mentioned condition or event] is detected”, “in response to the detection of [the mentioned condition or event]”, and / or, etc. Also, as used herein, the terms “have”, “have”, “have”, etc. are intended to be open-ended terms. Also, the phrase “based on” is intended to mean “at least partially based on” unless otherwise explicitly stated.

[0011] The embodiments exemplified in the accompanying drawings will now be described in detail. In the following detailed description, a number of specific details are provided to provide a complete understanding of the various embodiments described. However, it will be apparent to a person skilled in the art that the various embodiments described can be practiced without these specific details. In other cases, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure the aspects of the embodiments.

[0012] General Overview

[0013] In some aspects and / or embodiments, the systems, methods, and computer program products described herein include and / or implement improvements to autonomous vehicles or fleets of such vehicles, the design, manufacture, operation, and / or management of components or elements of autonomous vehicles, support systems for autonomous vehicles or fleets of such vehicles, or auxiliary systems related to autonomous vehicles or fleets of such vehicles.

[0014] By implementing the systems, methods, and computer program products described herein, technology for improving an autonomous vehicle or a fleet of such vehicles is realized (including the design, manufacture, operation, and / or management of the autonomous vehicle or a fleet of such vehicles, components or elements of the autonomous vehicle, support systems for the autonomous vehicle or a fleet of such vehicles, or auxiliary systems related to the autonomous vehicle or a fleet of such vehicles).

[0015] Further details regarding specific improvements considered herein are provided in the sections below. It should be understood that statements made within the sections below in the context of wide-angle imaging systems refer to embodiments disclosed in the context of wide-angle imaging systems and do not necessarily refer to all embodiments of the present disclosure. For example, statements in the context of wide-angle imaging systems indicating that certain features are required or essential should be understood to refer to the necessity or essentiality of such features for the embodiments in the sections regarding wide-angle imaging systems, insofar as such statements exist. Such statements do not indicate that such features are required or essential for all embodiments of the present disclosure. Similarly, definitions of terms in the sections below regarding wide-angle imaging systems should be understood to define terms in the context of wide-angle imaging systems and may not apply to the remainder of the present disclosure.

[0016] Improvements discussed in the context of a wide-angle imaging system may represent computer-implementable improvements implemented within the system described herein in relation to FIGS. 1 through 4. For example, improvements discussed within the section regarding the wide-angle imaging system may be implemented wholly or partially by the device (300) of FIG. 3 by the execution of computer code stored in the memory (306) of FIG. 3 on the processor (304) of FIG. 3. In one embodiment, the device (300) is located within a vehicle, such as the vehicle (200) of FIG. 2. In another embodiment, the device (300) is located outside the vehicle, such as the remote AV system (114), fleet management system (116), or vehicle-to-infrastructure system (118) of FIG. 1. Additionally or alternatively, improvements discussed within the sections regarding wide-angle imaging systems may represent improvement components for the elements of FIG. 1, including improvement components for the vehicle (102), remote AV system (114), fleet management system (116), or vehicle-to-infrastructure system (118) of FIG. 1. Thus, improvements discussed in the context of wide-angle imaging systems may be substantially incorporated into one or more of the elements mentioned above in FIG. 1 to improve these elements. As an example, improvements for LiDAR sensors may be used to modify the LiDAR sensor (202b) of FIG. 2. Details regarding an exemplary embodiment of the LiDAR sensor (202b) of FIG. 2 are provided in U.S. Patent Application No. 17 / 931,051 (Att'y docket no: MOTN.057A), filed on September 9, 2022, with the title of the invention “SYSTEMS AND METHODS FOR TIME-OF-FLIGHT (TOF) LIDAR SIGNAL-TO-NOISE IMPROVEMENT”, the entire contents of which are incorporated herein by reference.

[0017] Now, referring to FIG. 1, an exemplary environment (100) is illustrated in which vehicles including autonomous driving systems as well as vehicles not including autonomous driving systems are operated. As illustrated, the environment (100) includes vehicles (102a to 102n), objects (104a to 104n), routes (106a to 106n), a region (108), a V2I (vehicle-to-infrastructure) device (110), a network (112), a remote autonomous vehicle (AV) system (114), a fleet management system (116), and a V2I system (118). Vehicles (102a-102n), a V2I (vehicle-to-infrastructure) device (110), a network (112), an autonomous vehicle (AV) system (114), a fleet management system (116), and a V2I system (118) are interconnected via a wired connection, a wireless connection, or a combination of wired and wireless connections (e.g., establishing a connection for communication, etc.). In some embodiments, objects (104a-104n) are interconnected with at least one of the vehicles (102a-102n), the vehicle-to-infrastructure (V2I) device (110), the network (112), the autonomous vehicle (AV) system (114), the fleet management system (116), and the V2I system (118) via a wired connection, a wireless connection, or a combination of wired and wireless connections.

[0018] Vehicles (102a through 102n) (individually referred to as vehicles (102) and collectively referred to as vehicles (102)) comprise at least one device configured to transport goods and / or people. In some embodiments, vehicles (102) are configured to communicate with a V2I device (110), a remote AV system (114), a fleet management system (116), and / or a V2I system (118) via a network (112). In some embodiments, vehicles (102) include automobiles, buses, trucks, trains, and / or others. In some embodiments, vehicles (102) are identical or similar to the vehicles (200) described herein (see FIG. 2). In some embodiments, a vehicle (200) among a set of vehicles (200) is associated with an autonomous fleet manager. In some embodiments, the vehicles (102) travel along each of the routes (106a-106n) as described herein (individually referred to as route (106) and collectively referred to as routes (106)). In some embodiments, one or more vehicles (102) include an autonomous driving system (e.g., an autonomous driving system identical or similar to the autonomous driving system (202)).

[0019] Objects (104a to 104n) (individually referred to as objects (104) and collectively referred to as objects (104)) include, for example, at least one vehicle, at least one pedestrian, at least one cyclist, at least one structure (e.g., a building, a sign, a fire hydrant, etc.). Each object (104) is stationary (e.g., located in a fixed position for a period of time) or moving (e.g., having speed and associated with at least one trajectory). In some embodiments, the objects (104) are associated with corresponding locations within the area (108).

[0020] Routes (106a through 106n) (individually referred to as Route (106) and collectively referred to as Route (106)) are each associated with a sequence of actions (also referred to as a trajectory) connecting states in which the AV can operate (e.g., defines this). Each route (106) starts in an initial state (e.g., a state corresponding to a first spatiotemporal position, velocity, etc.) and a final target state (e.g., a state corresponding to a second spatiotemporal position different from the first spatiotemporal position) or a target area (e.g., a subspace of the permissible states (e.g., terminal states). In some embodiments, the first state includes a location where individuals or individuals are picked up by the AV, and the second state or area includes a location or locations where the individuals or individuals picked up by the AV are dropped off. In some embodiments, the routes (106) include a plurality of acceptable state sequences (e.g., a plurality of spatiotemporal position sequences), and the plurality of state sequences are associated with a plurality of trajectories (e.g., defines them). In an embodiment, the routes (106) include only higher-level actions or inaccurate state locations, such as a series of connected roads indicating a turning direction at a road intersection. Additionally or alternatively, the routes (106) may include more accurate actions or states, such as, for example, specific target lanes or accurate locations within lane regions and target speeds at these positions. In an example, the routes (106) include a plurality of accurate state sequences that follow at least one high-level action sequence having a limited lookahead horizon to reach intermediate goals, and cumulatively correspond to a plurality of trajectories that collectively form a high-level route such that a combination of consecutive repetitions of the limited horizon state sequences terminates in a final target state or region.

[0021] Area (108) includes a physical area (e.g., a geographical area) over which vehicles (102) can travel. In an embodiment, area (108) includes at least one state (e.g., a country, a region, an individual state among a plurality of states included in a country), at least one part of a state, at least one city, at least one part of a city, etc. In some embodiments, area (108) includes at least one named major road (referred to herein as “road”), such as a highway, an interstate highway, a park road, a general road, etc. Additionally, or alternatively, in some examples, area (108) includes at least one unnamed road, such as an access road, a section of a parking lot, a section of an open space and / or undeveloped land, a dirt road, etc. In some embodiments, the road includes at least one lane (e.g., a part of the road over which vehicles (102) can cross). In an embodiment, the road includes at least one lane associated with (e.g., identified based thereon) at least one lane.

[0022] A vehicle-to-infrastructure (V2I) device (110) (sometimes referred to as a vehicle-to-infrastructure or vehicle-to-everything (V2X) device) comprises at least one device configured to communicate with vehicles (102) and / or a V2I infrastructure system (118). In some embodiments, the V2I device (110) is configured to communicate with vehicles (102), a remote AV system (114), a fleet management system (116), and / or a V2I system (118) via a network (112). In some embodiments, the V2I device (110) includes a radio frequency identification (RFID) device, signage, a camera (e.g., a two-dimensional (2D) and / or three-dimensional (3D) camera), a lane marker, a street light, a parking meter, etc. In some embodiments, the V2I device (110) is configured to communicate directly with vehicles (102). Additionally or alternatively, in some embodiments, the V2I device (110) is configured to communicate with vehicles (102), a remote AV system (114), and / or a fleet management system (116) via a V2I system (118). In some embodiments, the V2I device (110) is configured to communicate with the V2I system (118) via a network (112).

[0023] The network (112) includes one or more wired and / or wireless networks. In an embodiment, the network (112) includes a cellular network (e.g., LTE (long term evolution) network, 3G (third generation) network, 4G (fourth generation) network, 5G (fifth generation) network, CDMA (code division multiple access) network, etc.), PLMN (public land mobile network), LAN (local area network), WAN (wide area network), MAN (metropolitan area network), telephone network (e.g., PSTN (public switched telephone network)), private network, ad-hoc network, intranet, internet, fiber-optic-based network, cloud computing network, etc., and a combination of some or all of these networks.

[0024] The remote AV system (114) includes at least one device configured to communicate with vehicles (102), a V2I device (110), the network (112), a fleet management system (116), and / or a V2I system (118) via a network (112). In an embodiment, the remote AV system (114) includes a server, a group of servers, and / or other similar devices. In some embodiments, the remote AV system (114) is co-located with the fleet management system (116). In some embodiments, the remote AV system (114) is involved in the installation of some or all of the vehicle components, including an autonomous driving system, an autonomous vehicle compute, and software implemented by the autonomous vehicle compute. In some embodiments, the remote AV system (114) maintains (e.g., updates and / or replacements) these components and / or software for the lifetime of the vehicle.

[0025] The fleet management system (116) includes at least one device configured to communicate with vehicles (102), a V2I device (110), a remote AV system (114), and / or a V2I infrastructure system (118). In an embodiment, the fleet management system (116) includes a server, a group of servers, and / or other similar devices. In some embodiments, the fleet management system (116) is associated with a ridesharing company (e.g., an organization controlling the operation of a number of vehicles (e.g., vehicles including an autonomous driving system and / or vehicles not including an autonomous driving system)).

[0026] In some embodiments, the V2I system (118) includes at least one device configured to communicate with vehicles (102), V2I devices (110), remote AV systems (114), and / or fleet management systems (116) via a network (112). In some embodiments, the V2I system (118) is configured to communicate with V2I devices (110) via a connection different from that of the network (112). In some embodiments, the V2I system (118) includes servers, groups of servers, and / or other similar devices. In some embodiments, the V2I system (118) is associated with a local government or a private organization (e.g., a private organization maintaining V2I devices (110), etc.).

[0027] The number and arrangement of elements illustrated in FIG. 1 are provided as examples. There may be additional elements, fewer elements, different elements, and / or differently arranged elements than those illustrated in FIG. 1. Additionally, or alternatively, at least one element of the environment (100) may perform one or more functions as described as performed by at least one different element of FIG. 1. Additionally, or alternatively, at least one set of elements of the environment (100) may perform one or more functions as described as performed by at least one different set of elements of the environment (100).

[0028] Now, referring to FIG. 2, a vehicle (200) (which may be identical or similar to the vehicles (102) of FIG. 1) includes or is associated with an autonomous driving system (202), a powertrain control system (204), a steering control system (206), and a brake system (208). In some embodiments, the vehicle (200) is identical or similar to the vehicles (102) (see FIG. 1). In some embodiments, the autonomous driving system (202) is configured to impart autonomous driving capabilities to the vehicle (200) [e.g., without limitation, fully autonomous vehicles (e.g., vehicles that eliminate reliance on human intervention such as Level 5 ADS-operated vehicles), highly autonomous vehicles (e.g., vehicles that eliminate reliance on human intervention in certain situations such as Level 4 ADS-operated vehicles), conditionally autonomous vehicles (e.g., vehicles that eliminate reliance on human intervention in limited situations such as Level 3 ADS-operated vehicles) and / or implement at least one driving automation or maneuver-based function, feature, device, and / or the like that enables the vehicle (200) to operate partially or fully without human intervention]. In one embodiment, the autonomous driving system (202) includes a dynamic or strategic function required to operate the vehicle (200) in on-road traffic and to perform some or all of the Dynamic Driving Task (DDT) on a continuous basis. In another embodiment, the autonomous driving system (202) includes an Advanced Driver Assistance System (ADAS) that includes driver assistance features. The autonomous driving system (202) supports various levels of driving automation ranging from no driving automation (e.g., Level 0) to full driving automation (e.g., Level 5).For a detailed description of fully autonomous vehicles and highly autonomous vehicles, refer to SAE International standard J3016. Classification and definition of terms related to On-Road Motor Vehicle Automated Driving Systems. In some embodiments, the vehicle (200) is associated with an autonomous driving fleet manager and / or a ride-sharing company.

[0029] The autonomous driving system (202) includes a sensor suite comprising one or more devices such as cameras (202a), LiDAR sensors (202b), radar sensors (202c), and microphones (202d). In some embodiments, the autonomous driving system (202) may include more or fewer devices and / or different devices (e.g., ultrasonic sensors, inertial sensors, GPS receivers (discussed below), mileage sensors that generate data associated with an indication of the distance traveled by the vehicle (200), etc.). In some embodiments, the autonomous driving system (202) uses one or more devices included in the autonomous driving system (202) to generate data associated with the environment (100) described herein. Data generated by one or more devices of the autonomous driving system (202) may be used by one or more systems described herein to observe the environment (e.g., environment (100)) in which the vehicle (200) is located. In some embodiments, the autonomous driving system (202) includes a communication device (202e), an autonomous driving vehicle compute (202f), a drive-by-wire (DBW) system (202h), and a safety controller (202g).

[0030] The cameras (202a) include at least one device configured to communicate with a communication device (202e), an autonomous vehicle compute (202f), and / or a safety controller (202g) via a bus (e.g., a bus identical or similar to the bus (302) of FIG. 3). The cameras (202a) include at least one camera (e.g., a digital camera using a light sensor such as a charge-coupled device (CCD), a thermal camera, an infrared (IR) camera, an event camera, etc.) for capturing images including physical objects (e.g., cars, buses, curbs, people, etc.). In some embodiments, the camera (202a) generates camera data as output. In some embodiments, the cameras (202a) may include one of the wide-angle imaging systems (500, 501, 600, or 700) described below in connection with FIG. 5a, FIG. 5b, FIG. 6, and FIG. 7. In some embodiments, the cameras (202a) may include one or more features described below for a wide-angle imaging system (500, 501, 600, or 700). In some embodiments, the camera (202a) generates camera data including image data associated with an image. In this embodiment, the image data may specify at least one parameter corresponding to the image (e.g., image characteristics such as exposure, brightness, etc., image timestamp, etc.). In this embodiment, the image may be in a format (e.g., RAW, JPEG, PNG, etc.). In some embodiments, the camera (202a) includes a plurality of independent cameras configured on a vehicle (e.g., positioned on a vehicle) to capture images for stereopsis (stereo vision).In some embodiments, the camera (202a) comprises a plurality of cameras, and the plurality of cameras generate image data and transmit the image data to an autonomous vehicle compute (202f) and / or a fleet management system (e.g., a fleet management system identical or similar to the fleet management system (116) of FIG. 1). In these embodiments, the autonomous vehicle compute (202f) determines the depth to one or more objects within the field of view of at least two of the plurality of cameras based on image data from at least two cameras. In some embodiments, the cameras (202a) are configured to capture images of objects within a distance (e.g., up to 100 meters, up to 1 kilometer, etc.) from the cameras (202a). Accordingly, the cameras (202a) include features such as sensors and lenses optimized to recognize objects at one or more distances from the cameras (202a).

[0031] In an embodiment, the camera (202a) comprises at least one camera configured to capture one or more images associated with one or more traffic lights, street signs, and / or other physical objects that provide visual navigation information. In some embodiments, the camera (202a) generates traffic light data associated with one or more images. In some embodiments, the camera (202a) generates Traffic Light Detection (TLD) data associated with one or more images, including formats (e.g., RAW, JPEG, PNG, etc.). In some embodiments, the camera (202a) generating TLD data differs from other systems described herein that integrate cameras in that the camera (202a) may include one or more cameras with a wide field of view (e.g., wide-angle lenses, fisheye lenses, lenses with a field of view of approximately 120 degrees, 150 degrees, 180 degrees or more) so that the camera (202a) can generate images of as many physical objects as possible. In some embodiments, the camera (202a) may include a protective layer (e.g., a curved protective layer) placed over the object lens of the camera (202a) to protect the front surface of the object lens (e.g., a wide-angle lens or a fisheye lens) facing the scene. In some cases, the protective layer may be configured to protect the object lens (e.g., the front surface of the object lens) from physical damage (e.g., scratches) while allowing the object lens to project a wide-angle or ultra-wide-angle image of the scene onto the image sensor of the camera (202a).

[0032] LiDAR (Light Detection and Ranging) sensors (202) include at least one device configured to communicate with a communication device (202), an autonomous vehicle computer (202f), and / or a safety controller (202g) via a bus (e.g., a bus identical or similar to the bus (302) of FIG. 3). LiDAR sensors (202b) include a system configured to transmit light from a light emitter (e.g., a laser transmitter). The light emitted by the LiDAR sensors (202b) includes light outside the visible spectrum (e.g., infrared light, etc.). In some embodiments, during operation, the light emitted by the LiDAR sensors (202b) encounters a physical object (e.g., a vehicle) and is reflected back to the LiDAR sensors (202b). In some embodiments, the light emitted by the LiDAR sensors (202b) does not penetrate the physical objects the light encounters. The LiDAR sensors (202b) also include at least one light detector that detects light emitted from a light emitter after light encounters a physical object. In some embodiments, at least one data processing system associated with the LiDAR sensors (202b) generates an image (e.g., a point cloud, a combined point cloud, etc.) representing objects included in the field of view of the LiDAR sensors (202b). In some embodiments, at least one data processing system associated with the LiDAR sensors (202b) generates an image representing the boundaries of physical objects, surfaces of physical objects (e.g., the topology of surfaces), etc. In these embodiments, the image is used to determine the boundaries of physical objects within the field of view of the LiDAR sensors (202b).

[0033] The radar (Radio Detection and Ranging) sensors (202c) include at least one device configured to communicate with a communication device (202e), an autonomous vehicle computer (202f), and / or a safety controller (202g) via a bus (e.g., a bus identical or similar to the bus (302) of FIG. 3). The radar sensors (202c) include a system configured to transmit radio waves (in pulsed form or continuously). The radio waves transmitted by the radar sensors (202c) include radio waves within a predetermined spectrum. In some embodiments, during operation, the radio waves transmitted by the radar sensors (202c) encounter a physical object and are reflected back to the radar sensors (202c). In some embodiments, the radio waves transmitted by the radar sensors (202c) are not reflected by some object. In some embodiments, at least one data processing system associated with the radar sensors (202c) generates signals representing objects included within the field of view of the radar sensors (202c). For example, at least one data processing system associated with the radar sensors (202c) generates an image representing the boundaries of physical objects, surfaces of physical objects (e.g., topology of surfaces), etc. In some embodiments, the image is used to determine the boundaries of physical objects within the field of view of the radar sensors (202c).

[0034] The microphones (202d) include at least one device configured to communicate with a communication device (202e), an autonomous vehicle computer (202f), and / or a safety controller (202g) via a bus (e.g., a bus identical or similar to the bus (302) of FIG. 3). The microphones (202d) include one or more microphones (e.g., array microphones, external microphones, etc.) that capture audio signals and generate data associated with (e.g., indicating) the audio signals. In some embodiments, the microphones (202d) include transducer devices and / or similar devices. In some embodiments, one or more systems described herein may receive data generated by the microphones (202d) and determine the position of an object (e.g., distance, etc.) relative to the vehicle (200) based on audio signals associated with the data.

[0035] The communication device (202e) includes at least one device configured to communicate with cameras (202a), LiDAR sensors (202b), radar sensors (202c), microphones (202d), an autonomous vehicle compute (202f), a safety controller (202g), and / or a Drive-By-Wire (DBW) system (202h). For example, the communication device (202e) may include a device identical or similar to the communication interface (314) of FIG. 3. In some embodiments, the communication device (202e) includes a vehicle-to-vehicle (V2V) communication device (e.g., a device that enables wireless communication of data between vehicles).

[0036] The autonomous vehicle compute (202f) includes at least one device configured to communicate with cameras (202a), LiDAR sensors (202b), radar sensors (202c), microphones (202d), a communication device (202e), a safety controller (202g), and / or a DBW system (202h). In some embodiments, the autonomous vehicle compute (202f) includes devices such as a client device, a mobile device (e.g., a cellular phone, a tablet, etc.), a server (e.g., a computing device including one or more central processing units, graphics processing units, etc.). In some embodiments, the autonomous vehicle compute (202f) is identical or similar to the autonomous vehicle compute (400) described herein. Additionally or alternatively, in some embodiments, the autonomous vehicle compute (202f) is configured to communicate with an autonomous vehicle system (e.g., an autonomous vehicle system identical or similar to the remote AV system (114) of FIG. 1), a fleet management system (e.g., a fleet management system identical or similar to the fleet management system (116) of FIG. 1), a V2I device (e.g., a V2I device identical or similar to the V2I device (110) of FIG. 1), and / or a V2I system (e.g., a V2I system identical or similar to the V2I system (118) of FIG. 1).

[0037] The safety controller (202g) includes at least one device configured to communicate with cameras (202a), LiDAR sensors (202b), radar sensors (202c), microphones (202d), a communication device (202e), an autonomous vehicle computer (202f), and / or a DBW system (202h). In some embodiments, the safety controller (202g) includes one or more controllers (electric controllers, electromechanical controllers, etc.) configured to generate and / or transmit control signals for operating one or more devices of the vehicle (200) (e.g., powertrain control system (204), steering control system (206), brake system (208), etc.). In some embodiments, the safety controller (202g) is configured to generate control signals that take precedence (e.g., nullify) over control signals generated and / or transmitted by the autonomous vehicle computer (202f).

[0038] The DBW system (202h) includes at least one device configured to communicate with a communication device (202e) and / or an autonomous vehicle computer (202f). In some embodiments, the DBW system (202h) includes one or more controllers (e.g., electric controllers, electromechanical controllers, etc.) configured to generate and / or transmit control signals to operate one or more devices of the vehicle (200) (e.g., powertrain control system (204), steering control system (206), brake system (208), etc.). Additionally, or alternatively, one or more controllers of the DBW system (202h) are configured to generate and / or transmit control signals to operate at least one different device of the vehicle (200) [e.g., turn signals, headlights, door locks, windshield wipers, etc.].

[0039] The powertrain control system (204) includes at least one device configured to communicate with the DBW system (202h). In some embodiments, the powertrain control system (204) includes at least one controller, actuator, etc. In some embodiments, the powertrain control system (204) receives control signals from the DBW system (202h), and the powertrain control system (204) causes the vehicle (200) to perform longitudinal vehicle motion, such as starting to move forward, stopping moving forward, starting to move backward, stopping moving backward, accelerating in one direction, decelerating in one direction, or lateral vehicle motion, such as turning left, turning right, and / or etc. In an embodiment, the powertrain control system (204) increases, keeps the energy (e.g., fuel, electricity, and / or etc.) supplied to the motor of the vehicle, or keeps it the same, or decreases it, thereby causing at least one wheel of the vehicle (200) to rotate or not rotate.

[0040] The steering control system (206) includes at least one device configured to rotate one or more wheels of the vehicle (200). In some embodiments, the steering control system (206) includes at least one controller, actuator, etc. In some embodiments, the steering control system (206) causes the two front wheels and / or two rear wheels of the vehicle (200) to rotate to the left or right, thereby causing the vehicle (200) to rotate to the left or right. In other words, the steering control system (206) causes activities necessary for controlling the y-axis component of the vehicle motion.

[0041] The brake system (208) includes at least one device configured to actuate one or more brakes to cause the vehicle (200) to reduce speed and / or remain at a stop. In some embodiments, the brake system (208) includes at least one controller and / or actuator configured to close one or more calipers associated with one or more wheels of the vehicle (200) on corresponding rotors of the vehicle (200). Additionally or alternatively, in some embodiments, the brake system (208) includes an automatic emergency braking (AEB) system, a regenerative braking system, etc.

[0042] In some embodiments, the vehicle (200) includes at least one platform sensor (not explicitly illustrated) that measures or infers attributes of the state or condition of the vehicle (200). In some embodiments, the vehicle (200) includes platform sensors such as a global positioning system (GPS) receiver, an inertial measurement unit (IMU), a wheel speed sensor, a wheel brake pressure sensor, a wheel torque sensor, an engine torque sensor, a steering angle sensor, and / or others. Although the brake system (208) is illustrated in FIG. 2 as being located near the vehicle (200), the brake system (208) may be located at any location within the vehicle (200).

[0043] Now, referring to FIG. 3, a schematic diagram of a device (300) is illustrated. As illustrated, the device (300) includes a processor (304), memory (306), storage component (308), input interface (310), output interface (312), communication interface (314), and bus (302). In some embodiments, the device (300) corresponds to at least one device of the vehicles (102) (e.g., at least one device of the system of the vehicles (102)), at least one device of the remote AV system (114), at least one device of the fleet management system (116), at least one device of the vehicle-to-infrastructure system (118), and / or one or more devices of the network (112) (e.g., one or more devices of the system of the network (112)). In some embodiments, one or more devices of the vehicles (102) (e.g., one or more devices of the system of the vehicles (102)), at least one device of the remote AV system (114), at least one device of the fleet management system (116), at least one device of the vehicle-to-infrastructure system (118), and / or one or more devices of the network (112) (e.g., one or more devices of the system of the network (112)) include at least one device (300) and / or at least one component of the device (300). As illustrated in FIG. 3, the device (300) includes a bus (302), a processor (304), a memory (306), a storage component (308), an input interface (310), an output interface (312), and a communication interface (314).

[0044] The bus (302) includes a component that enables communication between the components of the device (300). In some cases, the processor (304) includes a processor [e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), and / or the like], a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function [e.g., a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or the like]. The memory (306) includes random access memory (RAM), read-only memory (ROM), and / or other types of dynamic and / or static storage devices (e.g., flash memory, magnetic memory, optical memory, and / or etc.) that store data and / or instructions for use by the processor (304).

[0045] The storage component (308) stores data and / or software related to the operation and use of the device (300). In some embodiments, the storage component (308) includes a hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, solid-state disk, etc.), a CD (compact disc), a DVD (digital versatile disc), a floppy disk, a cartridge, a magnetic tape, a CD-ROM, RAM, PROM, EPROM, FLASH-EPROM, NV-RAM, and / or other types of computer-readable media, together with a corresponding drive.

[0046] The input interface (310) includes a component that enables the device (300) to receive information, for example, through user input (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, microphone, camera, etc.). Additionally or alternatively, in some embodiments, the input interface (310) includes a sensor that detects information (e.g., a GPS (global positioning system) receiver, accelerometer, gyroscope, actuator, etc.). The output interface (312) includes a component that provides output information from the device (300) (e.g., a display, speaker, one or more light-emitting diodes (LEDs), etc.).

[0047] In some embodiments, the communication interface (314) includes a transceiver-like component (e.g., a transceiver, an individual receiver and a transmitter, etc.) that enables the device (300) to communicate with other devices via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. In some embodiments, the communication interface (314) allows the device (300) to receive information from other devices and / or provide information to other devices. In some embodiments, the communication interface (314) includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0048] In some embodiments, the device (300) performs one or more processes described herein. The device (300) performs these processes based on a processor (304) executing software instructions stored by a computer-readable medium, such as a memory (305) and / or a storage component (308). A computer-readable medium (e.g., a non-transient computer-readable medium) is defined herein as a non-transient memory device. A non-transient memory device includes a memory space located within a single physical storage device or a memory space distributed across a plurality of physical storage devices.

[0049] In some embodiments, software instructions are read into memory (306) and / or storage components (308) from another computer-readable medium or another device via a communication interface (314). When executed, the software instructions stored in memory (306) and / or storage components (308) cause the processor (304) to perform one or more of the processes described herein. Additionally or alternatively, hardwired circuitry is used instead of or in combination with software instructions to perform one or more of the processes described herein. Accordingly, the embodiments described herein are not limited to any specific combination of hardware circuitry and software unless otherwise explicitly stated.

[0050] The memory (306) and / or storage component (308) comprises data storage or at least one data structure (e.g., a database, etc.). The device (300) may receive information from the data storage or at least one data structure within the memory (306) or storage component (308), store information therein, communicate information therein, or retrieve information stored therein. In some embodiments, the information comprises network data, input data, output data, or any combination thereof.

[0051] In some embodiments, the device (300) is configured to execute software instructions stored in memory (306) and / or in the memory of another device (e.g., another device identical or similar to the device (300)). As used herein, the term “module” refers to at least one instruction stored in memory (306) and / or in the memory of another device that causes the device (300) (e.g., at least one component of the device (300)) to perform one or more processes described herein when executed by the processor (304) and / or by the processor of another device (e.g., another device identical or similar to the device (300)). In some embodiments, the module is implemented in software, firmware, hardware, etc.

[0052] The number and arrangement of components illustrated in FIG. 3 are provided as examples. In some embodiments, the device (300) may include additional components, fewer components, different components, or differently arranged components than illustrated in FIG. 3. Additionally or alternatively, a set of components of the device (300) (e.g., one or more components) may perform one or more functions described as being performed by other components of the device (300) or a set of other components.

[0053] Now, referring to FIG. 4, an exemplary block diagram of an autonomous vehicle compute (400) (sometimes referred to as the “AV stack”) is illustrated. As illustrated, the autonomous vehicle compute (400) includes a perception system (402) (sometimes referred to as the perception module), a planning system (404) (sometimes referred to as the planning module), a localization system (406) (sometimes referred to as the localization module), a control system (408) (sometimes referred to as the control module), and a database (410). In some embodiments, the perception system (402), the planning system (404), the localization system (406), the control system (408), and the database (410) are included in and / or implemented in the autonomous vehicle navigation system of the vehicle (e.g., the autonomous vehicle compute (202f) of the vehicle (200)). Additionally or alternatively, in some embodiments, the perception system (402), planning system (404), localization system (406), control system (408), and database (410) are included in one or more standalone systems (e.g., one or more systems identical or similar to the autonomous vehicle computer (400)). In some embodiments, the perception system (402), planning system (404), localization system (406), control system (408), and database (410) are included in one or more standalone systems located in the vehicle and / or at least one remote system as described herein. In some embodiments, any and / or all systems included in the autonomous vehicle computer (400) are implemented in software (e.g., software instructions stored in memory), computer hardware (e.g., by microprocessors, microcontrollers, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.), or a combination of computer software and computer hardware.It will also be understood that in some embodiments, the autonomous vehicle computer (400) is configured to communicate with a remote system (e.g., an autonomous vehicle system identical or similar to the remote AV system (114), a fleet management system identical or similar to the fleet management system (116), a V2I system identical or similar to the V2I system (118), etc.).

[0054] In some embodiments, the perception system (402) receives data associated with at least one physical object in the environment (e.g., data used by the perception system (402) to detect at least one physical object) and classifies at least one physical object. In some embodiments, the perception system (402) receives image data captured by at least one camera (e.g., cameras (202a)), and the image is associated with one or more physical objects within the field of view of at least one camera (e.g., representing one or more physical objects). In these embodiments, the perception system (402) may include physical objects (e.g., bicycles, vehicles, traffic signs, pedestrians, and / or etc.). In some embodiments, the perception system (402) transmits data associated with the classification of physical objects to the planning system (404) based on the perception system (402) classifying physical objects.

[0055] In some embodiments, the planning system (404) receives data associated with a destination and generates data associated with at least one route (e.g., routes (106)) through which a vehicle (e.g., vehicles (102)) can travel toward the destination. In some embodiments, the planning system (404) receives data from the perception system (402) periodically or continuously (e.g., data associated with the classification of physical objects described above), and the planning system (404) updates at least one trajectory or generates at least one different trajectory based on the data generated by the perception system (402). In other words, the planning system (404) can perform tactical function-related tasks required to operate the vehicle (102) in on-road traffic. Tactical efforts include manipulating the vehicle in traffic situations during the journey, including but not limited to deciding whether and when to overtake another vehicle, changing lanes, or selecting appropriate speed, acceleration, deceleration, etc. In some embodiments, the planning system (404) receives data associated with the updated position of the vehicle (e.g., vehicle (102)) from the localization system (406), and the planning system (404) updates at least one trajectory or generates at least one different trajectory based on the data generated by the localization system (406).

[0056] In some embodiments, the localization system (406) receives data associated with (e.g., indicating) a location of a vehicle (e.g., vehicles (102)) in a region. In some embodiments, the localization system (406) receives LiDAR data associated with at least one point cloud generated by at least one LiDAR sensor (e.g., LiDAR sensors (202b)). In certain embodiments, the localization system (406) receives data associated with at least one point cloud from a plurality of LiDAR sensors, and the localization system (406) generates a combined point cloud based on each of the point clouds. In these embodiments, the localization system (406) compares at least one point cloud or the combined point cloud with a two-dimensional (2D) and / or three-dimensional (3D) map of the region stored in the database (410). Subsequently, the localization system (406) determines the position of the vehicle within the area based on the localization system (406) comparing at least one point cloud or a combined point cloud with a map. In some embodiments, the map includes a combined point cloud of the area created prior to the navigation of the vehicle. In some embodiments, the map includes, without limitation, a high-precision map of road geometric properties, a map describing road network connectivity properties, a map describing road physical properties (e.g., traffic speed, traffic volume, number of vehicle traffic lanes and cyclist traffic lanes, lane width, lane traffic direction, or lane marker type and location, or a combination thereof), and a map describing road features, such as crosswalks, traffic signs, or various types of other driving signals, spatial locations. In some embodiments, the map is generated in real time based on data received by a perception system.

[0057] In another embodiment, the localization system (406) receives Global Navigation Satellite System (GNSS) data generated by a Global Positioning System (GPS) receiver. In some embodiments, the localization system (406) receives GNSS data associated with the position of a vehicle in a corresponding area, and the localization system (406) determines the latitude and longitude of the vehicle in that area. In these embodiments, the localization system (406) determines the position of the vehicle in that area based on the latitude and longitude of the vehicle. In some embodiments, the localization system (406) generates data associated with the position of the vehicle. In some embodiments, the localization system (406) generates data associated with the position of the vehicle based on the determination of the position of the vehicle by the localization system (406). In these embodiments, the data associated with the position of the vehicle includes data associated with one or more semantic properties corresponding to the position of the vehicle.

[0058] In some embodiments, the control system (408) receives data associated with at least one trajectory from the planning system (404), and the control system (408) controls the operation of the vehicle. In some embodiments, the control system (408) receives data associated with at least one trajectory from the planning system (404), and the control system (408) controls the operation of the vehicle by generating and transmitting control signals to cause the powertrain control system (e.g., DBW system (202h), powertrain control system (204), etc.), the steering control system (e.g., steering control system (206)), and / or the brake system (e.g., brake system (208)) to operate. For example, the control system (408) is configured to perform operational functions such as lateral vehicle motion control or longitudinal vehicle motion control. Lateral vehicle motion control causes activities necessary for the adjustment of the y-axis component of the vehicle motion. Longitudinal vehicle motion control causes activities necessary for the adjustment of the x-axis component of the vehicle motion. In an embodiment where the trajectory includes a left turn, the control system (408) transmits a control signal to cause the steering control system (206) to adjust the steering angle of the vehicle (200) so that the vehicle (200) turns left. Additionally, or alternatively, the control system (408) generates and transmits control signals to cause other devices of the vehicle (200) (e.g., headlights, turn signals, door locks, windshield wipers, etc.) to change their states.

[0059] In some embodiments, the cognitive system (402), the planning system (404), the localization system (406), and / or the control system (408) implement at least one machine learning model (e.g., at least one multilayer perceptron (MLP), at least one convolutional neural network (CNN), at least one recurrent neural network (RNN), at least one autoencoder, at least one transformer, etc.). In some embodiments, the cognitive system (402), the planning system (404), the localization system (406), and / or the control system (408) implement at least one machine learning model alone or in combination with one or more of the aforementioned systems. In some embodiments, the perception system (402), the planning system (404), the localization system (406), and / or the control system (408) implement at least one machine learning model as part of a pipeline (e.g., a pipeline for identifying one or more objects located in an environment, etc.). Examples of implementations of machine learning models are included below with respect to FIGS. 4b through 4d.

[0060] The database (410) stores data transmitted to and received from and / or updated by the perception system (402), planning system (404), localization system (406) and / or control system (408). In some embodiments, the database (410) includes a storage component (e.g., a storage component identical or similar to the storage component (308) of FIG. 3) that stores data and / or software related to operation and uses at least one system of the autonomous vehicle compute (400). In some embodiments, the database (410) stores data associated with 2D and / or 3D maps of at least one area. In some embodiments, the database (410) stores data associated with 2D maps and / or 3D maps of parts of a city, parts of multiple cities, multiple cities, counties, states (e.g., regions), and / or the like. In this embodiment, a vehicle (e.g., a vehicle identical or similar to vehicle (102) and / or vehicle (200)) can travel along one or more drivable areas (e.g., a single-lane road, a multi-lane road, a highway, a back road, an off-road trail, etc.) and can enable at least one LiDAR sensor (e.g., a LiDAR sensor identical or similar to LiDAR sensor (202b)) to generate data associated with an image representing objects included in the field of view of at least one LiDAR sensor.

[0061] In some embodiments, the database (410) is implemented across multiple devices. In some embodiments, the database (410) is included in a vehicle (e.g., a vehicle identical or similar to vehicle (102) and / or vehicle (200)), an autonomous vehicle system (e.g., an autonomous vehicle system identical or similar to remote AV system (114)), a fleet management system (e.g., a fleet management system identical or similar to fleet management system (116) of FIG. 1), a V2I system (e.g., a V2I system identical or similar to V2I system (118) of FIG. 1), etc.

[0062] Now, referring to FIG. 4b, a diagram of an exemplary embodiment of a machine learning model is illustrated. More specifically, a diagram of an exemplary embodiment of a convolutional neural network (CNN) (420) is illustrated. For the purposes of illustration, the following description of the CNN (420) relates to an implementation of the CNN (420) by a cognitive system (402). However, it will be understood that in some embodiments, the CNN (420) (e.g., one or more components of the CNN (420)) is implemented by a system different from or in addition to the cognitive system (402), such as a planning system (404), a localization system (406), and / or a control system (408). Although the CNN (420) includes certain features as described herein, these features are provided for illustrative purposes only and are not intended to limit the present disclosure.

[0063] The CNN (420) includes a plurality of convolution layers, including a first convolution layer (422), a second convolution layer (424), and a convolution layer (426). In some embodiments, the CNN (420) includes a subsampling layer (428) (sometimes referred to as a pooling layer). In some embodiments, the subsampling layer (428) and / or other subsampling layers have a dimension less than the dimension of the upstream system (i.e., the number of nodes). By the subsampling layer (428) having a dimension smaller than the dimension of the upstream layer, the CNN (420) incorporates the amount of data associated with the initial input and / or output of the upstream layer, thereby reducing the amount of computation required for the CNN (420) to perform downstream convolution operations. Additionally or alternatively, by having a subsampling layer (428) associated with at least one subsampling function (e.g., configured to perform this) (as described below in relation to FIGS. 4c and 4d), the CNN (420) incorporates an amount of data associated with the initial input.

[0064] The cognitive system (402) performs convolution operations to generate each output based on the cognitive system (402) providing each input and / or output associated with each of the first convolution layer (422), the second convolution layer (424), and the convolution layer (426). In some embodiments, the cognitive system (402) implements a CNN (420) based on the cognitive system (402) providing data as input to the first convolution layer (422), the second convolution layer (424), and the convolution layer (426). In this embodiment, the perception system (402) provides data as input to the first convolution layer (422), the second convolution layer (424), and the convolution layer (426) based on the perception system (402) receiving data from one or more different systems (e.g., one or more systems of a vehicle identical or similar to the vehicle (102), a remote AV system identical or similar to the remote AV system (114), a fleet management system identical or similar to the fleet management system (116), a V2I system identical or similar to the V2I system (118), etc.). A detailed description of the convolution operation is included below in relation to FIG. 4c.

[0065] In some embodiments, the perception system (402) provides data associated with the input (referred to as the initial input) to the first convolution layer (422), and the perception system (402) uses the first convolution layer (422) to generate data associated with the output. In some embodiments, the perception system (402) provides the output generated by the convolution layer as an input to a different convolution layer. For example, the perception system (402) provides the output of the first convolution layer (422) as an input to a subsampling layer (428), a second convolution layer (424), and / or a convolution layer (426). In these embodiments, the first convolution layer (422) is referred to as the upstream layer, and the subsampling layer (428), the second convolution layer (424), and / or a convolution layer (426) are referred to as the downstream layer. Similarly, in some embodiments, the perception system (402) provides the output of the subsampling layer (428) to the second convolution layer (424) and / or the convolution layer (426), and in this example, the subsampling layer (428) will be referred to as the upstream layer, and the second convolution layer (424) and / or the convolution layer (426) will be referred to as the downstream layer.

[0066] In some embodiments, the cognitive system (402) processes data associated with the input provided to the CNN (420) before the cognitive system (402) provides input to the CNN (420). For example, the cognitive system (402) processes data associated with the input provided to the CNN (420) based on the cognitive system (402) normalizing sensor data (e.g., image data, LiDAR data, radar data, etc.).

[0067] In some embodiments, the CNN (420) generates an output based on the cognitive system (402) performing convolution operations associated with each convolution layer. In some examples, the CNN (420) generates an output based on the cognitive system (402) performing convolution operations associated with each convolution layer and initial data. In some embodiments, the cognitive system (402) generates an output and provides the output as a whole connected layer (430). In some examples, the cognitive system (402) provides the output of the convolution layer (426) as a whole connected layer (430), wherein the whole connected layer (430) includes data associated with a plurality of feature values ​​referred to as F1, F2 ... FN. In this embodiment, the output of the convolution layer (426) includes data associated with a plurality of output feature values ​​representing a prediction.

[0068] In some embodiments, the cognitive system (402) identifies a prediction among a plurality of predictions based on the cognitive system (402) identifying a feature value associated with the highest probability of being an accurate prediction among a plurality of predictions. For example, the entire connected layer (430) includes feature values ​​(F1, F2, ... FN), where F1 is the largest feature value, and the cognitive system (402) identifies the prediction associated with F1 as the accurate prediction among a plurality of predictions. In some embodiments, the cognitive system (402) trains the CNN (420) to generate predictions. In some embodiments, the cognitive system (402) trains the CNN (420) to generate predictions based on the cognitive system (402) providing training data associated with predictions to the CNN (420).

[0069] Now, referring to FIGS. 4c and FIGS. 4d, a diagram of an exemplary operation of the CNN (440) by the cognitive system (402) is illustrated. In some embodiments, the CNN (440) (e.g., one or more components of the CNN (440)) is identical or similar to the CNN (420) (e.g., one or more components of the CNN (420)) (see FIG. 4d).

[0070] In step 450, the perception system (402) provides data associated with an image as input to the CNN (440) (step 450). For example, as illustrated, the perception system (402) provides data associated with an image to the CNN (440), where the image is a grayscale image represented as values ​​stored in a two-dimensional (2D) array. In some embodiments, the data associated with the image may include data associated with a color image, and the color image is represented as values ​​stored in a three-dimensional (3D) array. Additionally, or alternatively, the data associated with the image may include data associated with an infrared image, a radar image, and / or other.

[0071] In step (455), the CNN (440) performs a first convolution function. For example, the CNN (440) performs the first convolution function based on the CNN (440) providing values ​​representing an image as input to one or more neurons (not explicitly shown) included in the first convolution layer (442). In this embodiment, the values ​​representing the image may correspond to values ​​representing regions of the image (sometimes referred to as receptive fields). In some embodiments, each neuron is associated with a filter (not explicitly illustrated). The filter (sometimes referred to as a kernel) may be represented as an array of values ​​of a size corresponding to the values ​​provided as input to the neuron. In one embodiment, the filter may be configured to identify edges (e.g., horizontal lines, vertical lines, straight lines, etc.). In successive convolution layers, filters associated with neurons can be configured to identify successively more complex patterns (e.g., arcs, objects, etc.).

[0072] In some embodiments, the CNN (440) performs a first convolution function based on multiplying a value provided as an input for each of one or more neurons included in the first convolution layer (442) by a value of a filter corresponding to each of one or more neurons. For example, the CNN (440) may generate a single value or an array of values ​​as an output by multiplying a value provided as an input for each of one or more neurons included in the first convolution layer (442) by a value of a filter corresponding to each of one or more neurons. In some embodiments, the collective output of the neurons of the first convolution layer (442) is referred to as the convolution output. In some embodiments, where each neuron has the same filter, the convolution output is referred to as the feature map.

[0073] In some embodiments, the CNN (440) provides the output of each neuron of the first convolution layer (442) to a neuron of the downstream layer. For clarity, the upstream layer may be a layer that transmits data to a different layer (referred to as the downstream layer). For example, the CNN (440) may provide the output of each neuron of the first convolution layer (442) to a corresponding neuron of the subsampling layer. In an embodiment, the CNN (440) provides the output of each neuron of the first convolution layer (442) to a corresponding neuron of the first subsampling layer (444). In some embodiments, the CNN (440) adds a bias value to an aggregate of all values ​​provided to each neuron of the downstream layer. For example, the CNN (440) adds a bias value to an aggregate of all values ​​provided to each neuron of the first subsampling layer (444). In this example, the CNN (440) determines the final value to be provided to each neuron of the first subsampling layer (444) based on the aggregation of all values ​​provided to each neuron and the activation function associated with each neuron of the first subsampling layer (444).

[0074] In step (460), the CNN (440) performs a first subsampling function. For example, the CNN (440) may perform the first subsampling function based on the CNN (440) providing the value output by the first convolution layer (442) to the corresponding neuron of the first subsampling layer (444). In some embodiments, the CNN (440) performs the first subsampling function based on an aggregation function. In an embodiment, the CNN (440) performs the first subsampling function based on the CNN (440) determining the maximum input among the values ​​provided to a given neuron (referred to as the maximum pooling function). In another embodiment, the CNN (440) performs the first subsampling function based on the CNN (440) determining the average input among the values ​​provided to a given neuron (referred to as the average pooling function). In some embodiments, the CNN (440) generates an output based on the CNN (440) providing values ​​to each neuron of the first subsampling layer (444), and the output is sometimes referred to as the subsampled convolution output.

[0075] In step (465), the CNN (440) performs a second convolution function. In some embodiments, the CNN (440) performs the second convolution function in a manner similar to how the CNN (440) described above performs the first convolution function. In some embodiments, the CNN (440) performs the second convolution function based on the CNN (440) providing the value output by the first subsampling layer (444) as an input to one or more neurons (not explicitly shown) included in the second convolution layer (446). In some embodiments, each neuron of the second convolution layer (446) is associated with a filter as described above. The filter(s) associated with the second convolution layer (446) may be configured to identify more complex patterns than the filter associated with the first convolution layer (442), as described above.

[0076] In some embodiments, the CNN (440) performs a second convolution function based on multiplying the value provided as input to each of one or more neurons included in the second convolution layer (446) by the value of the filter corresponding to each of one or more neurons. For example, the CNN (440) may generate a single value or an array of values ​​as output by multiplying the values ​​provided as input to each of one or more neurons included in the second convolution layer (446) by the values ​​of the filter corresponding to each of one or more neurons.

[0077] In some embodiments, the CNN (440) provides the outputs of each neuron of the second convolution layer (446) to the neurons of the downstream layer. For example, the CNN (440) may provide the outputs of each neuron of the first convolution layer (442) to the corresponding neurons of the subsampling layer. In an embodiment, the CNN (440) provides the outputs of each neuron of the first convolution layer (442) to the corresponding neurons of the second subsampling layer (448). In some embodiments, the CNN (440) adds a bias value to the aggregate of all values ​​provided to each neuron of the downstream layer. For example, the CNN (440) adds a bias value to the aggregate of all values ​​provided to each neuron of the second subsampling layer (448). In this embodiment, the CNN (440) determines the final value to be provided to each neuron of the second subsampling layer (448) based on the aggregation of all values ​​provided to each neuron and the activation function associated with each neuron of the second subsampling layer (448).

[0078] In step (470), the CNN (440) performs a second subsampling function. For example, the CNN (440) may perform the second subsampling function based on the CNN (440) providing the value output by the second convolution layer (446) to the corresponding neuron of the second subsampling layer (448). In some embodiments, the CNN (440) performs the second subsampling function based on the CNN (440) using an aggregation function. In an embodiment, the CNN (440) performs the first subsampling function based on the CNN (440) determining the maximum input or average input among the values ​​provided to a given neuron. In some embodiments, the CNN (440) generates an output based on the CNN (440) providing a value to each neuron of the second subsampling layer (448).

[0079] In step (475), the CNN (440) provides the output of each neuron of the second subsampling layer (448) to the total connection layer (449). For example, the CNN (440) provides the output of each neuron of the second subsampling layer (448) to the total connection layers (449) so that the total connection layers (449) generate an output. In some embodiments, the total connection layers (449) are configured to generate an output associated with a prediction (sometimes referred to as a classification). The prediction may include an indication that an object contained in an image provided as input to the CNN (440) includes an object, a set of objects, etc. In some embodiments, the cognitive system (402) performs one or more operations and / or provides the data associated with the prediction to the different system described herein.

[0080] As discussed above, the systems and / or processors mentioned above may be modified to incorporate the improvements disclosed in the context of wide-angle imaging systems.

[0081] In the foregoing description, the aspects and embodiments of the present disclosure have been described with reference to numerous specific details that may vary depending on the embodiment. Accordingly, the description and drawings should be regarded as illustrative rather than restrictive. The sole and exclusive denotation of the scope of the invention, and what the applicant intends to define as the scope of the invention, is the literal and equivalent scope of the set of claims issued in this application in the specific form in which these claims are issued, including any subsequent amendments. Any definitions expressly provided herein for terms included in these claims govern the meaning of such terms as used in the claims. Furthermore, when the term “comprising” is used in the foregoing description or in the following claims, what follows this phrase may be an additional step or entity, or a sub-step / sub-entity of the previously mentioned step or entity.

[0082] Wide-angle imaging system with protection and heating layer

[0083] Wide-angle and ultra-wide-angle imaging systems (collectively referred to herein as wide-angle imaging systems) may be used in various systems, including security and navigation or other applications, in which most of the surrounding environment can be captured and projected onto an image sensor to generate an image signal that may include, as a non-limiting example, a digital image. In some cases, wide-angle and ultra-wide-angle cameras may include wide-angle or ultra-wide-angle imaging systems. Such wide-angle imaging systems may use wide-angle or fisheye lenses having a small focal length configured to provide a large field of view (FOV). A fisheye lens may be a type of ultra-wide-angle lens designed and formed to provide a wide panoramic or hemispherical image of the scene. The fisheye lens may have a focal length of 180 degrees or more, close to 150 degrees, and a very wide FOV of any range formed by 6 to 8 millimeters (mm), 8 to 10 mm, 10 to 12 mm, 12 to 14 mm, 14 to 16 mm, 16 to 18 mm, 18 to 20 mm, or values ​​greater or smaller than these. In some embodiments, the fisheye lens may be a circular fisheye lens that provides a wide-field circular image or a full-frame fisheye lens that provides a wide field across the diagonal of the image sensor. In some embodiments, the FOV of the fisheye lens (e.g., diagonal, horizontal, or vertical FOV) may be 100 to 120 degrees, 120 to 150 degrees, 150 to 170 degrees, 170 to 200 degrees, or any range formed by values ​​greater than these.

[0084] In some embodiments, a wide-field imaging system may be used by a navigation system of an autonomous or semi-autonomous vehicle to monitor the environment surrounding the vehicle. For example, as described above, the camera (202a) of the vehicle (200) may include a camera and / or imaging system having a wide field of view (e.g., a wide-angle lens, a fisheye lens, a lens having a field of view of approximately 120 degrees or more) to generate images of as many physical objects around the vehicle as possible.

[0085] In many applications such as environmental monitoring and navigation, wide-field imaging systems may be exposed to harsh environmental conditions and / or be vulnerable to physical damage (e.g., scratches) caused by direct physical contact with objects and / or particles within the surrounding environment. For example, a wide-field imaging system used by a navigation system of an autonomous or semi-autonomous vehicle to monitor the environment surrounding the vehicle may be exposed to atmospheric / environmental conditions and / or physical impacts that may cause scratches or otherwise reduce the optical quality of the exposed surface (e.g., the front or outer surface facing the scene) of a wide-angle or fisheye lens configured to capture an image of a scene (e.g., the scene surrounding the vehicle) within a large field of view (e.g., greater than 100, 150, or 180 degrees).

[0086] In various embodiments, a camera or imaging system may be placed in an enclosure (e.g., a sensor pod) having a window through which light can enter the object lens of the camera in order to avoid damage to an optical surface that serves as the incident surface of the camera. In some cases, using the window (e.g., a window connected to a sensor pod containing the imaging system) to report the exposure surface (e.g., an outer surface) of the wide-angle or fisheye lens of the wide-angle imaging system may reduce the FOV of the wide-angle imaging system, the amount of light transmitted to the object lens (e.g., a wide-angle or fisheye lens), and in some cases, reduce the quality of the image captured by the wide-angle imaging system. For example, if the fisheye lens of an imaging system has a FOV of 180 degrees or more, surrounding the fisheye lens within the sensor pod or protecting the fisheye lens with a window (e.g., a flat window) may reduce the FOV to less than 180 degrees and limit the amount of light received by the imaging system to form an image. In some cases, internal reflection by the window or otherwise a transparent part of the sensor pod (e.g., associated with a high field angle) may significantly reduce the light redirected toward the image sensor of the imaging system and / or distort the image formed on the image sensor.

[0087] Some of the embodiments disclosed herein include a customized window or protective layer (e.g., a customized transparent dome) that may be configured to protect a wide-angle or fisheye object lens of an imaging system (e.g., a wide-angle imaging system) without (or with minimal) reduction in the FOV of the imaging system, image quality, and / or the relative illumination of an image formed on a corresponding image sensor. In some embodiments, the protective layer may include a scratch-resistant transparent layer having a hardness greater than that of the object lens of the imaging system (e.g., a fisheye lens or a wide-angle lens). Additionally, the protective layer may have a refractive index greater than that of the object lens. In some embodiments, the refractive index of the protective layer may be greater than that of the object lens, and the difference between the refractive index of the protective layer and the refractive index of the object lens may be 0.1 to 0.2, 0.2 to 0.3, 0.3 to 0.4, or any range formed by these values ​​or values ​​greater or smaller. In some embodiments, the protective layer may have a geometry that substantially or closely matches the geometry of the object lens. In some embodiments, the protective layer may include a shape having parameters that match the shape of the object lens but have different values. For example, if the front or outer surface of the object lens facing the imaged scene is spherical or near-spherical, the protective layer may be a spherical or near-spherical shell or window having a different curvature (e.g., smaller). In some embodiments, the radius of curvature of the protective layer may be smaller than the corresponding radius of curvature of the object lens, and the difference between the radii of curvature may be 2 to 5 micrometers (microns), 5 to 10 microns, 10 to 100 microns, 100 to 1000 microns, 1 to 3 mm, 3 to 5 mm, or any range formed by these values ​​or values ​​greater or smaller.

[0088] In some embodiments, the protective layer may be a dome-shaped layer or window comprising sapphire or other scratch-resistant material (e.g., polycarbonate, acrylic, special scratch-resistant glasses, etc.) having low optical absorption loss within the operating wavelength range of the corresponding imaging system. In some embodiments, the transparent protective layer may provide light transmission greater than 80%, greater than 90%, greater than 95%, or greater than that for light (or electromagnetic radiation) having wavelengths within the operating wavelength range of the wide-field imaging system. In some embodiments, the operating wavelength range of the wide-field imaging system may be within the visible light wavelength range (e.g., 380 nm to 780 nm), the near-infrared (near-IR) wavelength range (e.g., 780 nm to 1800 nm), the mid-wave IR (MWIR) wavelength range (e.g., 1.8 microns to 6 microns), or the long-wave IR (LWIR) wavelength range (e.g., 6 microns to 14 microns). In some cases, the protective layer may follow the curvature of the object lens (e.g., the curvature of the outer lens surface facing the imaged scene). In some embodiments, the protective layer may be placed over the object lens and / or mechanically connected to the object lens. In some cases, the protective layer may be mechanically connected to a sensor pod or enclosure comprising at least a part of the corresponding imaging system (e.g., a part of an object lens, a group of imaging lenses, and one or more of an image sensor).

[0089] In some embodiments, at least a portion of the protective layer may be separated from the object lens (e.g., wide-angle or fisheye lens) by a gap between the protective layer (e.g., the inner surface of the protective layer), the outer surface of the object lens facing the protective layer, and the imaged scene. In some of these embodiments, the gap may allow airflow or otherwise fluid flow between the region between the protective layer and the object lens and the surrounding environment (or the internal volume of the sensor pod). In some embodiments, the gap may be provided by making the radius of curvature of the protective layer smaller than the radius of curvature of the outer surface of the object lens. In some embodiments, the gap may be provided by a spacer placed between the outer surface of the object lens and the protective layer. In some embodiments, the spacer may be placed near the edge region of the protective layer and the object lens. In some cases, the spacer may comprise a material that is transparent within the operating wavelength range of the imaging system.

[0090] In some embodiments, a heating layer may be disposed between the protective layer and the object lens to controllably generate and provide heat to at least the protective layer and to control the temperature of the protective layer (e.g., maintain the temperature above a lower limit), thereby preventing the formation of ice, water, or other residues on the outer surface of the protective layer (e.g., a convex outer surface). In some cases, the heating layer may be disposed on the inner surface of the protective layer (e.g., an inner concave surface facing the object lens). In some cases where the object lens is at least partially separated from the protective layer by a gap, the heating layer may be disposed on the inner surface of the protective layer (e.g., an inner concave surface) and separated from the outer surface of the object lens (e.g., an outer convex surface) by the gap. In some cases, the gap between the front surface of the object lens and the protective layer and / or heating layer may be a uniform gap or a tapered gap (e.g., tapering from the center or central region near the optical axis of the object lens toward the edge region). In some cases, the size of the uniform gap or the maximum size of the tapered gap between the front surface of the object lens and the protective layer and / or heating layer may be 0.5 to 1 micron, 1 to 2 micron, 2 to 5 micron, 5 to 10 micron, 10 to 100 micron, 100 to 250 micron, 250 to 500 micron, 500 micron to 1 mm, 1 mm to 5 mm, or any range formed by these values ​​or values ​​larger or smaller.

[0091] In some cases, the protective layer may comprise a hard material, such as sapphire or special glass, having a hardness greater than 500 Knoop (KHN), greater than 700 Knoop (KHN), greater than 1000 Knoop (KHN), greater than 1500 Knoop, greater than 1800 Knoop, or greater than that. In some cases, the hardness of the protective layer may be greater than the hardness of the object lens by more than 100 KHN (Knoop), greater than 200 Knoop, greater than 500 Knoop, greater than 1000 Knoop, or greater than that.

[0092] In some cases, the heating layer may comprise an indium tin oxide (ITO) layer, a carbon nanotube (CNT) layer, or a layer formed by conductive nanoparticles and / or nanowires (e.g., silver nanowires). However, embodiments are not so limited, and layers having other compositions may be used as the heating layer. In some embodiments, the heating layer may be a conductive transparent layer that generates heat when receiving current, while allowing light having a wavelength within the operating wavelength range of the corresponding imaging system (e.g., with a transmittance of 80% or more than 90%) to pass through.

[0093] FIG. 5a schematically illustrates a side cross-sectional view of an exemplary wide-angle imaging system (500) or camera comprising a group of lenses (504) and an object lens (502) configured to form an image of a scene on an image sensor (516) over a very large FOV (507) (e.g., FOV greater than 120 degrees, greater than 140 degrees, or greater than 160 degrees). In some embodiments, the object lens (502) may be a fisheye lens having a focal length of less than 8 mm, less than 10 mm, less than 12 mm, less than 14 mm, or less than 16 mm. A lens group (504) (referred herein to as an imaging lens group) may include a plurality of positive, negative, achromatic, compound, and / or other types of lenses selected and positioned to form an image of a scene using a light beam redirected from a large FOV (507) to the imaging lens group (504) by an object lens (502). In some cases, an aperture may be positioned between two lenses of the imaging lens group (504) or between the imaging lens group (504) and the object lens (502). In the illustrated embodiment, the imaging lens group (504) includes a first positive lens (506) (e.g., plano-convex lens), an aperture (508), a negative doublet (e.g., achromatic doublet), a positive doublet (e.g., achromatic doublet), and a negative lens (514) (e.g., meniscus lens). However, it will be understood that the imaging lens group (504) may include fewer (e.g., one lens) or more lenses (e.g., two, three, four, or more lenses). Accordingly, the exemplary imaging lens group (504) should not be interpreted as being limited.

[0094] In some embodiments, a portion of the object lens (502), an imaging lens group (504), and an image sensor (516) may be contained within an enclosure (518) (e.g., a sensor pod). In some cases, the outer portion of the object lens (502) may be exposed to the environment surrounding the imaging system (500), and the inner portion of the object lens (502) may be positioned inside the enclosure (518). In some cases, the outer portion of the object lens (502) may include the outer surface of the object lens (502), through which a light beam is received and redirected to the imaging lens group (504) to form an image on the image sensor (516).

[0095] In various embodiments, the image sensor (516) may be configured to generate an image signal in response to receiving an image formed by light having wavelengths within the operating wavelength range of the imaging system (500) from the imaging lens group (504). In some cases, the operating wavelength range of the imaging system (500) may be within the visible or infrared (IR) wavelength range (e.g., near-wave, mid-wave, or long-wave infrared). Accordingly, the object lens (502) and the imaging lens group (504) may be configured to transmit light having wavelengths within the operating wavelength range. For example, the object lens (502) and the imaging lens group (504) may be formed from materials having low absorption within the operating wavelength range. In some embodiments, the object lens (502) may include silica glass for visible light and near-infrared imaging, or zinc selenide, zinc sulfide, chalcogenide glasses, tellurite glasses, germanium, silicon for mid-wave IR or long-wave IR imaging. However, embodiments are not so limited, and the object lens (502) may comprise other materials having low absorption within the operating wavelength range of the imaging system. In some embodiments, the object lens (502) may have a diameter (D) in a plane perpendicular to the optical axis (520) of the imaging system (500), wherein D may be 2 mm to 1 cm, 1 cm to 5 cm, 5 cm to 10 cm, 10 cm to 15 cm, or greater. In some embodiments, the object lens (502) may comprise a wide-angle or fisheye lens having a convex surface (e.g., an incident surface toward the scene) having a first radius of curvature (R1) and a concave surface (e.g., an exit surface toward the imaging lens group) having a second radius of curvature (R2). In some embodiments, R1 may be 2, 3, 4, 6, 8, 10 or more times larger than R2.In some embodiments, R1 may be 5 mm to 1 cm, 1 cm to 5 cm, 5 cm to 10 cm, 10 cm to 15 cm or more.

[0096] In some embodiments, a protective layer may be formed on the outer surface (e.g., a convex outer surface) of the object lens (502) facing the imaged scene. In some cases, the protective layer may comprise a dome-shaped layer having substantially uniform thickness formed on the outer surface of the object lens (502). In some cases, the protective layer may comprise a double dome. In some embodiments, the thickness (or maximum thickness) of the protective layer may be 0.1 mm to 0.5 mm, 0.5 mm to 0.7 mm, 0.7 mm to 1 mm, 1 mm to 3 mm, 3 mm to 5 mm, 5 mm to 7 mm, or any range formed by these values ​​or values ​​greater or smaller. In some cases, the protective layer may comprise sapphire or other hard and / or scratch-resistant material (e.g., a material having a hardness greater than 600 Knoop or greater than 100 Knoop).

[0097] In some embodiments, the protective layer may be formed or manufactured separately from the object lens (502) and then attached to the object lens (e.g., bonded) using a bonding method (e.g., thermal bonding) or an adhesive. In some embodiments, the protective layer may comprise a curved layer having a substantially uniform thickness between a convex outer surface and a concave inner surface. In some embodiments, the radius of curvature of the protective layer may be substantially the same as the radius of curvature (R1) of the outer surface (e.g., convex surface) of the object lens (502). In some of these embodiments, the protective layer may be connected to the object lens (502) through a common interface extending to a central region of the object lens (503) near the optical axis (520) of the imaging system (500). In some embodiments, the radius of curvature of the protective layer may be greater than or less than R1. In some embodiments, when the radius of curvature of the protective layer is smaller than R1, the protective layer may be connected to an area of ​​the object lens near the edge or corner of the object lens (502). In various embodiments, the protective layer may be manufactured using etching (wet or dry), molding, polishing, or a combination thereof.

[0098] FIG. 5b schematically illustrates a side cross-sectional view of a wide-angle imaging system (501) comprising a fisheye (or wide-angle) object lens (503) and an imaging lens group (504) protected by a protective layer (505) formed above. The wide-angle imaging system (501) may include one or more features described above in relation to the imaging system (500). In some cases, the protective layer (505) may be placed on or bonded to the convex outer surface of the object lens (503) to form a composite object lens having an outer surface that may be resistant to mechanical damage (e.g., scratching). In some cases, the protective layer (505) may follow the radius of curvature (R1) of the outer surface of the object lens (503). Thus, the protective layer (505) and the object lens (503) may share a continuous interface extending from a central region (near the optical axis (520)) to an edge region of the protective layer (505) and the object lens (503) (away from the optical axis (520)). In some embodiments, the thickness along the radius of curvature of the protective layer (505) [and the radius of convex curvature (R1) of the object lens (503) by this] may be 0.01 to 0.1 mm, 0.1 to 0.5 mm, 0.5 to 1 mm, 1 to 2 mm, 2 to 3 mm, 3 to 5 mm, or any range formed by these values ​​or values ​​greater or smaller.

[0099] In some embodiments, the imaging system (501) may further include a heating layer (not shown) formed between the protective layer (505) and the object lens (503). In some embodiments, the heating layer may be an electric heating layer configured to generate heat and increase the temperature of at least the protective layer (505) in response to receiving current from a power source (e.g., a power supply). The heating layer may include a curved surface that follows the curvature of the concave surface of the protective layer (505) and the convex surface of the object lens (503). In some embodiments, the thickness of the heating layer (e.g., along a radial direction perpendicular to the concave main surface of the protective layer (505)) may be substantially equal to the gap between the protective layer (505) and the object lens (503). Thus, in some embodiments, the heating layer may be interposed between the protective layer (505) and the object lens (503).

[0100] The heating layer may be formed of a conductive material having low optical absorption for light having a wavelength within the operating wavelength range of the imaging system (501). In some embodiments, the heating layer may be configured to allow the transmission of light having a wavelength within the operating wavelength range of the imaging system (501) from the protective layer through its thickness to the object lens (503). In some embodiments, the thickness of the heating layer may be selected based on the optical absorption of the structural material so that the heating layer is transparent to light or electromagnetic radiation having a wavelength within the operating wavelength range of the corresponding imaging system (e.g., to provide optical transmission greater than 90%).

[0101] In some embodiments, the heating layer may be a layer of indium tin oxide (ITO) formed between the protective layer and the object lens (503). In various embodiments, the heating layer may be manufactured by physical vapor deposition (PVD), sputtering, or electron beam evaporation, among other methods.

[0102] In some embodiments, after the manufacture of the protective layer (505), a heating layer may be formed on the concave inner surface of the protective layer (505) to form a double-layer structure combined by the convex surface of the protective layer (505) and the concave surface of the heating layer (e.g., an ITO layer). Next, the resulting double-layer structure may be attached or bonded to the object lens (503). For example, the concave surface of the double-layer structure containing ITO may be bonded to the convex surface of the object lens (503) by a bonding method (e.g., thermal bonding).

[0103] FIG. 6a schematically illustrates a side cross-sectional view of a wide-angle imaging system (600) comprising a fisheye (or wide-angle) object lens (603) protected by a protective layer (605) (e.g., a curved protective layer), an image sensor (516), and an imaging lens group (504) that receives light from the object lens (603) and forms an image on the image sensor (516). The wide-angle imaging system (600) may include one or more features described above in relation to the imaging system (500 and / or 501).

[0104] In some embodiments, the protective layer (605) may comprise a dome-shaped layer having a uniform thickness between an outer convex surface and an inner concave surface. In some cases, the protective layer (605) may have a thickness that varies (e.g., along the radial direction) from a central region of the protective layer (605) (near the optical axis (520)) toward an edge region of the protective layer. In some embodiments, the thickness of the protective layer (605) may vary uniformly in all directions from the center of the protective layer (605) (where the optical axis (520) crosses the protective layer (605)) toward the edge region. In some cases, the thickness of the protective layer (605) may taper or flare from the center of the protective layer (605) toward the edge region. In some cases, the protective layer (605) may include a meniscus lens having a concave inner surface having a radius of curvature different (e.g., greater than) the radius of curvature of the convex outer surface of the protective layer (605). In some embodiments, the protective layer (605) may have a constant thickness or maximum thickness of any range formed by 0.1 mm to 0.5 mm, 0.5 to 1 mm, 1 mm to 1.5 mm, 1.5 to 2 mm, 2 to 4 mm, 4 to 6 mm, or any value greater or smaller than these values.

[0105] In some embodiments, the inner concave radius of curvature of the protective layer (605) may differ from the radius of curvature of the outer convex surface of the object lens (603) by more than 5 microns and less than 10 microns, more than 10 microns and less than 100 microns, more than 0.1 mm and less than 0.4 mm, more than 0.4 mm and less than 2 mm, more than 0.4 mm and less than 1 mm, more than 1 mm and less than 5 mm, or other values.

[0106] In some embodiments, the edge region of the protective layer (605) may be connected, attached, or otherwise bonded to the object lens (603) so that the outer convex surface of the object lens is separated or isolated from the surrounding environment by the protective layer (605). In some of these embodiments, if the radius of the inner concave curvature radius of the protective layer (605) is smaller than the radius of the outer convex surface of the object lens (603) (as in the embodiment shown in FIG. 6a), a gap may be formed between the inner concave surface of the protective layer (605) and the outer convex surface of the object lens (603). In some embodiments, the size (g) (608) of the gap in the radial direction (perpendicular to the inner convex surface of the protective layer) may taper from the center of the protective layer (605) toward the edge region. In some embodiments, the maximum size (g) of the gap (e.g., at the center of the protective layer) max ) may be 0.05 mm to 0.1 mm, 0.1 mm to 1 mm, 1 mm to 2 mm, or any range formed by these values ​​or values ​​greater than or less than these.

[0107] The gap formed between the protective layer (605) and the object lens (603) can enable fluid connection (e.g., air flow (612)) between the volume enclosed between the protective layer (605) and the object lens (603), the internal volume of the enclosure (513) enclosed within the portion of the object lens (603) and the imaging lens group (504), and / or the surrounding environment.

[0108] In some embodiments, to reduce the optical effect of the protective layer on the imaging system (600) [e.g., to reduce the deviation of light radiation incident on the object lens (603) from the original direction and / or incident points of the object lens (603) in the absence of the protective layer (605)], one or more of the radius of curvature of the inner concave surface and the outer convex surface of the protective layer (605), the thickness of the protective layer (605), the thickness of the air gap formed between the protective layer (605) and the object lens (603), and the refractive index of the protective layer (605) may be designed, customized, or selected based on the characteristics and properties of the object lens (603).

[0109] In some examples, the protective layer (605) can improve the optical performance of the imaging system (600) while protecting the object lens (603) and without reducing the intensity of light received by the object lens (603). For example, in some cases, the protective layer (605) (e.g., a meniscus lens) may be configured to provide specific optical power and contribute to the formation of an image on an image sensor.

[0110] In some embodiments, the imaging system (600) may further include a heating layer (606) (e.g., an electric heating layer) formed on the concave inner surface of the protective layer (605) (facing the object lens (603)). The heating layer (606) may be configured to generate heat to increase the temperature of at least the protective layer (605) in response to receiving current from a power source (e.g., a power supply unit (610)). The heating layer (606) may include a curved surface that follows the curvature of the concave surface of the protective layer (605). In some embodiments, the thickness of the heating layer (606) (e.g., along a radial direction perpendicular to the concave surface of the protective layer (605)) may be 20 nm to 100 nm, 100 nm to 1 micrometer, 1 micrometer to 2 micrometers or more.

[0111] The heating layer (606) may comprise a conductive material having low optical absorption for light having a wavelength within the operating wavelength range of the imaging system (600). In some embodiments, the heating layer (606) may allow transmission of light having a wavelength within the operating wavelength range of the imaging system (600) (e.g., visible light wavelength range) from the protective layer (605) to the object lens (603) through its thickness. In some embodiments, the thickness and material composition of the heating layer (60) may be selected to transmit more than 80%, more than 85%, more than 90%, more than 95%, or more than 97% of light or electromagnetic radiation having a wavelength within the operating wavelength range of the imaging system (600) through its thickness. In some embodiments, the heating layer (606) may be a layer of indium tin oxide (ITO). In some embodiments, the heating layer (606) may be a transparent conductive layer formed of other materials, combinations of materials, nanostructures, nanoparticles (e.g., metallic nanoparticles such as gold or silver nanoparticles), nanowires, etc.

[0112] In some embodiments, after the protective layer (605) is prepared, a heating layer (606) may be formed on the concave inner surface of the protective layer (605) to form a double-layer structure joined by the convex surface of the protective layer (605) and the concave surface of the heating layer (606) (e.g., an ITO layer). Next, the resulting double-layer structure may be attached to or bonded to the object lens (603) through the edge regions of the protective layer (605) and the object lens (603). In some embodiments, the edge regions may be connected using an adhesive (e.g., UV-curable glue), thermal bonding, or a combination thereof.

[0113] In some embodiments, one or more side openings may be provided between the object lens (603) and the edge region of the protective layer (605) to enable fluid flow (e.g., air flow (612)) from the enclosed volume between the protective layer (605) and the object lens (603) to the enclosure (518) or the surrounding environment, or vice versa.

[0114] FIG. 6b schematically illustrates a front view of a protective layer (605) of an imaging system (600) showing two side openings (g1, g2) configured to connect the electrical connection between the heating layer (606) and the power supply (610), and the volume enclosed between the protective layer (605) and the object lens (603) and the surrounding environment (or enclosure (518)). In some cases, the side openings (g1, g2) may have different sizes (e.g., thickness and width) and may be positioned at different angular positions with respect to the center of the protective layer (605). In various embodiments, the lateral width of the side openings (g1, g2) in a direction parallel to a plane perpendicular to the optical axis (520) may be 0.5 to 1 mm, 1 mm to 2 mm, 2 to 5 mm, 5 to 10 mm, or any range formed by these values ​​or values ​​greater than or less than these. In various embodiments, the lateral width of the lateral openings (g1, g2) in a direction parallel to a plane perpendicular to the optical axis (520) may be greater than 5%, 10%, 20%, or 30% of the perimeter of the protective layer (605).

[0115] FIG. 7a schematically illustrates a side cross-sectional view of a wide-angle imaging system (700) comprising a fisheye (or wide-angle) object lens (703) and an imaging lens group (504) protected by a protective layer (705). The wide-angle imaging system (700) may include one or more features described above in relation to the imaging systems (600, 500, and 501).

[0116] In some embodiments, the protective layer (705) may include one or more features described above in relation to the protective layer (605).

[0117] In some embodiments, the inner concave radius of curvature of the protective layer (705) may be substantially the same as the radius of curvature of the object lens (703). In some embodiments, the inner concave radius of curvature of the protective layer (705) may differ from the curvature of the outer convex surface of the object lens (703) by more than 0.1 mm and less than 0.4 mm, more than 0.4 mm and less than 0.8 mm, more than 0.8 mm and less than 1 mm, more than 1 mm and less than 2 mm, or by other values ​​(e.g., may be smaller than this).

[0118] In some embodiments, a spacer (704) may be disposed between the edge regions of the protective layer (705) and the object lens (703) to connect the edge regions of the protective layer (705) and the object lens (703) and to provide a gap between the protective layer (705) and the object lens (703). In some of these embodiments, the size or radial distance of the gap between the inner concave surface of the protective layer (705) and the outer convex surface of the object lens (703) in the radial direction may be defined by the spacer (704). In some embodiments, the spacer (704) may be configured to provide a specific gap size or a specific radial distance between the protective layer (705) and the object lens (703). In some embodiments, a specific gap size or radial distance may be 0.05 mm to 0.1 mm, 0.1 mm to 1 mm, 1 mm to 2 mm, or any range formed by these values ​​or values ​​greater or smaller than these.

[0119] The gap formed between the protective layer (705) and the object lens (703) can enable fluid connection (e.g., air flow (612)) between the volume enclosed between the protective layer (705) and the object lens (703), the internal volume of the enclosure (518) in which the portion of the object lens (703) and the imaging lens group (504) are enclosed, and / or the surrounding environment.

[0120] In some embodiments, the spacer (704) may be configured to provide a gap between the protective layer (705) and the object lens (703) by not reducing or at least reducing the area of ​​the object lens (703) or the FOV of the imaging system that can transmit light from the scene to the imaging lens group (504). For example, the spacer (704) may be positioned at or very close to the edge of the protective layer (705). In some embodiments, the spacer (704) may include a plurality of angular segments distributed at different angular positions near the edge of the protective layer (705). In some cases, the protective layer (704) may include a transparent material within the operating wavelength range of the imaging system (700).

[0121] In some embodiments, to reduce the optical effect of the protective layer on the imaging system (700) [e.g., to reduce the deviation of light radiation incident on the object lens (603) from the original direction and / or incident points of the object lens (603) in the absence of the protective layer (705)], one or more of the radius of curvature of the inner concave surface and the outer convex surface of the protective layer (705), the thickness of the protective layer (705), the thickness of the air gap formed between the protective layer (705) and the object lens (703), and the refractive index of the protective layer (705) may be designed, customized, or selected based on the characteristics and properties of the object lens (703).

[0122] In some embodiments, the protective layer (705) can improve the optical performance of the imaging system (700) while protecting the object lens (603) and without reducing the intensity of light received by the object lens (703). For example, in some cases, the protective layer (705) (e.g., a meniscus lens) may be configured to provide specific optical power and contribute to the formation of an image on an image sensor.

[0123] In some embodiments, the imaging system (700) may further include a heating layer (706) formed on the concave inner surface of the protective layer (705) (facing the object lens (703)). The heating layer (706) may be configured to increase the temperature of at least the protective layer (705) in response to receiving current from a power source (e.g., power supply unit (610)). The heating layer (706) may include a curved surface that follows the curvature of the concave surface of the protective layer (705). In some embodiments, the thickness of the heating layer (706) (e.g., along a radial direction perpendicular to the concave surface of the protective layer (705)) may be 20 nm to 100 nm, 100 nm to 1 micrometer or more. The heating layer (706) may include one or more features described above in relation to the heating layer (606).

[0124] In some embodiments, the spacer (704) may include one or more side openings configured to enable fluid flow (e.g., air flow (612)) from the enclosed volume between the protective layer (705) and the object lens (703) to the enclosure (518) or the surrounding environment, or vice versa.

[0125] FIG. 7b schematically illustrates a front view of a protective layer (705) showing an electrical connection between the heating layer (606) and the power supply (610), and two side openings (g3, g4) formed in the spacer (704) and configured to connect the surrounding environment (or enclosure (518)) with the volume enclosed between the protective layer (705) and the object lens (703). In some cases, the side openings (g3, g4) may have different sizes (e.g., thickness and width) and may be positioned at different angles relative to the center of the protective layer (705).

[0126] In various embodiments, the heating layer (606, 706), or the heating layer between the protective layer (505) and the object lens (503), can be manufactured by physical vapor deposition (PVD), sputtering, or electron beam evaporation.

[0127] In the various imaging systems described above, one or both of the protective layer and the object lens may include an antireflection (AR) coating configured to reduce reflections (e.g., Fresnel reflections) from a surface facing the scene within the operating wavelength range of the imaging system. In some cases, the AR coating may be formed on the convex outer surface of the protective layer and / or the object lens.

[0128] <Exemplary Implementations>

[0129] The exemplary embodiments described herein have various features, none of which are essential or entirely responsible for the desirable properties thereof. Various exemplary systems and methods are provided below.

[0130] [Example 1] As an imaging system,

[0131] An object lens configured to capture light rays received from a scene within the field of view (FOV) of an imaging system;

[0132] A lens group configured to receive light from the object lens and to form an image on an image sensor; and

[0133] A curved protective layer that separates the scene from the object lens to protect the object lens.

[0134] Includes,

[0135] An imaging system in which the difference in hardness between the object lens and the protective layer is greater than 500 KHN.

[0136] [Example 2] In Example 1,

[0137] An imaging system in which the object lens includes a wide-angle lens or a fisheye lens.

[0138] [Example 3] In Example 1,

[0139] An imaging system in which the above FOV is greater than 150 degrees.

[0140] [Example 4] In Example 1,

[0141] An imaging system in which the focal length of the object lens is less than 16 millimeters.

[0142] [Example 5] In Example 1,

[0143] An imaging system in which the above protective layer comprises sapphire.

[0144] [Example 6] In Example 1,

[0145] An imaging system in which the protective layer comprises a dome-shaped layer having a convex outer surface facing the scene and a concave surface facing the convex surface of the object lens.

[0146] [Example 7] In Example 6,

[0147] An imaging system in which the radius of curvature of the convex outer surface is different from the radius of curvature of the concave surface.

[0148] [Example 8] In Example 6,

[0149] An imaging system in which the radius of curvature of the convex outer surface is smaller than the radius of curvature of the concave surface.

[0150] [Example 9] In Example 6,

[0151] An imaging system in which the radius of curvature of the convex outer surface is substantially the same as the radius of curvature of the concave surface.

[0152] [Example 10] In Example 6,

[0153] An imaging system in which the radius of curvature of the convex outer surface of the object lens is substantially the same as the radius of curvature of the concave surface of the protective layer.

[0154] [Example 11] In Example 10,

[0155] An imaging system in which the above protective layer is disposed on the object lens forming a common interface.

[0156] [Example 12] In Example 6,

[0157] An imaging system in which the radius of curvature of the convex outer surface of the object lens is greater than the radius of curvature of the concave surface of the protective layer.

[0158] [Example 13] In Example 12,

[0159] An imaging system in which the edge region of the protective layer is connected to or bonded to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer.

[0160] [Example 14] In Example 1,

[0161] An imaging system further comprising a spacer connecting the edge region of the protective layer to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer.

[0162] [Example 15] In Example 13 or Example 14,

[0163] An imaging system in which the size of the gap along a direction perpendicular to the main surface of the protective layer is greater than 0.05 millimeters.

[0164] [Example 16] In Example 13 or Example 14,

[0165] An imaging system further comprising an opening connecting the volume between the object lens and the protective layer in an enclosure or surrounding environment including the lens group above.

[0166] [Example 17] In Example 1,

[0167] An imaging system in which the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters.

[0168] [Example 18] In Example 17,

[0169] An imaging system in which the thickness of the protective layer is less than 2 millimeters.

[0170] [Example 19] In Example 1,

[0171] An imaging system further comprising a transparent heating layer disposed between the object lens and the protective layer, wherein the heating layer is configured to provide heat to at least the protective layer.

[0172] [Example 20] In Example 19,

[0173] An imaging system in which the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system.

[0174] [Example 21] In Example 19,

[0175] An imaging system in which the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system.

[0176] [Example 22] In Example 21,

[0177] An imaging system in which the heating layer comprises indium tin oxide (ITO).

[0178] [Example 23] In Example 22,

[0179] An imaging system in which the thickness of the heating layer is 10 nm to 1000 nm.

[0180] [Example 24] In Example 19,

[0181] An imaging system in which the heating layer is disposed on the inner concave surface of the protective layer facing the object lens.

[0182] [Example 25] In Example 19,

[0183] An imaging system in which the heating layer is electrically connected to a power supply to receive current.

[0184] [Example 26] In Example 25,

[0185] An imaging system in which the heating layer is configured to generate heat in response to receiving the current.

[0186] [Example 27] In Example 1,

[0187] An imaging system in which the protective layer has a light transmittance greater than 95% within the operating wavelength range of the imaging system.

[0188] [Example 28] In Example 1,

[0189] An imaging system further comprising an image sensor, wherein the imaging system is configured to generate an image of the scene on the image sensor.

[0190] [Example 29] In Example 21 or Example 27,

[0191] An imaging system in which the operating wavelength range of the above imaging system is within the visible light wavelength range.

[0192] [Example 30] In Example 21 or Example 27,

[0193] An imaging system in which the operating wavelength range of the above imaging system is within the MWIR (mid-wave infrared) or LWIR (long-wave infrared) wavelength range.

[0194] [Example 31] As an imaging system,

[0195] An object lens configured to capture a ray received from a scene within the field of view (FOV) of an imaging system;

[0196] A group of lenses configured to receive light from the object lens and form an image on an image sensor;

[0197] A curved protective layer separating the scene from the object lens; and

[0198] A transparent heating layer disposed between the protective layer and the object lens? The transparent heating layer is configured to generate heat and provide it to the protective layer.

[0199] An imaging system including

[0200] [Example 32] In Example 31,

[0201] An imaging system in which the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters.

[0202] [Example 33] In Example 32,

[0203] An imaging system in which the thickness of the protective layer is less than 2 millimeters.

[0204] [Example 34] In Example 31,

[0205] An imaging system in which the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system.

[0206] [Example 35] In Example 31,

[0207] An imaging system in which the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system.

[0208] [Example 36] In Example 35,

[0209] An imaging system in which the heating layer comprises indium tin oxide.

[0210] [Example 37] In Example 31,

[0211] An imaging system in which the thickness of the heating layer is 10 nm to 1000 nm.

[0212] [Example 38] In Example 31,

[0213] An imaging system in which the heating layer is disposed on the inner concave surface of the protective layer facing the convex surface of the object lens.

[0214] [Example 39] In Example 38,

[0215] An imaging system in which the radius of curvature of the concave surface of the protective layer is substantially the same as the radius of curvature of the convex surface of the object lens.

[0216] [Example 40] In Example 38,

[0217] An imaging system in which at least the central region of the heating layer is separated from the object lens by a gap from the convex surface of the object lens.

[0218] [Example 41] In Example 40,

[0219] An imaging system in which the radius of curvature of the concave surface of the protective layer is smaller than the radius of curvature of the convex surface of the object lens.

[0220] [Example 42] In Example 31,

[0221] An imaging system in which the heating layer is electrically connected to a power supply to receive current.

[0222] [Example 43] In Example 31,

[0223] An imaging system in which the protective layer has a light transmittance greater than 95% within the operating wavelength range of the imaging system.

[0224] [Example 44] In Example 35 or Example 43,

[0225] An imaging system in which the operating wavelength range of the above imaging system is within the visible light wavelength range.

[0226] [Example 45] In Example 35 or Example 43,

[0227] An imaging system in which the operating wavelength range of the above imaging system is within the MWIR (mid-wave infrared) or LWIR (long-wave infrared) wavelength range.

[0228] [Example 46] In Example 31,

[0229] An imaging system in which the object lens includes a wide-angle lens or a fisheye lens.

[0230] [Example 47] In Example 31,

[0231] An imaging system in which the above FOV is greater than 150 degrees.

[0232] [Example 48] In Example 31,

[0233] An imaging system in which the focal length of the object lens is less than 16 millimeters.

[0234] [Example 49] In Example 31,

[0235] An imaging system in which the above protective layer comprises sapphire.

[0236] [Example 50] As an imaging system,

[0237] An object lens configured to capture a ray received from a scene within the field of view (FOV) of an imaging system;

[0238] A group of lenses configured to receive light from the object lens and form an image on an image sensor;

[0239] A curved protective layer separating the scene from the object lens.

[0240] Includes,

[0241] An imaging system in which the edge regions of the object layer and the protection layer are connected, and the central portion of the protection layer is separated from the central portion of the object by a gap.

[0242] [Example 51] In Example 49,

[0243] An imaging system in which the edge region of the protective layer is connected or bonded to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer.

[0244] [Example 52] In Example 50,

[0245] An imaging system further comprising a spacer connecting the edge region of the protective layer to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer.

[0246] [Example 53] In Example 50 or Example 51,

[0247] An imaging system in which the maximum size of the gap along a direction perpendicular to the main surface of the protective layer is greater than 0.05 millimeters.

[0248] [Example 54] In Example 49,

[0249] An imaging system in which the object lens includes a wide-angle lens or a fisheye lens.

[0250] [Example 55] In Example 49,

[0251] An imaging system in which the above FOV is greater than 150 degrees.

[0252] [Example 56] In Example 49,

[0253] An imaging system in which the focal length of the object lens is less than 16 millimeters.

[0254] [Example 57] In Example 49,

[0255] An imaging system further comprising a transparent heating layer disposed between the object lens and the protective layer, wherein the heating layer is configured to provide heat to at least the protective layer.

[0256] [Example 58] In Example 56,

[0257] An imaging system in which the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system.

[0258] [Example 59] In Example 57,

[0259] An imaging system in which the thickness of the heating layer is 10 nm to 1000 nm.

[0260] [Example 60] In Example 57,

[0261] An imaging system in which the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system.

[0262] [Example 61] In Example 56,

[0263] An imaging system in which the heating layer comprises indium tin oxide.

[0264] [Example 62] In Example 56,

[0265] An imaging system in which the heating layer is disposed on the inner concave surface of the protective layer facing the object lens.

[0266] [Example 63] In Example 56,

[0267] An imaging system in which the heating layer is electrically connected to a power supply to receive current.

[0268] [Example 64] As a method for manufacturing a protected object lens,

[0269] A step of forming a protective layer comprising a first material;

[0270] A step of providing an object lens including a second material;

[0271] A step of forming a transparent heating layer on the surface of the above protective layer;

[0272] A step of bonding the protective layer to the object lens such that the heating layer faces the object lens.

[0273] A method for manufacturing a protected object lens comprising

[0274] [Example 65] In Example 64,

[0275] A method for manufacturing a protected object lens, wherein the step of forming a transparent heating layer on the surface of the protective layer comprises the step of forming the heating layer using a physical film formation method.

[0276] [Example 66] In Example 65,

[0277] A method for manufacturing a protected object lens, wherein the physical film deposition method comprises sputtering, physical vapor deposition (PVD), or electron-beam evaporation.

[0278] [Example 67] In Example 64,

[0279] A method for manufacturing a protected object lens, wherein the heating layer comprises indium tin oxide.

[0280] [Example 68] In Example 64,

[0281] A method for manufacturing a protected object lens, wherein the difference in hardness between the first material and the second material is greater than 500 KHN.

[0282] [Example 69] In Example 64,

[0283] A method for manufacturing a protected object lens, wherein the protective layer comprises sapphire.

[0284] [Example 70] In Example 64,

[0285] A method for manufacturing a protected object lens, wherein the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters.

[0286] [Example 71] In Example 70,

[0287] A method for manufacturing a protected object lens, wherein the thickness of the protective layer is less than 2 millimeters.

[0288] [Example 72] In Example 64,

[0289] A method for manufacturing a protected object lens, wherein the thickness of the heating layer is 10 nm to 1000 nm.

[0290] [Example 73] In Example 64,

[0291] A method for manufacturing a protected object lens, wherein the heating layer is electrically conductive and transparent within the operating wavelength range of the protected object lens.

[0292] [Example 74] In Example 64,

[0293] A method for manufacturing a protected object lens, wherein the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the protected object lens.

[0294] [Example 75] In Example 64,

[0295] A method for manufacturing a protected object lens, wherein the step of connecting a protective layer to the object lens includes the step of bonding the protective layer to the object lens.

[0296] [Example 76] In Example 75,

[0297] A method for manufacturing a protected object lens, wherein the step of connecting a protective layer to the object lens comprises the step of forming a common interface between the protective layer and the object lens near the central region of the protective layer for the object lens.

[0298] [Example 77] In Example 64,

[0299] A method for manufacturing a protected object lens, wherein the step of connecting a protective layer to the object lens comprises the step of connecting an edge region of the protective layer to an edge region of the object lens to form a gap between a center region of the object lens and a center region of the protective layer.

[0300] [Example 78] In Example 77,

[0301] A method for manufacturing a protected object lens, wherein the radius of curvature of the protective layer is smaller than the radius of curvature of the object lens.

[0302] <Term>

[0303] In this description, numerous specific details are presented for the purpose of explanation and to provide a thorough understanding of the disclosure. However, it will be apparent that the embodiments described by this disclosure can be practiced without these specific details. In some instances, well-known structures and devices are illustrated in the form of block diagrams to avoid unnecessarily obscuring aspects of the disclosure.

[0304] Specific arrangements or sequences of schematic elements, such as those representing systems, devices, modules, instruction blocks, data elements, etc., are illustrated in the drawings for convenience of explanation. However, a person skilled in the art will understand that a specific order or arrangement of schematic elements in the drawings does not imply that a specific processing order or sequence of processes, or a separation of processes, is required unless explicitly stated otherwise. Furthermore, the inclusion of schematic elements in the drawings does not imply that such elements are required in all embodiments, or that features represented by such elements may not be included in or combined with other elements in some embodiments.

[0305] Terms such as first, second, third, etc. are used to describe various components, but these elements should not be limited by these terms. Terms such as first, second, third, etc. are used solely to distinguish one element from another. For example, the first contact may be referred to as the second contact, and similarly, the second contact may be referred to as the first contact without departing from the scope of the described embodiments. Both the first contact and the second contact are contacts, but they are not the same contact.

[0306] The technical terms used in the description of the various described embodiments herein are included solely to describe specific embodiments and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include plural forms as well and may be used interchangeably with “one or more” or “at least one” unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and encompasses any and all possible combinations of one or more of the associated enumerated items. It will be further understood that when the terms “includes,” “including,” “comprises,” and / or “comprising” are used in this description, they specify the presence of the mentioned features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0307] As used herein, the term “in case” is optionally interpreted, depending on the context, to mean “when” or “at” or “in response to a decision”, “in response to detecting”, etc. Similarly, the phrase “when determined” or “when [the mentioned condition or event] is detected” is optionally interpreted, depending on the context, to mean “as soon as it is determined”, “in response to the determination”, “as soon as [the mentioned condition or event] is detected”, “in response to the detection of [the mentioned condition or event]”, and / or, etc. Also, as used herein, the terms “have”, “have”, “have”, etc. are intended to be open-ended terms. Also, the phrase “based on” is intended to mean “at least partially based on” unless otherwise explicitly stated.

Claims

Claim 1 An imaging system comprising: an object lens configured to capture light rays received from a scene within the field of view (FOV) of the imaging system; a group of lenses configured to receive light from the object lens and form an image on an image sensor; and a curved protective layer separating the scene from the object lens to protect the object lens, wherein the difference in hardness between the object lens and the protective layer is greater than 500 KHN. Claim 2 An imaging system according to claim 1, wherein the object lens comprises a wide-angle lens or a fisheye lens. Claim 3 An imaging system according to claim 1, wherein the FOV is greater than 150 degrees. Claim 4 An imaging system according to claim 1, wherein the focal length of the object lens is less than 16 millimeters. Claim 5 An imaging system according to claim 1, wherein the protective layer comprises sapphire. Claim 6 An imaging system according to claim 1, wherein the protective layer comprises a dome-shaped layer having a convex outer surface facing the scene and a concave surface facing the convex surface of the object lens. Claim 7 An imaging system according to claim 6, wherein the radius of curvature of the convex outer surface is different from the radius of curvature of the concave surface. Claim 8 An imaging system according to claim 6, wherein the radius of curvature of the convex outer surface is smaller than the radius of curvature of the concave surface. Claim 9 An imaging system according to claim 6, wherein the radius of curvature of the convex outer surface is substantially the same as the radius of curvature of the concave surface. Claim 10 An imaging system according to claim 6, wherein the radius of curvature of the convex outer surface of the object lens is substantially the same as the radius of curvature of the concave surface of the protective layer. Claim 11 An imaging system according to claim 10, wherein the protective layer is disposed on the object lens forming a common interface. Claim 12 An imaging system according to claim 6, wherein the radius of curvature of the convex outer surface of the object lens facing the scene is greater than the radius of curvature of the concave surface of the protective layer facing the object lens. Claim 13 An imaging system according to claim 12, wherein the edge region of the protective layer is connected to or bonded to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer. Claim 14 An imaging system according to claim 1, further comprising a spacer connecting the edge region of the protective layer to the edge region of the object lens to form a gap between at least the center region of the object lens and the center region of the protective layer. Claim 15 An imaging system according to claim 13 or 14, wherein the maximum size of the gap along a direction perpendicular to the main surface of the protective layer is greater than 0.05 millimeters. Claim 16 An imaging system according to claim 13 or 14, further comprising an opening connecting the volume between the object lens and the protective layer in an enclosure or surrounding environment containing the lens group. Claim 17 An imaging system according to claim 1, wherein the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters. Claim 18 An imaging system according to claim 17, wherein the thickness of the protective layer is less than 2 millimeters. Claim 19 An imaging system according to claim 1, further comprising a transparent heating layer disposed between the object lens and the protective layer, wherein the heating layer is configured to provide heat to at least the protective layer. Claim 20 An imaging system according to claim 19, wherein the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system. Claim 21 An imaging system according to claim 19, wherein the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system. Claim 22 An imaging system according to claim 21, wherein the heating layer comprises indium tin oxide. Claim 23 An imaging system according to claim 22, wherein the thickness of the heating layer is 10 nm to 1000 nm. Claim 24 An imaging system according to claim 19, wherein the heating layer is disposed on the inner concave surface of the protective layer facing the object lens. Claim 25 An imaging system according to claim 19, wherein the heating layer is electrically connected to a power supply to receive current. Claim 26 An imaging system according to claim 25, wherein the heating layer is configured to generate heat in response to receiving the current. Claim 27 An imaging system according to claim 1, wherein the protective layer has a light transmittance greater than 95% within the operating wavelength range of the imaging system. Claim 28 An imaging system according to claim 1, further comprising an image sensor, wherein the imaging system is configured to generate an image of the scene on the image sensor. Claim 29 An imaging system according to claim 21 or 27, wherein the operating wavelength range of the imaging system is within the visible light wavelength range. Claim 30 An imaging system according to claim 21 or 27, wherein the operating wavelength range of the imaging system is within the MWIR (mid-wave infrared) or LWIR (long-wave infrared) wavelength range. Claim 31 An imaging system comprising: an object lens configured to capture a ray of light received from a scene within the field of view (FOV) of the imaging system; a group of lenses configured to receive light from the object lens and form an image on an image sensor; a curved protective layer separating the scene from the object lens; and a transparent heating layer disposed between the protective layer and the object lens, wherein the transparent heating layer is configured to provide heat to at least the protective layer. Claim 32 An imaging system according to claim 31, wherein the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters. Claim 33 An imaging system according to claim 32, wherein the thickness of the protective layer is less than 2 millimeters. Claim 34 An imaging system according to claim 31, wherein the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system. Claim 35 An imaging system according to claim 31, wherein the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system. Claim 36 An imaging system according to claim 35, wherein the heating layer comprises indium tin oxide. Claim 37 An imaging system according to claim 31, wherein the thickness of the heating layer is 10 nm to 1000 nm. Claim 38 An imaging system according to claim 31, wherein the heating layer is disposed on the inner concave surface of the protective layer facing the convex surface of the object lens. Claim 39 An imaging system according to claim 38, wherein the radius of curvature of the concave surface of the protective layer is substantially the same as the radius of curvature of the convex surface of the object lens. Claim 40 An imaging system according to claim 38, wherein at least the central region of the heating layer is separated from the object lens by a gap from the convex surface of the object lens. Claim 41 An imaging system according to claim 40, wherein the radius of curvature of the concave surface of the protective layer is smaller than the radius of curvature of the convex surface of the object lens. Claim 42 An imaging system according to claim 31, wherein the heating layer is electrically connected to a power supply to receive current. Claim 43 An imaging system according to claim 31, wherein the protective layer has a light transmittance greater than 95% within the operating wavelength range of the imaging system. Claim 44 An imaging system according to claim 35 or 43, wherein the operating wavelength range of the imaging system is within the visible light wavelength range. Claim 45 An imaging system according to claim 35 or 43, wherein the operating wavelength range of the imaging system is within the MWIR (mid-wave infrared) or LWIR (long-wave infrared) wavelength range. Claim 46 An imaging system according to claim 31, wherein the object lens comprises a wide-angle lens or a fisheye lens. Claim 47 An imaging system according to claim 31, wherein the FOV is greater than 150 degrees. Claim 48 An imaging system according to claim 31, wherein the focal length of the object lens is less than 16 millimeters. Claim 49 An imaging system according to claim 31, wherein the protective layer comprises sapphire. Claim 50 An imaging system comprising: an object lens configured to capture a light beam received from a scene within the field of view (FOV) of the imaging system; a group of lenses configured to receive light from the object lens and form an image on an image sensor; and a curved protective layer separating the scene from the object lens, wherein the edge regions of the object layer and the protective layer are connected, and the central portion of the protective layer is separated from the central portion of the object by a gap. Claim 51 An imaging system according to claim 49, wherein the edge region of the protective layer is connected to or bonded to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer. Claim 52 An imaging system according to claim 50, further comprising a spacer connecting the edge region of the protective layer to the edge region of the object lens to form a gap between the center region of the object lens and the center region of the protective layer. Claim 53 An imaging system according to claim 50 or 51, wherein the maximum size of the gap along a direction perpendicular to the main surface of the protective layer is greater than 0.05 millimeters. Claim 54 An imaging system according to claim 49, wherein the object lens comprises a wide-angle lens or a fisheye lens. Claim 55 An imaging system according to claim 49, wherein the FOV is greater than 150 degrees. Claim 56 An imaging system according to claim 49, wherein the focal length of the object lens is less than 16 millimeters. Claim 57 An imaging system according to claim 49, further comprising a transparent heating layer disposed between the object lens and the protective layer, wherein the heating layer is configured to provide heat to at least the protective layer. Claim 58 An imaging system according to claim 56, wherein the heating layer is electrically conductive and transparent within the operating wavelength range of the imaging system. Claim 59 An imaging system according to claim 57, wherein the thickness of the heating layer is 10 nm to 1000 nm. Claim 60 An imaging system according to claim 57, wherein the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the imaging system. Claim 61 An imaging system according to claim 56, wherein the heating layer comprises indium tin oxide. Claim 62 An imaging system according to claim 56, wherein the heating layer is disposed on the inner concave surface of the protective layer facing the object lens. Claim 63 An imaging system according to claim 56, wherein the heating layer is electrically connected to a power supply to receive current. Claim 64 A method for manufacturing a protected object lens, comprising: forming a protective layer comprising a first material; providing an object lens comprising a second material; forming a transparent heating layer on the surface of the protective layer; and connecting the protective layer to the object lens such that the heating layer faces the object lens. Claim 65 A method for manufacturing a protected object lens according to claim 64, wherein the step of forming a transparent heating layer on the surface of the protective layer comprises the step of forming the heating layer using a physical film formation method. Claim 66 A method for manufacturing a protected object lens according to claim 65, wherein the physical film formation method comprises sputtering or evaporation. Claim 67 A method for manufacturing a protected object lens according to claim 64, wherein the heating layer comprises indium tin oxide. Claim 68 A method for manufacturing a protected object lens according to claim 64, wherein the difference in hardness between the first material and the second material is greater than 500 KHN. Claim 69 A method for manufacturing a protected object lens according to claim 64, wherein the protective layer comprises sapphire. Claim 70 A method for manufacturing a protected object lens according to claim 64, wherein the thickness of the protective layer along a direction perpendicular to the main surface of the protective layer is greater than 0.5 millimeters. Claim 71 A method for manufacturing a protected object lens according to claim 70, wherein the thickness of the protective layer is less than 2 millimeters. Claim 72 A method for manufacturing a protected object lens according to claim 64, wherein the thickness of the heating layer is 10 nm to 1000 nm. Claim 73 A method for manufacturing a protected object lens according to claim 64, wherein the heating layer is electrically conductive and transparent within the operating wavelength range of the protected object lens. Claim 74 A method for manufacturing a protected object lens according to claim 64, wherein the heating layer is configured to provide a light transmittance greater than 90% within the operating wavelength range of the protected object lens. Claim 75 A method for manufacturing a protected object lens according to claim 64, wherein the step of connecting a protective layer to the object lens includes the step of bonding the protective layer to the object lens. Claim 76 A method for manufacturing a protected object lens according to claim 75, wherein the step of connecting a protective layer to the object lens comprises the step of forming a common interface between the protective layer and the object lens near a central region of the protective layer for the object lens. Claim 77 A method for manufacturing a protected object lens according to claim 64, wherein the step of connecting a protective layer to the object lens comprises the step of connecting an edge region of the protective layer to an edge region of the object lens to form a gap between a center region of the object lens and a center region of the protective layer. Claim 78 A method for manufacturing a protected object lens according to claim 77, wherein the radius of curvature of the protective layer is smaller than the radius of curvature of the object lens.

Images