Method and system for long distance sign or surface identification using LIDAR

Abstract

A method of operating a distance sensor includes transmitting light pulses at one or more specified wavelengths in an invisible spectrum from a transmitter on the distance sensor, and receiving a set of reflected light pulses with a receiver on the distance sensor, the set of reflected light pulses corresponds to light pulses that are reflected away from a surface of an object within a line of sight of the receiver. A point cloud of a region is generated based on the set of reflected light pulses, the region including a surface of the object. A pattern on the surface of the object is identified based on at least one of the areas identified in the point cloud where the reflectance of the one or more specified wavelengths in the invisible spectrum increases or decreases. The embedded data is decoded from the pattern on the surface of the object.

Description

[0001] introduce

[0002] Light detection and ranging (LIDAR) is used in autonomous driving systems (ADS), advanced driver assistance systems (ADAS), and other sophisticated transportation structures for sensing nearby objects. LIDAR is used in addition to cameras or radio radar because of its ability to generate high-resolution geometry of the vehicle's surroundings under a range of conditions. Summary of the Invention

[0003] This document discloses a method for operating a distance sensor. The method includes: emitting light pulses of one or more specified wavelengths in an invisible spectrum from a transmitter on the distance sensor; and receiving a set of reflected light pulses using a receiver on the distance sensor, the set of reflected light pulses corresponding to light pulses reflected away from the surface of an object within the line of sight of the receiver. A point cloud comprising a region of the object's surface is generated based on the set of reflected light pulses. A pattern on the object's surface is identified based on at least one region in the point cloud where reflectance increases or decreases to the one or more specified wavelengths in the invisible spectrum. Embedded data is decoded from the pattern on the object's surface.

[0004] In another aspect of this disclosure, the distance sensor includes a LiDAR.

[0005] In another aspect of this disclosure, the distance sensor includes a SWIR camera.

[0006] In another aspect of this disclosure, the set of reflected light pulses includes multiple sets of reflected light pulses, which are accumulated and aligned to generate spatiotemporal point cloud fusion.

[0007] In another aspect of this disclosure, the embedded data or portions thereof are associated with information identified in a cloud storage location accessible via a network.

[0008] In another aspect of this disclosure, information is transmitted from the cloud storage location to the driver using one or more of a display screen, head-up display, augmented reality (AR) device, or a speaker that uses acoustic data.

[0009] In another aspect of this disclosure, the embedded data further includes an error detection and correction code (EDC).

[0010] In another aspect of this disclosure, the embedded data corresponds to at least one of road infrastructure or building, sign, structure, vest, bridge or other infrastructure.

[0011] In another aspect of this disclosure, the embedded data corresponds to at least one of the vehicle type or vest type.

[0012] In another aspect of this disclosure, the embedded data or portions thereof are associated in real time using information identified in a lookup table stored on the vehicle.

[0013] In another aspect of this disclosure, the pattern is generated on the surface of the object by at least one of chemical treatment or a retroreflector.

[0014] A system includes: a distance sensor having a transmitter and a receiver; and a controller in communication with the distance sensor. The controller is configured to: instruct the transmitter to emit light pulses at one or more specified wavelengths in an invisible spectrum; and instruct the receiver to receive a set of reflected light pulses corresponding to light pulses reflected away from the surface of an object within the receiver's line of sight. The controller is configured to: generate a point cloud comprising a region of the object's surface based on the set of reflected light pulses; and identify a pattern on the object's surface based on at least one region identified in the point cloud that shows an increase or decrease in reflectivity to the one or more specified wavelengths in the invisible spectrum.

[0015] In another aspect of this disclosure, the controller is configured to decode embedded data from the pattern on the surface of the object based on information from at least one of a lookup table or a cloud storage location accessible via a network.

[0016] In another aspect of this disclosure, the information includes at least one of color or text on the object.

[0017] In another aspect of this disclosure, the set of reflected light pulses includes multiple sets of reflected light pulses, and the controller is configured to accumulate and align the multiple sets of reflected light pulses to generate spatiotemporal point cloud fusion.

[0018] In another aspect of this disclosure, the pattern corresponds to at least one of road infrastructure or building, sign, structure, vest, bridge or other infrastructure.

[0019] In another aspect of this disclosure, the pattern corresponds to at least one of the vehicle type or vest type.

[0020] In another aspect of this disclosure, the pattern is generated on the surface of the object by at least one of chemical treatment or a retroreflector.

[0021] This document discloses a vehicle. The vehicle includes: a body defining a passenger compartment and supported by wheels; a distance sensor having a transmitter and a receiver; and a controller communicating with the distance sensor. The controller is configured to: instruct the transmitter to emit light pulses at one or more specified wavelengths in an invisible spectrum; and instruct the receiver to receive a set of reflected light pulses corresponding to light pulses reflected away from the surface of an object within the receiver's line of sight. The controller is configured to: generate a point cloud comprising a region of the object's surface based on the set of reflected light pulses; and identify a pattern on the object's surface based on at least one region identified in the point cloud that shows an increase or decrease in reflectivity to the one or more specified wavelengths in the invisible spectrum.

[0022] In another aspect of this disclosure, the controller is configured to decode embedded data from the pattern on the surface of the object.

[0023] This disclosure provides the following examples:

[0024] Example 1. A method for operating a distance sensor, the method comprising:

[0025] The transmitter on the distance sensor emits light pulses of one or more specified wavelengths in the invisible spectrum;

[0026] The distance sensor receives a set of reflected light pulses, which correspond to light pulses reflected from the surface of an object within the receiver's line of sight.

[0027] A point cloud is generated based on the set of reflected light pulses, including the area of ​​the object's surface;

[0028] A pattern on the surface of the object is identified based on at least one of the regions in the point cloud where reflectance increases or decreases to one or more specified wavelengths in the invisible spectrum; and

[0029] Decode the embedded data from the pattern on the surface of the object.

[0030] Example 2. The method according to Example 1, wherein the distance sensor includes a LIDAR.

[0031] Example 3. The method according to Example 1, wherein the distance sensor includes a SWIR camera.

[0032] Example 4. According to the method of Example 1, wherein the set of reflected light pulses comprises multiple sets of reflected light pulses, the multiple sets of reflected light pulses being accumulated and aligned to generate spatiotemporal point cloud fusion.

[0033] Example 5. The method according to Example 1 includes using information identified in a cloud storage location accessible via a network to associate the embedded data or a portion thereof.

[0034] Example 6. The method according to Example 5 includes using one or more of a display screen, head-up display, augmented reality (AR) device, or a speaker using acoustic data to transmit information from the cloud storage location to the driver.

[0035] Example 7. According to the method of Example 1, the embedded data further includes an error detection and correction code (EDC).

[0036] Example 8. The method according to Example 1, wherein the embedded data corresponds to at least one of road infrastructure or building, sign, building, vest, bridge or other infrastructure.

[0037] Example 9. The method according to Example 1, wherein the embedded data corresponds to at least one of the vehicle type or vest.

[0038] Example 10. The method according to Example 1 includes using information identified in a lookup table stored in the vehicle to associate the embedded data or a portion thereof in real time.

[0039] Example 11. The method according to Example 1, wherein the pattern is generated on the surface of the object by at least one of chemical treatment or a retroreflector.

[0040] Example 12. A system comprising:

[0041] A distance sensor with a transmitter and a receiver;

[0042] A controller that communicates with the distance sensor, the controller being configured to:

[0043] The transmitter is instructed to emit light pulses at one or more specified wavelengths in the invisible spectrum;

[0044] The receiver is instructed to receive a set of reflected light pulses, the set of reflected light pulses corresponding to light pulses reflected from the surface of an object within the receiver's line of sight;

[0045] A point cloud is generated based on the set of reflected light pulses, including a region encompassing the surface of the object; and

[0046] The pattern on the surface of the object is identified based on at least one of the regions in the point cloud where the reflectance increases or decreases for one or more specified wavelengths in the invisible spectrum.

[0047] Example 13. The system according to Example 12, wherein the controller is configured to decode embedded data from the pattern on the surface of the object based on information from at least one of a lookup table or a cloud storage location accessible via a network.

[0048] Example 14. The system according to Example 13, wherein the information includes at least one of color or text on the object.

[0049] Example 15. The system according to Example 12, wherein the set of reflected light pulses comprises multiple sets of reflected light pulses, and the controller is configured to accumulate and align the multiple sets of reflected light pulses to generate spatiotemporal point cloud fusion.

[0050] Example 16. The system according to Example 12, wherein the pattern corresponds to at least one of road infrastructure or building, sign, building, vest, bridge or other infrastructure.

[0051] Example 17. The system according to Example 12, wherein the pattern corresponds to at least one of the vehicle type or vest.

[0052] Example 18. The system according to Example 12, wherein the pattern is generated on the surface of the object by at least one of chemical treatment or a retroreflector.

[0053] Example 19. A vehicle comprising:

[0054] The vehicle body defines the passenger compartment and is supported by multiple wheels;

[0055] A distance sensor with a transmitter and a receiver;

[0056] A controller that communicates with the distance sensor, the controller being configured to:

[0057] The transmitter is instructed to emit light pulses at one or more wavelengths in the invisible spectrum;

[0058] The receiver is instructed to receive a set of reflected light pulses, the set of reflected light pulses corresponding to light pulses reflected from the surface of an object within the receiver's line of sight;

[0059] A point cloud is generated based on the set of reflected light pulses, including a region encompassing the surface of the object; and

[0060] The pattern on the surface of the object is identified based on at least one of the regions in the point cloud where the reflectance to one or more wavelengths of the invisible spectrum increases or decreases.

[0061] Example 20. The vehicle according to Example 19, wherein the controller is configured to decode embedded data from the pattern on the surface of the object.

[0062] The foregoing summary is not intended to represent every embodiment or aspect of this disclosure. Rather, it provides only examples of some novel concepts and features set forth herein. The foregoing features and advantages, as well as other features and accompanying advantages, will become apparent from the following detailed description of illustrative examples and representative modes for implementing this disclosure when understood in conjunction with the accompanying drawings and appended claims. Furthermore, this disclosure explicitly includes various combinations and sub-combinations of the elements and features presented above and below. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with the description, explain the principles of the present disclosure.

[0064] Figure 1 The diagram illustrates a vehicle and battery system in which the principles of this disclosure can be implemented, the battery system being coupled to an electronic control unit (ECU) and a power inverter module (PIM).

[0065] Figure 2 This is a conceptual diagram of a technology used to embed digital data encoded in the infrared spectrum into street signs with visible messages using chemical coatings and selective retroreflectors to produce signs with visible messages along with invisible digital data encoded therein.

[0066] Figure 3 It is a diagram of a sign with an angle, which is used to embed a micro QR code on the surface of an object such as a traffic sign, based at least in part on specifications from the Uniform Traffic Control Devices Manual (MUTDC).

[0067] Figure 4 This is an example of selective retroreflection, a technique used to encode digital data on a surface such that light reflecting off a point follows the same direction of reflection as the incident light.

[0068] Figure 5 This is an example of selectively embedding digital data into a vest-like object on a rear surface according to an embodiment.

[0069] Figure 6 It is a context flowchart that identifies the methods used to create digitally encoded surfaces and the regions or layers in which the methods are performed.

[0070] The accompanying drawings are not necessarily drawn to scale and may present simplified representations of various features of the present disclosure as disclosed herein, including, for example, specific dimensions, orientations, positions, shapes, and scales. Details associated with such features will be determined in part by the specific intended application and usage environment. Detailed Implementation

[0071] This disclosure allows for many different forms of embodiments. Representative examples of this disclosure are shown in the accompanying drawings and are described in detail herein as non-limiting examples of the disclosed principles. Therefore, elements and limitations described in the abstract, introduction, summary, and detailed description sections but not expressly set forth in the claims should not be incorporated into the claims individually or collectively by implication, inference, or otherwise.

[0072] This disclosure describes systems, vehicles, and methods for identifying readable information embedded in signs or the surfaces of other objects, such as road markings, other road users (e.g., construction workers), commercially based signs (e.g., restaurant signs near the carriageway), etc., using distance sensors (such as LiDAR or SWIR). The embedded information can be read by distance sensors (such as LiDAR or SWIR) that operate in the invisible spectrum. While the principles of this disclosure apply to such information embedded in the surfaces of many types of objects used by vehicles, on, above, or otherwise adjacent to a carriageway, for simplicity, embodiments of this disclosure often use signs (such as road signs) associated with examples of them. However, this disclosure applies to areas other than road signs, such as clothing or other vehicles.

[0073] The coating of each sign is enhanced to allow specific geometric areas of the sign to reflect specific wavelengths of light, such as wavelengths of light outside the visible spectrum. These geometric areas can be used to embed digital information, which is detected and decoded by one or more distance sensors on the vehicle or other transport structure to extract information beyond the merely visible text of the sign; such as the type and purpose of the sign, and details that associate the visible text with other data. This information can be transmitted to the vehicle's user or driver by means of one or more microcontroller units (MCUs) in the vehicle (often referred to as a processing system, which may have one or more cores, cache memory, and dynamic memory or other types of memory such as read-only memory).

[0074] In one configuration, one or more distance sensors (such as LIDAR or SWIR) are fixed to the vehicle, for example, via a frame or other location. When in line of sight (LOS) of a sign or another object (e.g., lane markings embedded to demarcate lanes on a road, a police officer's vest, etc.), the LIDAR and SWIR sensors on the vehicle are able to capture additional information besides the text on the sign (i.e., the driver's reading of the sign in the visible spectrum (e.g., "Slow Down") or the vehicle's capture of the text on the sign using one or more visible wavelength cameras integrated into the vehicle). The digital information encoded in the sign is in one or more infrared wavelengths or within a certain range of infrared wavelengths, and therefore the encoded digital information is advantageously invisible to the driver, thus avoiding clutter. In some examples, the digital information includes error detection and correction data (EDC), allowing the processing system in the vehicle to appropriately encode the information obtained from the LIDAR and SWIR sensors mounted thereon. Different methods for transmitting digital information are used on the surface of an object in the geometric area. While one such example is a standard QR code, many other information formats can be used. Infrared wavelengths can be within one or more wavelengths or a range of wavelengths, and the distance sensors for a corresponding vehicle can be tuned to those wavelengths so that they can successfully extract the embedded digital data. In one example, the embedding is formed by writing LiDAR-readable code using a specific chemical coating (such as a multilayer ZnS / Ge for wavelength-selective emission). The chemical coating can be used such that at a given point on the surface of a sign or object, the coating can increase or decrease the reflectivity of the wave emitted by the distance sensor at a specific wavelength. When the distance sensor emits a light pulse in the applicable infrared spectrum, the distance sensor can use the reflected infrared light and its increase / decrease relative to a baseline to identify the geometric area with the embedded information.

[0075] In other configurations, besides using the chemical coatings described above to embed digital data, the following methods can also be used: Figure 4The wavelength-selective retroreflector under discussion. A characteristic of a selective retroreflector is that infrared light emitted by a distance sensor, when received at the surface of the marker, is reflected back to the distance sensor at an angle matching the angle of incidence. The result is that when the marker is enhanced using a retroreflector, the distance sensor may be able to see the reflected infrared light from a much greater distance because the infrared light will be reflected on an axis parallel to the axis from which the infrared light arrives at the distance sensor. In short, by enabling the distance sensor to detect reflected light from a greater distance, the reading of digital data by the sensor (or by one or more MCUs / processing systems coupled to the sensor) can be made much more accurate. Therefore, when decoding, the digital code embedded on the surface of the marker / object can provide accuracy close to a reference level of truth, similar to the accuracy of a reference marker for a camera.

[0076] Various embodiments of systems and methods for enhancing existing signs, road users, and carriageway infrastructure using specific information that can be accurately read by distance sensors are disclosed, as well as various embodiments of similar systems that do not alter the appearance of the sign's surface in the visible light band. The methods enable distance sensors mounted on or fixed to vehicles to capture additional environmental information, including the meaning of the signs, and to classify other road users. The methods can employ specific data codes to enhance traffic and information signs, vehicles, and vests for distance sensor identification. Some configurations further include error detection and correction data embedded in the codes to ensure accurate data extraction. As described above, wavelength-selective retroreflectors or chemical coatings can be used to write distance sensor-readable codes to embed the information. The coatings can increase or decrease the retroreflectance at specific wavelengths.

[0077] While the principles of this disclosure are broadly applicable to various architectures involving infrared reading and decoding of digital information embedded on the surface of an object, electric vehicles (EVs) are considered for illustrative purposes. However, it should be understood that distance sensors can be mounted on other types of transport structures (such as, for example, snowplows, garbage trucks, delivery trucks, low-flying aircraft, etc.). In one such configuration, Figure 1 This is a plan view illustration of a vehicle and battery system in which the principles of this disclosure can be implemented, the battery system being coupled to an electronic control unit (ECU) and a power inverter module (PIM). Figure 1 In this embodiment, the ECU performs calibration within the vehicle. Advantageously, whenever or even upon power-on, if it is determined that the calibration has deteriorated for whatever reason, the ECU can apply the principles of this disclosure to recalibrate each camera on the vehicle, including distance sensors 119 mounted on the vehicle body, rear of the vehicle, or elsewhere. In other words, calibration can be performed on the vehicle using the described in-vehicle electronics.

[0078] Although Figure 1 An electric vehicle is shown, but it will be appreciated that this disclosure is not limited thereto, and the internal calibration procedure can be performed by each vehicle with appropriately programmed circuitry. While the hysteresis model above can be applied to many different physical configurations, Figure 1 An example of this is shown. Figure 1 Depicting a high-voltage battery pack (B HV The electrified power system 110 of the high-voltage battery pack 112 can perform SOC estimation for this high-voltage battery pack 112. In a non-limiting example, the battery pack 112 may embody a high-capacity battery with a voltage capability of approximately 400-800 volts or higher, wherein the actual voltage capability of the battery pack 112 is provided based on the expected operating / SOC range of the load connected to the battery pack 112, the total weight, and the rated power. In possible configurations, the battery pack 112 may be a propulsion battery pack, which generally consists of an array of lithium-ion or lithium-ion polymer rechargeable electrochemical battery cells, which may be cylindrical battery cells. In configuration, the present teachings are also applicable to prismatic battery cells, as well as pouch battery cells, and therefore cylindrical battery cells are exemplary and not limiting.

[0079] Although internal details of the battery cells in battery pack 112 are omitted for simplicity, those skilled in the art will recognize that the battery cells contain electrolyte material, working electrodes in the form of cathodes and anodes, and permeable separators (not shown) within their cell cavities, all encapsulated within an electrically insulating canister or housing. Groups of battery cells can be connected in series or parallel using electrical interconnects and associated buses, sensing hardware, and power electronics (not shown but well known in the art). A specific number of battery cells in battery pack 112 can be arranged in columns and rows relative to battery tray 113. For example, in a nominal “xyz” Cartesian reference system, battery tray 113 may have a length (x-dimension) and width (y-direction) when viewed from above or below, and a height (z-dimension) extending in orthogonal directions away from battery tray 113.

[0080] In a typical application scenario, the electrified power system 110 can be used as part of an EV 111 or another mobility system. As shown, the EV 111 can be embodied as a battery electric vehicle, wherein this teaching can also be extended to plug-in hybrid electric vehicles. Alternatively, the electrified power system 110 can be used as part of another mobility system (such as, but not limited to, rail vehicles, aircraft, ships, robots, farm equipment, etc.). Similarly, the electrified power system 110 can be stationary, such as in the case of a power unit, crane, belt drive, or conveyor system. Therefore, Figure 1The electrified powertrain 110 in the representative vehicle embodiment is intended to illustrate this teaching and not to limit it.

[0081] Figure 1 The EV 111 shown includes a vehicle body 122. The vehicle body 122 may include a frame within the body 122 for defining areas for housing mechanical and electrical components and a passenger compartment. The EV may further include wheels 124F and 124R, where “F” and “R” indicate corresponding front and rear positions. Wheels 124F and 124R rotate about corresponding axles 125 and 150, wherein wheels 124F, 124R, or both are powered by a rotary motor (M) from the electrified powertrain 110. E )126 output torque (arrow T) O Power is provided, as indicated by arrow

[24] . Thus, in this embodiment, wheels 124F and 124R represent mechanical loads, where other possible mechanical loads are possible in different host systems. For this purpose, the electrified power system 110 includes a power inverter module (PIM) 128 (also referred to herein as a power module (PM)) and a high-voltage battery pack 112 (e.g., a multi-cell lithium-ion propulsion battery or a battery with another suitable chemistry for the application), both arranged on a high-voltage DC bus 127. As is known in the art, the PIM 128 includes a DC side (180) and an AC side 120, with the AC side 120 connected to a single-phase winding (not shown) of the rotary motor 126 when the rotary motor 126 is configured as a multiphase rotary motor in the form of a propulsion or traction motor as shown.

[0082] Figure 1 The battery pack 112 is then connected to the DC side 180 of the PIM 128, such that during the propulsion mode of the EV 111, the battery voltage from the battery pack 112 is supplied to the power inverter module (PIM) 128. The PIM 128, or more precisely, a set of semiconductor switches (not shown) residing therein, is controlled via pulse width modulation (PWM), pulse density modulation (PDM), or other suitable switching control techniques to invert the DC input voltage on the DC bus 127 to an AC output voltage suitable for powering the high-voltage AC bus 120. As mentioned above, the PIM 128 can also be simply referred to as a power module (PM), which may include an inverter or converter. In other embodiments, high-speed switching of the residing semiconductor switches of the PIM 128 powers the rotary motor 126, causing the rotary motor 126 to output torque (arrow T). O As a motor drive torque, it is delivered to one or more of the wheels 124F and / or 124R in another coupled mechanical load.

[0083] The electrical components of the electrified power system 110 may also include an accessory power module (APM) 129 and an auxiliary battery (B). AUX 130. As known in the art, APM 129 is configured as a DC-DC converter connected to DC bus 127. In operation, APM 129 is capable of reducing the voltage level on DC bus 127 to a lower level suitable for charging auxiliary battery 130 and / or supplying low-voltage power to one or more accessories (not shown), such as lamps, displays, etc., via internal switching and voltage transformation. Therefore, "high voltage" refers to a voltage level far exceeding the typical 12-15V low / auxiliary voltage level, where 400V or higher is an exemplary high voltage level in some embodiments of battery pack 112.

[0084] In some configurations, Figure 1 The electrified powertrain 110 may include an on-board charger (OBC) 132, which is selectively connected to an off-board charging station 133 via an input / output (I / O) block 132A during charging mode, during which the battery pack 112 is charged by an AC charging voltage (V) from the off-board charging station 133. CH Recharging. I / O block 132 can be connected to charging port 117 on vehicle body 122. For example, charging cable 135 can be connected to charging port 117, for example, via SAE J1772 connection. As is understood in the art, in one or more embodiments, electrified powertrain 110 can also be configured to selectively receive DC charging voltage, in which case OBC 132 will be selectively bypassed using circuitry (not shown), for example, circuitry that can be used to gradually charge and / or discharge battery pack 112 to perform various functions (such as testing SOC). OBC 132 can also operate in different modes, including: a charging mode, during which OBC 132 receives AC charging voltage (V) from off-board charging station 133. CH The battery pack 112 will recharge after the low battery indicator light is displayed on the dashboard; and in discharge mode, indicated by arrow V. X This indicates that during discharge mode, OBC 132 offloads power from battery pack 112 to an external AC electrical load (L). In this way, OBC 132 can function as a bidirectional charger.

[0085] Still referencing Figure 1 The electrified power system 110 may also include an electronic control unit (ECU) 134. The ECU 134 is operable via electronic control signals (arrow CC). O The ECU 134 responds to the transmission of electronic input signals (arrow CC) to regulate the ongoing operation of the electrified powertrain 110.I And so it is done. In different embodiments, this input signal (arrow CC) I The signal can be actively transmitted or passively detected, enabling the ECU 134 to operate to determine a specific operating mode. In response, the ECU 134 controls the operation of the electrified powertrain 110. Therefore, the ECU and its associated components can act as a BMS to perform functions including estimating the state of charge (SOC).

[0086] To this end, ECU 134 may be equipped with one or more processors (P) (e.g., logic circuitry, combinational logic circuitry, application-specific integrated circuits (ASICs), electronic circuitry, central processing units, semiconductor IC devices, etc.) and one or more input / output (I / O) circuitry, appropriate signal conditioning and buffering circuitry, and other components such as a high-speed clock, to provide the SOC functions described in the previous figures and the various functions identified by the CC input signal. ECU 134 also includes associated computer-readable storage media, i.e., memory (M) containing read-only, programmable read-only, random access, hard disk, etc., whether resident, remote, or a combination of both. Control routines (including code for executing a SOC model with hysteresis) are executed by the processor to monitor relevant inputs from sensing devices and other networked control modules (not shown) and to execute control and diagnostic routines to manage the operation of the electrified powertrain 110. I / O circuitry may be directly coupled to ECU 134 along with the memory M and one or more processors P to execute code for estimating the SOC. In one aspect, the BMS system may be implemented collectively as ECU 134, OBC 132, and bus 127. OBC 132 and bus 127 may be devices within the BMS or included as part of the BMS, said parts being capable of connecting to external terminals of the battery pack 112 to perform the functions described herein. In some embodiments, the BMS may be directly coupled to the battery pack.

[0087] Similar to other vehicles, the EV 111 may include an instrument panel embedded within or otherwise connected to the EV 111's body. The body houses the driver and occupant compartment. The aforementioned devices may include control signals to the instrument panel and switching circuitry, enabling the driver to assess the remaining State of Charge (SOC) based on the amount or percentage of remaining battery power, an estimated time when the vehicle will stop operating or urgently require recharging, and other data. At least some of these aspects may be calculated by the BMS (including the ECU 134 running code from memory M and its associated processor P). Messages may be transmitted to other parts of the vehicle via I / O circuitry, via CC0, or another connection not specifically shown.

[0088] In another embodiment, ECU 134, along with its I / O, memory, and processor, can be additionally or alternatively used to calibrate and recalibrate each or selected camera in the EV. In this case, a flowchart (such as flowchart 600) can run on the processor, and ECU 134 can be appropriately connected to perform calibration for each camera. This may occur during a suspected calibration error caused by a force event, or it can simply be recalibrated every time the driver starts the EV 111X times. It should be noted again that another type of combustion-based vehicle or hybrid vehicle can be used in this embodiment. This embodiment also eliminates the costly and time-consuming need to perform calibration independently for each vehicle at startup. ECU 134 can be coupled to each camera via hardwired or network connection, or it can be connected to selected cameras.

[0089] In the example above, PIM 128 (or more simply, PM) may include a set of semiconductor switches driven by a modulation technique such as PWM (although other suitable modulation techniques such as PDM may be used). In other configurations, an ECU or a microcontroller unit (MCU) therein (e.g., processor P) may also be used to manage the transmission of the modulated signal. The semiconductor switches of PIM 128 may include power transistors, and the modulation technique used to drive them may include intermediate circuitry for properly decoding the PWM signal and for adjusting the rail-to-rail voltage swing from the power used by the logic circuitry (e.g., 0 to 5 volts, etc.) to the higher voltage required by the gate driver for switching the power transistors driving the rotary motor 126. Referring to PIM 128, a gate driver may be employed to turn the power transistors / switches on and off.

[0090] Figure 2 This is a conceptual diagram of a technology used to embed digital data encoded in the infrared spectrum into street signs with visible messages using chemical coatings and selective retroreflectors, to produce signs with visible messages along with invisible digital data encoded therein. It should be noted initially that, for the purpose of writing the initial sign 205 for the exemplary purpose of “sightseeing,” the Department of Transportation has issued a series of specifications, including in the form of the Uniform Traffic Control Device Manual (MUTDC).

[0091] Reference Figure 2For illustrative purposes, a sign 205 with the text "Scenic Spot" can be created. A random quick-response code (QR code) 207 is further shown for conceptual purposes. To achieve the objectives of this disclosure, it may be desirable to include the QR code 207 (possibly along with other digital embedding codes) on the surface of sign 205. Using chemical coating techniques and optionally the selective retroreflector techniques described above, in this example, the QR code 207 is embedded in the surface of sign 209. While the QR code 207 is shown on sign 209 to indicate that the digitally embedded data is now available, in fact, the QR code 207 is etched at an infrared wavelength. Therefore, the sign "Scenic Spot" 205 with the corresponding arrow is unobstructed and easily read by the driver (and in some cases, by a visible camera or sensor fixed to the vehicle and coupled to a processing system), and a distance sensor can then use a tuned infrared wavelength to acquire the information in the QR code 207. In some cases, the data may be directly related to the visible content of the sign. Therefore, the vehicle's processing system (which may include an MCU, or one or more (or a network) of electronic control units (ECUs), each equipped with an MCU) can decode the data, and, if necessary, can transmit the embedded data to the vehicle's driver, for example, via a display screen, head-up display, augmented reality (AR) device, or display. The processing system can also acoustically transmit these details through speakers in the vehicle. The digital data can involve further details about the "scenic area," such as parking areas, danger zones, etc.

[0092] Figure 2 The principle represents a simple example of a wide range of applications that can be used in conjunction with the distance sensor capabilities of the vehicles or structures under discussion. A multitude of applications can be considered for transmitting data to processing systems on moving vehicles, regardless of whether the data is specifically related to visible information on other surfaces of signs or objects (e.g., the side of a bridge adjacent to a carriageway, the surface of a facility adjacent to a carriageway, etc.). In other cases, embedded infrared-based information can enable distance sensors on vehicles to capture additional environmental information (including the meaning of signs), classify other road users (e.g., construction workers, functional personnel, etc.), and perform other actions. In real-world applications, there may be different distance sensors, each pointing in the same, different, and / or overlapping directions. Different distance sensors on a vehicle may also be tuned to different wavelengths. For example, one distance sensor might use 900 nm while another uses 1300 nm. Signs may reflect at one wavelength or at both.

[0093] Figure 3It is a diagram of a sign with an angle α, which is used at least in part based on specifications from (MUTDC) to embed micro QR codes on the surface of objects such as traffic signs. When constructing visible signs (such as...) Figure 3 When designing a sign (with a length D), designers can consider various criteria, such as the minimum distance at which a driver of a moving vehicle can see the sign, the size of the letters and numbers on the sign as discussed above, the color of the sign, and other criteria. These criteria may include, for example, requirements and specifications in MUTCD, which can specify a minimum sign clarity distance of 180 to 250 feet. While the size range of warning signs can vary depending on the application and environment, for simplicity, a 36×36-inch sign is assumed here. Other criteria that may be relevant to the system design include the impact of rain and fog on LiDAR performance. Based on current recommendations, for a metal warning sign at highway speeds, it can be assumed that the same number of points exist on the target at 180 feet (horizontal distance) in rain at 40 mm / h (1.57 in / h) as at 250 feet in clear conditions. Furthermore, resolution varies depending on the distance sensor used. For example, LiDAR currently has a resolution of approximately 0.05 degrees and SWIR cameras currently have a resolution of approximately 0.015 degrees.

[0094] Reference Figure 3 An embodiment of the solution is disclosed, in which a micro QR code 207 is embedded in a traffic sign. The traffic sign may have a length D. In the example, D = 3 feet, and the traffic sign has a square shape factor of 3 feet × 3 feet. For simplicity, Figure 3 The configuration can include a detection distance of 250 feet, which could be the distance between the moving vehicles currently located at the leftmost edge of triangle 300. The symbol α can represent the degree that enables a moving vehicle at this position to perform detection. In one example, α = 0.67 degrees. Information about the sign type can be encoded onto the traffic sign using the rectangles that make up the sign. For example, using the MUTCD specification, the sign could be divided into thirteen 3-inch by 3-inch coded squares. Each coded square can use a different data format. For example, some coded squares can use QR codes. Other coded squares (or other signs) can use AprilTags, another system that uses unique descriptors to encode digital content. In this example, digital encoding is used because it allows for the reduction or elimination of errors through the use of, for example, EDCs.

[0095] Figure 4This is an example of selective retroreflection, a technique used to encode digital data on a surface such that light reflecting off a point follows the same direction of reflection as the incident light. As described above, selective retroreflection can be used to reflect infrared light of a specific wavelength or wavelength range while remaining invisible and not obscuring visible light from a sign or other object surface. Figure 4 In the diagram, pattern 400 represents a portion of a multilayer coating. Wavelength-selective reflective dielectric layers (or multiple layers) can be strategically coated onto existing retroreflectors, such as arrays of cornerstone prisms. The thickness of each layer can be varied to determine the wavelength(s) to be selectively reflected. An additional advantage of this technique is that selective retroreflection further allows reflected infrared light to be reflected along the same axis as the light was initially transmitted. This phenomenon increases the accuracy of code reading because the intensity reflected onto the distance sensor will be stronger.

[0096] Figure 5 This is an example of a vest object with digital data selectively embedded on its rear surface according to an embodiment. For example, a police officer's vest 506 could have the name "Police" 510. An example QR code 508 could be covered using a chemical coating, possibly with an additional selective reflector. The vest could also use highly visible colors to enhance the officer's visibility. These colors would not be obscured by the infrared digitally embedded information (e.g., QR code 508). Thus, if the officer is on an accident-prone lane, a distance sensor 119 fixed to an approaching vehicle can read the digitally embedded information, which is transmitted to a processing system (e.g., an MCU) for decoding and possibly to a user. The data on the vest could represent information about the officer's identity, role, or instructions to the driver to proceed through an obstacle ahead. In one example, a sign could be posted one mile before a road closure, informing vehicles that the left lane is currently available. This information could be advantageously integrated with GPS information or vehicle mapping information to update the vehicle's route.

[0097] Figure 6Context flowchart 600 identifies the methods used to create the digitally encoded surface and the areas, layers, or environments in which the steps of performing those methods are executed. Logic block 602 indicates that the steps in this block are cloud-based. That is, they are securely created using the processing and storage resources allocated to the tag manufacturer. The information is secure and encrypted. Cloud-based logic block 602 first identifies that the designer can use appropriate applications on the cloud to select the tag format definition. Examples already described include QR codes and Apriltag. However, current or future tag formats are also possible and equally suitable for the purposes of this disclosure. Thus, with the tag format definition in place, cloud-based applications can be used to create vehicle- and cloud-based static or dynamic mappings from tag values ​​to related data based on various factors. One such important factor is the location of the intended configuration (e.g., the United States). Therefore, during the tag creation process, the relevant DOT specifications can be consulted to identify various important data related to the QR tag or context (such as its identity, links to data (e.g., via the cloud or local), etc.).

[0098] Following the arrows from step 608 to step 610, the latter step resides in logic block 604, which represents the physical infrastructure, such as the application of a label to a sign. In this example, at step 610, a label is applied to the sign using at least one of a chemical coating or a selective reflector. The result is a sign 620a with an invisible overlay of a QR label. Thus, logic blocks 602 and 604 represent the initial phenomena that occur in order to properly embed information into the sign. In other embodiments, more complex signs with a more sophisticated label format may be used.

[0099] Logic block 606 illustrates the steps that occur within the vehicle when distance sensors 119, fixed to the vehicle, collect information from infrared pulses they emit and send that information to a processing system for decoding, error correction, and other procedures. In the described configuration, the vehicle is configured to travel along a roadway and approach sign 620a. Thus, the distance sensors within the line of sight of the surface of sign 620a acquire multiple point clouds and aggregate them to improve accuracy. At step 612, the distance sensors on the moving vehicle within the LOS of sign 620a generate a point cloud immediately after emitting an infrared pulse toward the sign. The point cloud is generated when the area (here, the surface of sign 620a) is scanned using laser pulses from the transmitter of the distance sensor and reflected light pulses within the relevant infrared wavelength(s) or range thereof are returned to the receiver on the distance sensor, thereby creating data at specific points in a Cartesian coordinate system. Because the distance sensors are propelled forward by the movement of the vehicle, the distance sensors generate a point cloud in step 612 that includes the surface of sign 620a, as shown in Figure 612a. The process continues at 614, where the distance sensor can advantageously utilize machine learning techniques to improve or increase the resolution of the scanned objects (including signs and tags). Thus, machine learning can be used to improve the resolution of the point cloud in a procedure called “spatiotemporal point cloud fusion,” as shown in Figure 614a. Here, as described above, multiple point clouds are acquired sequentially and then aligned temporally and spatially (e.g., by a processing system) to create a fused point cloud, which provides a guarantee of capturing as many tags as possible. Tags are then extracted from the final fused point cloud, as shown in Figures 616a and 620, and digital data is retrieved and decoded from the tags. In some embodiments, as indicated by the bidirectional arrow between steps 616 and 608, the decoded tags are used to access information in cloud-based logical block 602, such as via a cellular network, Wi-Fi network, proprietary wireless network, or other network connection. The tags can then be advantageously associated with corresponding meanings, a set of values, the color of the object, and text or other criteria associated with the object. In other configurations, the tag can be passed to another ECU in the vehicle, which resides a database, allowing designers to associate the tag with other data stored within the vehicle itself in real time, such as by using a lookup table of the tag and associated data (e.g., the object's color and the text associated with the object). Subsequently, depending on the implementation, the tag can use cloud-based data or vehicle data, such as in the case of ADS, to control another part of the vehicle. In other configurations, the information associated with the tag can be transmitted to the driver or occupants of the vehicle via a display or via a speaker.

[0100] The principles of this disclosure offer significant benefits in applications including autonomous vehicles and existing driven vehicles. In the former, driveways can be equipped with numerous tags embedded in various surfaces, from which digital tags are extracted and decoded. If necessary, the vehicle's processing system can access other sources to obtain additional details related to the tags or their meaning. These procedures can be executed in real time, enabling the embedded tags to control the vehicle's movement (including speed and braking). In vehicles involving active drivers, the benefits extend further to the invisibility of the embedded tags to the driver. This avoids clutter or data contamination, as the driver can clearly see the tags via standard colors in the visible spectrum, while the tag data is hidden in infrared wavelengths and therefore does not represent a distraction from the driver's attention.

[0101] It should be noted that while a microcontroller unit (MCU) may be an MCU integrated into a dedicated ECU or its network, for the purposes of this disclosure, the terms "MCU" and "processor" may refer to more than one processor. The term may refer to each controller or microcontroller used to perform the various tasks listed in this disclosure, including SR analysis and modification, current sharing, and dead zone reduction. In some cases, at least a portion of the processor may include dedicated hardware, such as in a digital signal processor (DSP). The processor may also be implemented (partially or entirely) using an application-specific integrated circuit (ASIC), a system-on-a-chip (SoC), combinational Boolean logic circuits performing the necessary digital functions, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or another type of programmable logic device (PLD). Transistors used in the processor may include complementary metal-oxide-semiconductor (CMOS) technology, bipolar junction transistors, gallium arsenide transistors, or combinations thereof. The processor may execute middleware, and in some embodiments, it may rely at least partially on one or more application programming interfaces to communicate with other systems. The processor may also include upgradable firmware. Memory may be logically partitioned to include databases or repositories, or relational or non-relational data tables.

[0102] A processor can be part of a processing system. A processing system can include memory (e.g., various levels of cache memory, dynamic random access memory (DRAM), static random access memory (SRAM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM or EPROM)), flash memory, magnetic-based hard disk drives, solid-state hard disk drives, and so on. Memory can include code and data stored therein. The overall architecture and functional structure of a processing system are designed to simplify the terminology and confirm that code can be executed using, for example, a processor located in a separate location or in several different ways (only a few of which are described above).

[0103] Detailed descriptions and accompanying drawings are provided to support and illustrate this teaching, but the scope of this teaching is defined solely by the claims. While some preferred modes and other embodiments for carrying out this teaching have been described in detail, various alternative designs and embodiments exist for practicing the teaching as defined in the appended claims. Furthermore, this disclosure expressly includes combinations and sub-combinations of the elements and features presented herein.

Claims

1. A method for operating a distance sensor, the method comprising: The transmitter on the distance sensor emits light pulses of one or more specified wavelengths in the invisible spectrum; The distance sensor receives a set of reflected light pulses, which correspond to light pulses reflected from the surface of an object within the receiver's line of sight. A point cloud is generated based on the set of reflected light pulses, including the area of ​​the object's surface; The pattern on the surface of the object is identified based on at least one of the regions in the point cloud in which the reflectance increases or decreases to one or more specified wavelengths in the invisible spectrum. as well as Decode the embedded data from the pattern on the surface of the object.

2. The method according to claim 1, wherein the distance sensor comprises a LIDAR.

3. The method according to claim 1, wherein the distance sensor comprises a SWIR camera.

4. The method according to claim 1, wherein the set of reflected light pulses comprises multiple sets of reflected light pulses, the multiple sets of reflected light pulses being accumulated and aligned to generate spatiotemporal point cloud fusion.

5. The method of claim 1, further comprising using information identified in a cloud storage location accessible via a network to associate the embedded data or a portion thereof.

6. The method of claim 5, comprising using one or more of a display screen, a head-up display, an augmented reality (AR) device, or a speaker using acoustic data to transmit information from the cloud storage location to the driver.

7. The method of claim 1, wherein the embedded data further comprises an error detection and correction code (EDC).

8. The method of claim 1, wherein the embedded data corresponds to at least one of road infrastructure or building, sign, building, vest, bridge or other infrastructure.

9. The method of claim 1, wherein the embedded data corresponds to at least one of the vehicle type or vest.

10. The method of claim 1, further comprising using information identified in a lookup table stored in the vehicle to associate the embedded data or a portion thereof in real time.

Images