Method for a vehicle, vehicle and storage medium

Abstract

The present disclosure provides methods, vehicles, and storage media for a vehicle. Some of the described methods also include receiving information associated with a first acoustic emission sensor, a second acoustic emission sensor, and a third acoustic emission sensor. In embodiments, the information corresponds to an event. From the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information, a first timestamp, a second timestamp, and a third timestamp, and a geometry of the surface, a unit vector is calculated in a direction of an origin of the event from each acoustic emission sensor. A parameter is calculated based on the unit vectors. Computer program products are also provided.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for a vehicle, a vehicle and a storage medium. BACKGROUND

[0002] A vehicle, such as an autonomous vehicle, is operated under various environmental conditions. Environmental conditions generally refer to the state of the environment. Environmental conditions generally describe temperature, wind, precipitation, and debris, among others. Environmental conditions can contaminate or otherwise degrade sensors and devices of the vehicle. SUMMARY

[0003] According to an aspect of the present application, there is provided a method for a vehicle, comprising: detecting, with at least one processor, vibrations generated from particles impacting a surface of the vehicle, wherein a first acoustic emission sensor is located at a first position on an arm of a wiper blade assembly, a second acoustic emission sensor is located at a second position on the arm, and a third acoustic emission sensor is located at a third position on the arm; receiving, with the at least one processor, first acoustic emission sensor information from the first acoustic emission sensor, second acoustic emission sensor information from the second acoustic emission sensor, and third acoustic emission sensor information from the third acoustic emission sensor corresponding to an event; receiving, with the at least one processor, a first timestamp associated with the first acoustic emission sensor information, a second timestamp associated with the second acoustic emission sensor information, and a third timestamp associated with the third acoustic emission sensor information; calculating, with the at least one processor, unit vectors from each acoustic emission sensor in a direction of origin of the event based on the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information, the first timestamp, the second timestamp, the third timestamp, and a geometry of the surface; calculating, with the at least one processor, a parameter associated with an environmental condition based on the unit vectors; and operating, with the at least one processor, the vehicle based on the parameter.

[0004] According to another aspect of the present application, there is provided a vehicle, comprising: at least three acoustic emission sensors configured to detect vibrations generated from particles impacting a surface of the vehicle, wherein a first acoustic emission sensor is located at a first position on an arm of a wiper blade assembly, a second acoustic emission sensor is located at a second position on the arm, and a third acoustic emission sensor is located at a third position on the arm; at least one computer readable medium storing computer executable instructions; and at least one processor communicatively coupled to the at least three acoustic emission sensors and configured to execute the computer executable instructions, the execution performing the method described above.

[0005] According to yet another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium comprising at least one program for execution by at least one processor of a first device, the at least one program comprising instructions, which, when executed by the at least one processor, cause the first device to perform the above-mentioned method. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 is an example environment in which a vehicle comprising one or more components of an autonomous system can be implemented;

[0007] Figure 2 is a diagram of one or more systems of a vehicle comprising an autonomous system;

[0008] Figure 3 is Figure 1 and Figure 2 a diagram of one or more devices and / or components of one or more systems;

[0009] Figure 4 is a diagram of certain components of an autonomous system;

[0010] Figure 5 is a diagram of an implementation of a process for acoustic emission (AE) based device control;

[0011] Figure 6 is an illustration of a wiper blade assembly having multiple sensors;

[0012] Figure 7 is an example of acoustic triangulation for locating a position in a multi-source scenario;

[0013] Figure 8 is a process flow diagram for acoustic emission based wiper blade assembly control;

[0014] Figure 9 is an illustration of a LiDAR assembly having multiple sensors; and

[0015] Figure 10 is a block diagram of a process for acoustic emission based device control. DETAILED DESCRIPTION

[0016] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that the embodiments described herein can be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present disclosure.

[0017] In the drawings, specific arrangements or orders of illustrative elements (such as those representing systems, devices, modules, instruction blocks, and / or data elements, etc.) are shown for ease of description. However, one skilled in the art will appreciate that the specific order or arrangement of illustrative elements in the drawings is not intended to imply that a particular processing order or sequence, or separation of processing, is required unless explicitly described as such. Additionally, the inclusion of an illustrative element in a drawing does not imply that such element is required in all embodiments, nor that the features represented by such element cannot be included in or associated with other elements in some embodiments.

[0018] Further, in the drawings, connecting elements, such as lines or arrows or the like, are used to illustrate connections, relationships or associations between or among two or more other illustrative elements, and the absence of such connecting elements is not intended to imply the absence of a connection, relationship or association. In other words, some connections, relationships or associations between elements are not illustrated in the drawings in order not to obscure the disclosure. Additionally, for ease of illustration, a single connecting element can be used to represent multiple connections, relationships or associations between elements. For example, if a connecting element represents the communication of signals, data or instructions (e.g., "software instructions"), one skilled in the art will appreciate that such element can represent one or more signal paths (e.g., buses) that can be required for the communication.

[0019] Although the terms "first," "second," and / or "third," etc. are used to describe various elements, these elements should not be limited by these terms. The terms "first," "second," and / or "third" are only used to differentiate one element from another. For example, a first contact can be called a second contact, and similarly, a second contact can be called a first contact, without departing from the scope of the described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

[0020] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” and / or “has a” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0021] As used herein, the terms “communication” and “in communication” mean at least one of receiving, receiving, transmitting, transferring, and / or providing, among others, information (or information represented by, for example, data, signals, messages, instructions, and / or commands). For a unit (e.g., a device, a system, a component of a device or system, and / or a combination thereof, among others) to be in communication with another unit means that the one unit is capable of either directly or indirectly receiving information from and / or transmitting (e.g., sending) information to the other unit. This can refer to a direct or indirect connection that is wired and / or wireless in nature. Additionally, two units can be in communication with each other even though the transmitted information can be modified, processed, relayed, and / or routed between the first and second units. For example, a first unit can be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit can be in communication with a second unit if at least one intermediary unit (e.g., a third unit positioned between the first and second units) processes information received from the first unit and transmits processed information to the second unit. In some embodiments, a message can refer to a network packet (e.g., a data packet, among others) that includes data.

[0022] As used herein, depending on the context, the term "if" may optionally be interpreted as "when," "in," "in response to being determined," and / or "in response to being detected," etc. Similarly, depending on the context, the phrases "if determined" or "if [the stated condition or event] is detected" may optionally be interpreted as "in response to being determined," "in response to being determined," "or" "in response to being detected," and / or "in response to being detected," etc. Furthermore, as used herein, the terms "have," "possess," or "own," etc., are intended to be open-ended terms. Additionally, unless explicitly stated otherwise, the phrase "based on" is intended to mean "at least partially based on."

[0023] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0024] General Overview

[0025] In some aspects and / or embodiments, the systems, methods, and computer program products described herein include and / or implement acoustic emission-based device control. Vehicles (such as autonomous vehicles) may have multiple sensors mounted at different locations on the vehicle. The sensors are mounted in the structure of at least one device (such as the arm of a wiper blade assembly). In embodiments, the wiper blade assembly is associated with: a windshield (e.g., front and rear windshields), side windows (e.g., driver's and passenger windows), sensor housings (e.g., LiDAR, camera, and radar lenses / covers), and lamp housings or covers, etc. During environmental conditions involving particle collisions with the surface of the vehicle, the sensors mounted on at least one device can detect acoustic emissions (e.g., vibrations) from the particle-surface collision, wherein the wiper blade assembly is adjacent to or in physical contact with the surface. The acoustic emissions captured by the sensors are used to map the acoustic emissions to parameters associated with the environmental conditions. Acoustic emission can also be used to detect deterioration of the device assembly, including the device and wiper blades, or contamination of the surface cleaned by the assembly when starting a device assembly (e.g., cleaning a surface using a cleaning device, or operating a wiper blade for cleaning a surface). Other cleaning methods can be used in response to the detection of deterioration or contamination.

[0026] By implementing the systems, methods, and computer program products described herein, techniques for controlling acoustic emission-based devices are provided. Some advantages of these techniques include the accurate detection of environmental condition parameters (e.g., rainfall rate). Furthermore, these environmental condition parameters are calculated at a very fine level (e.g., per car-basis), enabling more precise operation of vehicle functions based on environmental parameters. Additionally, this technique enables wiper-based cleaning methods that do not abrasively damage optical surfaces and thin films (e.g., AR coatings). In the example, determination is made of when abrasive damage may occur, and the operation of the AV (e.g., components of the AV) avoids abrasive damage.

[0027] Now for reference Figure 1 Example environment 100 is illustrated, in which vehicles including autonomous systems and vehicles not including autonomous systems operate. As illustrated, environment 100 includes vehicles 102a-102n, objects 104a-104n, routes 106a-106n, area 108, vehicle-to-infrastructure (V2I) device 110, network 112, remote autonomous vehicle (AV) system 114, queue management system 116, and V2I system 118. Vehicles 102a-102n, vehicle-to-infrastructure (V2I) device 110, network 112, autonomous vehicle (AV) system 114, queue management system 116, and V2I system 118 are interconnected via wired connections, wireless connections, or a combination of wired and wireless connections (e.g., establishing connections for communication, etc.). In some embodiments, objects 104a-104n are interconnected with at least one of vehicles 102a-102n, vehicle-to-infrastructure (V2I) devices 110, network 112, autonomous vehicle (AV) system 114, queue management system 116, and V2I system 118 via wired connection, wireless connection, or a combination of wired and wireless connection.

[0028] Vehicles 102a-102n (specifically referred to as vehicle 102 and collectively as vehicle 102) include at least one device configured to transport goods and / or people. In some embodiments, vehicle 102 is configured to communicate with V2I device 110, remote AV system 114, queue management system 116 and / or V2I system 118 via network 112. In some embodiments, vehicle 102 includes cars, buses, trucks and / or trains, etc. In some embodiments, vehicle 102 is associated with vehicle 200 described herein (see Figure 2The vehicles 102 are the same as or similar to autonomous vehicles 202. In some embodiments, vehicles 200 in a group of vehicles 200 are associated with an autonomous queue manager. In some embodiments, as described herein, vehicles 102 travel along corresponding routes 106a-106n (each individually referred to as route 106 and collectively as route 106). In some embodiments, one or more vehicles 102 include an autonomous system (e.g., an autonomous system that is the same as or similar to autonomous system 202).

[0029] Objects 104a-104n (each individually referred to as object 104 and collectively as object 104) include, for example, at least one vehicle, at least one pedestrian, at least one cyclist, and / or at least one structure (e.g., a building, a sign, a fire hydrant, etc.). Each object 104 (e.g., located at a fixed location and for a period of time) is either stationary or (e.g., having a speed and associated with at least one trajectory) moving. In some embodiments, object 104 is associated with a corresponding location in area 108.

[0030] Routes 106a-106n (each individually referred to as Route 106 and collectively as Route 106) are each associated with (e.g., defining) a series of actions (also referred to as trajectories) along which the connecting AV can navigate. Each Route 106 begins with an initial state (e.g., a state corresponding to a first spatiotemporal location and / or speed, etc.) and ends with a final target state (e.g., a state corresponding to a second spatiotemporal location different from the first spatiotemporal location) or a target area (e.g., a subspace of an acceptable state (e.g., a termination state)). In some embodiments, a first state includes a location where one or more individuals will board the AV, and a second state or area includes a location where one or more individuals boarding the AV will disembark. In some embodiments, Route 106 includes multiple acceptable state sequences (e.g., multiple spatiotemporal location sequences) associated with multiple trajectories (e.g., defining multiple trajectories). In the example, Route 106 includes only high-level actions or imprecise state locations, such as a series of connecting roads indicating a change of direction at a roadway intersection. Additionally or alternatively, route 106 may include more precise actions or states, such as, for example, specific target lanes or precise locations within a lane area and target rates at those locations. In the example, route 106 includes multiple precise state sequences along at least one high-level action with a finite look-ahead horizon leading to an intermediate target, wherein the cumulative combination of successive iterations of the finite horizon state sequences corresponds to multiple trajectories that collectively form a high-level route terminating at a final target state or region.

[0031] Region 108 includes a physical area (e.g., a geographic region) that the vehicle 102 can navigate. In the example, region 108 includes at least one state (e.g., a country, a province, a single state among multiple states included in a country, etc.), at least a portion of a state, at least one city, at least a portion of a city, etc. In some embodiments, region 108 includes at least one named arterial road (referred to herein as a "road"), such as a highway, interstate highway, park road, city street, etc. Additionally or alternatively, in some examples, region 108 includes at least one unnamed road, such as a driving lane, a section of a parking lot, a section of vacant land and / or undeveloped area, dirt road, etc. In some embodiments, a road includes at least one lane (e.g., a portion of the road that the vehicle 102 can traverse). In the example, a road includes at least one lane associated with at least one lane marking (e.g., identified based on at least one lane marking).

[0032] The Vehicle-to-Infrastructure (V2I) device 110 (sometimes referred to as a Vehicle-to-Everything (V2X) device) includes at least one device configured to communicate with vehicle 102 and / or V2I system 118. In some embodiments, V2I device 110 is configured to communicate with vehicle 102, remote AV system 114, queue management system 116, and / or V2I system 118 via network 112. In some embodiments, V2I device 110 includes radio frequency identification (RFID) devices, signs, cameras (e.g., two-dimensional (2D) and / or three-dimensional (3D) cameras), lane markings, streetlights, parking meters, etc. In some embodiments, V2I device 110 is configured to communicate directly with vehicle 102. Additionally or alternatively, in some embodiments, V2I device 110 is configured to communicate with vehicle 102, remote AV system 114, and / or queue management system 116 via V2I system 118. In some embodiments, V2I device 110 is configured to communicate with V2I system 118 via network 112.

[0033] Network 112 includes one or more wired and / or wireless networks. In the example, network 112 includes cellular networks (e.g., Long Term Evolution (LTE) networks, third-generation (3G) networks, fourth-generation (4G) networks, fifth-generation (5G) networks, Code Division Multiple Access (CDMA) networks, etc.), Public Land Mobile Networks (PLMNs), Local Area Networks (LANs), Wide Area Networks (WANs), Metropolitan Area Networks (MANs), telephone networks (e.g., Public Switched Telephone Networks (PSTN)), private networks, self-organizing networks, intranets, the Internet, fiber-based networks, cloud computing networks, etc., and / or combinations of some or all of these networks.

[0034] The remote AV system 114 includes at least one device configured to communicate with the vehicle 102, V2I device 110, network 112, queue management system 116, and / or V2I system 118 via network 112. In examples, the remote AV system 114 includes a server, server group, and / or other similar devices. In some embodiments, the remote AV system 114 is located in the same location as the queue management system 116. In some embodiments, the remote AV system 114 participates in the installation of some or all of the components of the vehicle, including autonomous systems, autonomous vehicle computing, and / or software implemented by autonomous vehicle computing. In some embodiments, the remote AV system 114 maintains (e.g., updates and / or replaces) these components and / or software during the lifespan of the vehicle.

[0035] The queue management system 116 includes at least one device configured to communicate with vehicle 102, V2I device 110, remote AV system 114, and / or V2I system 118. In examples, the queue management system 116 includes servers, server groups, and / or other similar devices. In some embodiments, the queue management system 116 is associated with a ride-sharing company (e.g., an organization for controlling the operation of multiple vehicles (e.g., vehicles including autonomous systems and / or vehicles not including autonomous systems)).

[0036] In some embodiments, the V2I system 118 includes at least one device configured to communicate with the vehicle 102, the V2I device 110, the remote AV system 114, and / or the queue management system 116 via a network 112. In some examples, the V2I system 118 is configured to communicate with the V2I device 110 via a connection different from the network 112. In some embodiments, the V2I system 118 includes a server, a server group, and / or other similar devices. In some embodiments, the V2I system 118 is associated with a municipality or private entity (e.g., a private entity maintaining the V2I device 110).

[0037] supply Figure 1 The number and arrangement of the elements are shown as examples. (and) Figure 1 Compared to the illustrated elements, there may be additional elements, fewer elements, different elements, and / or elements arranged differently. Additionally or alternatively, at least one element of environment 100 may be described as being composed of… Figure 1 One or more functions performed by at least one different element of environment 100. Additionally or alternatively, at least one group of elements of environment 100 may perform one or more functions described as performed by at least one different group of elements of environment 100.

[0038] Now for referenceFigure 2 The vehicle 200 includes an autonomous system 202, a powertrain control system 204, a steering control system 206, and a braking system 208. In some embodiments, the vehicle 200 and the vehicle 102 (see...) Figure 1 The vehicle 200 is similar to or the same as the vehicle in question. In some embodiments, the vehicle 200 has autonomous capabilities (e.g., implementing at least one function, feature, and / or device that enables the vehicle 200 to operate partially or fully without human intervention, including but not limited to fully autonomous vehicles (e.g., vehicles that abandon human intervention) and / or highly autonomous vehicles (e.g., vehicles that abandon human intervention in certain situations)). For a detailed description of fully autonomous and highly autonomous vehicles, refer to SAE International's standard J3016: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems, the entire contents of which are incorporated herein by reference. In some embodiments, the vehicle 200 is associated with an autonomous queue manager and / or a ride-sharing company.

[0039] Autonomous system 202 includes a sensor suite comprising one or more devices such as camera 202a, LiDAR sensor 202b, radar sensor 202c, and microphone 202d. In some embodiments, autonomous system 202 may include more or fewer devices and / or different devices (e.g., ultrasonic sensors, inertial sensors, GPS receivers (discussed below), and / or odometer sensors for generating data associated with an indication of the distance traveled by vehicle 200). In some embodiments, autonomous system 202 uses one or more devices included in autonomous system 202 to generate data associated with environment 100 as described herein. The data generated by one or more devices of autonomous system 202 may be used by one or more systems as described herein to observe the environment in which vehicle 200 is located (e.g., environment 100). In some embodiments, autonomous system 202 includes communication device 202e, autonomous vehicle computing 202f, and safety controller 202g.

[0040] Camera 202a includes components configured to communicate with communication device 202e, autonomous vehicle computing 202f, and / or safety controller 202g via a bus (e.g., with...). Figure 3At least one means of communicating with the same or similar bus as bus 302. Camera 202a includes at least one camera (e.g., a digital camera using a light sensor such as a charge-coupled device (CCD), a thermal camera, an infrared (IR) camera, and / or an event camera, etc.) for capturing images of physical objects (e.g., cars, buses, curbs, and / or people, etc.). In some embodiments, camera 202a generates camera data as output. In some examples, camera 202a generates camera data including image data associated with an image. In this example, the image data may specify at least one parameter corresponding to the image (e.g., image characteristics such as exposure, brightness, etc., and / or image timestamp, etc.). In such examples, the image may be in a format (e.g., RAW, JPEG, and / or PNG, etc.). In some embodiments, camera 202a includes multiple independent cameras configured (e.g., positioned on) a vehicle to capture images for stereoscopic imaging (stereoscopic vision). In some examples, camera 202a includes generating image data and transmitting the image data to an autonomous vehicle computing 202f and / or a queue management system (e.g., with...). Figure 1 The queue management system 116 (same as or similar to a queue management system) has multiple cameras. In such an example, the autonomous vehicle calculation 202f determines the depth of one or more objects in the fields of view of at least two of the multiple cameras based on image data from at least two cameras. In some embodiments, camera 202a is configured to capture images of objects within a distance relative to camera 202a (e.g., up to 100 meters and / or up to 1 kilometer, etc.). Therefore, camera 202a includes features such as sensors and lenses optimized for sensing objects at one or more distances relative to camera 202a.

[0041] In embodiments, camera 202a includes at least one camera configured to capture one or more images associated with one or more traffic lights, street signs, and / or other physical objects providing visual navigation information. In some embodiments, camera 202a generates traffic light data associated with one or more images. In some examples, camera 202a generates TLD data associated with one or more images, including formats such as RAW, JPEG, and / or PNG. In some embodiments, camera 202a, which generates TLD data, differs from other systems containing cameras described herein in that camera 202a may include one or more cameras with a wide field of view (e.g., a wide-angle lens, a fisheye lens, and / or a lens with an angle of view of about 120 degrees or greater) to generate images associated with as many physical objects as possible.

[0042] The laser detection and ranging (LiDAR) sensor 202b includes components configured to communicate with a communication device 202e, an autonomous vehicle computing unit 202f, and / or a safety controller 202g via a bus (e.g., with...). Figure 3 At least one device that communicates with the same or similar bus (bus 302). The LiDAR sensor 202b includes a system configured to emit light from a emitter (e.g., a laser emitter). The light emitted by the LiDAR sensor 202b includes light outside the visible spectrum (e.g., infrared light, etc.). In some embodiments, during operation, the light emitted by the LiDAR sensor 202b encounters a physical object (e.g., a vehicle) and is reflected back to the LiDAR sensor 202b. In some embodiments, the light emitted by the LiDAR sensor 202b does not penetrate the physical object it encounters. The LiDAR sensor 202b also includes at least one photosensor that detects the light after it has encountered a physical object. In some embodiments, at least one data processing system associated with the LiDAR sensor 202b generates an image (e.g., point cloud and / or combined point cloud, etc.) representing objects included in the field of view of the LiDAR sensor 202b. In some examples, at least one data processing system associated with the LiDAR sensor 202b generates an image representing the boundaries of a physical object and / or the surface of the physical object (e.g., the topology of the surface). In such examples, the image is used to determine the boundaries of the physical object within the field of view of the LiDAR sensor 202b.

[0043] The radio detection and ranging (radar) sensor 202c includes components configured to communicate with the communication device 202e, the autonomous vehicle computing 202f, and / or the safety controller 202g via a bus (e.g., with...). Figure 3 At least one device that communicates with the same or similar bus (bus 302). The radar sensor 202c includes a system configured to emit (pulsed or continuous) radio waves. The radio waves emitted by the radar sensor 202c include radio waves within a predetermined spectrum. In some embodiments, during operation, the radio waves emitted by the radar sensor 202c encounter a physical object and are reflected back to the radar sensor 202c. In some embodiments, the radio waves emitted by the radar sensor 202c are not reflected by some objects. In some embodiments, at least one data processing system associated with the radar sensor 202c generates a signal representing objects included in the field of view of the radar sensor 202c. For example, at least one data processing system associated with the radar sensor 202c generates an image representing the boundaries of physical objects and / or the surfaces of physical objects (e.g., surface topology). In some examples, this image is used to determine the boundaries of physical objects in the field of view of the radar sensor 202c.

[0044] Microphone 202d includes components configured to communicate with communication device 202e, autonomous vehicle computing 202f, and / or safety controller 202g via a bus (e.g., with...). Figure 3 At least one device that communicates with the same or similar bus as bus 302. Microphone 202d includes one or more microphones (e.g., array microphones and / or external microphones, etc.) that capture audio signals and generate data associated with (e.g., representing) the audio signals. In some examples, microphone 202d includes transducer devices and / or similar devices. In some embodiments, one or more systems described herein can receive data generated by microphone 202d and determine the position (e.g., distance, etc.) of an object relative to vehicle 200 based on the audio signal associated with the data.

[0045] The communication device 202e includes at least one device configured to communicate with a camera 202a, a LiDAR sensor 202b, a radar sensor 202c, a microphone 202d, an autonomous vehicle computing system 202f, a safety controller 202g, and / or a drive-by-wire (DBW) system 202h. For example, the communication device 202e may include communication with… Figure 3 The communication device 202e is the same as or similar to the communication interface 314. In some embodiments, the communication device 202e includes a vehicle-to-vehicle (V2V) communication device (e.g., a device for enabling wireless communication of data between vehicles).

[0046] The autonomous vehicle computing 202f includes at least one device configured to communicate with a camera 202a, a LiDAR sensor 202b, a radar sensor 202c, a microphone 202d, a communication device 202e, a security controller 202g, and / or a DBW system 202h. In some examples, the autonomous vehicle computing 202f includes devices such as client devices, mobile devices (e.g., cellular phones and / or tablets) and / or servers (e.g., computing devices including one or more central processing units and / or graphics processing units). In some embodiments, the autonomous vehicle computing 202f is the same as or similar to the autonomous vehicle computing 400 described herein. Additionally or alternatively, in some embodiments, the autonomous vehicle computing 202f is configured to communicate with an autonomous vehicle system (e.g., with...). Figure 1 Remote AV systems 114 are the same as or similar to autonomous vehicle systems), queue management systems (e.g., with...). Figure 1 The queue management system 116 is the same as or similar to the queue management system 116), and V2I devices (e.g., with Figure 1 V2I devices (same as or similar to V2I devices 110) and / or V2I systems (e.g., with V2I devices 110)Figure 1 The V2I system 118 communicates with the same or similar V2I system.

[0047] The safety controller 202g includes at least one device configured to communicate with a camera 202a, a LiDAR sensor 202b, a radar sensor 202c, a microphone 202d, a communication device 202e, an autonomous vehicle computing system 202f, and / or a DBW system 202h. In some examples, the safety controller 202g includes one or more controllers (electrical controllers and / or electromechanical controllers, etc.) configured to generate and / or transmit control signals to operate the vehicle 200 (e.g., powertrain control system 204, steering control system 206, and / or braking system 208, etc.). In some embodiments, the safety controller 202g is configured to generate control signals that take precedence over (e.g., override) the control signals generated and / or transmitted by the autonomous vehicle computing system 202f.

[0048] The DBW system 202h includes at least one device configured to communicate with the communication device 202e and / or the autonomous vehicle computing 202f. In some examples, the DBW system 202h includes one or more controllers (e.g., electrical controllers and / or electromechanical controllers, etc.) configured to generate and / or transmit control signals to operate the vehicle 200. Additionally or alternatively, one or more controllers of the DBW system 202h are configured to generate and / or transmit control signals to operate at least one different device (e.g., turn signals, headlights, door locks, and / or windshield wipers, etc.) of the vehicle 200.

[0049] The powertrain control system 204 includes at least one device configured to communicate with the DBW system 202h. In some examples, the powertrain control system 204 includes at least one controller and / or actuator, etc. In some embodiments, the powertrain control system 204 receives control signals from the DBW system 202h, and the powertrain control system 204 causes the vehicle 200 to start moving forward, stop moving forward, start moving backward, stop moving backward, accelerate in a certain direction, decelerate in a certain direction, make a left turn and / or make a right turn, etc. In examples, the powertrain control system 204 increases, keeps the same, or decreases the energy (e.g., fuel and / or electricity, etc.) supplied to the motor of the vehicle, thereby causing at least one wheel of the vehicle 200 to rotate or not rotate.

[0050] The steering control system 206 includes at least one device configured to rotate one or more wheels of the vehicle 200. In some examples, the steering control system 206 includes at least one controller and / or actuator, etc. In some embodiments, the steering control system 206 causes the two front wheels and / or the two rear wheels of the vehicle 200 to turn left or right, thereby causing the vehicle 200 to turn left or right.

[0051] The braking system 208 includes at least one device configured to actuate one or more brakes to decelerate and / or keep the vehicle 200 stationary. In some examples, the braking system 208 includes at least one controller and / or actuator configured to close one or more calipers associated with one or more wheels of the vehicle 200 on the respective rotor of the vehicle 200. Additionally or alternatively, in some examples, the braking system 208 includes an automatic emergency braking (AEB) system and / or a regenerative braking system, etc.

[0052] In some embodiments, the vehicle 200 includes at least one platform sensor (not explicitly illustrated) for measuring or inferring the nature of the state or conditions of the vehicle 200. In some examples, the vehicle 200 includes platform sensors such as a Global Positioning System (GPS) receiver, an Inertial Measurement Unit (IMU), a wheel rate sensor, a wheel brake pressure sensor, a wheel torque sensor, an engine torque sensor, and / or a steering angle sensor.

[0053] Now for reference Figure 3 A schematic diagram of device 300 is illustrated. As illustrated, device 300 includes a processor 304, a memory 306, a storage component 308, an input interface 310, an output interface 312, a communication interface 314, and a bus 302. In some embodiments, device 300 corresponds to: at least one device of vehicle 102 (e.g., at least one device of system of vehicle 102); and / or one or more devices of network 112 (e.g., one or more devices of system of network 112). In some embodiments, one or more devices of vehicle 102 (e.g., one or more devices of system of vehicle 102), and / or one or more devices of network 112 (e.g., one or more devices of system of network 112) include at least one device 300 and / or at least one component of device 300. Figure 3 As shown, the device 300 includes a bus 302, a processor 304, a memory 306, a storage component 308, an input interface 310, an output interface 312, and a communication interface 314.

[0054] Bus 302 includes components for communication between the components of the licensed device 300. In some embodiments, processor 304 is implemented in hardware, software, or a combination of hardware and software. In some examples, processor 304 includes a processor (e.g., a central processing unit (CPU), graphics processing unit (GPU), and / or accelerated processing unit (APU), a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function (e.g., a field-programmable gate array (FPGA) and / or application-specific integrated circuit (ASIC), etc.). Memory 306 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic and / or static storage device (e.g., flash memory, magnetic memory, and / or optical memory, etc.) that stores data and / or instructions for use by processor 304.

[0055] Storage component 308 stores data and / or software related to the operation and use of device 300. In some examples, storage component 308 includes hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), compact discs (CDs), digital versatile discs (DVDs), floppy disks, cassette tapes, magnetic tapes, CD-ROMs, RAM, PROMs, EPROMs, FLASH-EPROMs, NV-RAMs, and / or other types of computer-readable media, and corresponding drives.

[0056] Input interface 310 includes components that enable the device 300 to receive information, such as via user input (e.g., a touchscreen display, keyboard, keypad, mouse, buttons, switches, microphone, and / or camera). Additionally or alternatively, in some embodiments, input interface 310 includes sensors for sensing information (e.g., a Global Positioning System (GPS) receiver, accelerometer, gyroscope, and / or actuator). Output interface 312 includes components for providing output information from device 300 (e.g., a display, speaker, and / or one or more light-emitting diodes (LEDs)).

[0057] In some embodiments, the communication interface 314 includes transceiver-like components (e.g., a transceiver and / or separate receivers and transmitters) that enable the licensing device 300 to communicate with other devices via a wired connection, a wireless connection, or a combination of wired and wireless connections. In some examples, the communication interface 314 enables the licensing device 300 to receive information from and / or provide information to another device. In some examples, the communication interface 314 includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, etc. Interfaces and / or cellular network interfaces, etc.

[0058] In some embodiments, device 300 performs one or more of the processes described herein. Device 300 performs these processes based on software instructions stored in a computer-readable medium, such as memory 306 and / or storage component 308, executed by processor 304. Computer-readable medium (e.g., non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes storage space located within a single physical storage device or storage space distributed across multiple physical storage devices.

[0059] In some embodiments, software instructions are read from another computer-readable medium or from another device via communication interface 314 into memory 306 and / or storage component 308. When executed, the software instructions stored in memory 306 and / or storage component 308 cause processor 304 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, unless explicitly stated otherwise, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0060] The memory 306 and / or storage component 308 include a data storage unit or at least one data structure (e.g., a database). The device 300 is capable of receiving information from the data storage unit or at least one data structure in the memory 306 or storage component 308, storing the information in the data storage unit or at least one data structure, communicating information to the data storage unit or at least one data structure, or searching for information stored in the data storage unit or at least one data structure. In some examples, the information includes network data, input data, output data, or any combination thereof.

[0061] In some embodiments, device 300 is configured to execute software instructions stored in the memory of memory 306 and / or another device (e.g., another device identical or similar to device 300). As used herein, the term "module" refers to at least one instruction stored in the memory of memory 306 and / or the other device, which, when executed by the processor of processor 304 and / or the processor of another device (e.g., another device identical or similar to device 300), causes device 300 (e.g., at least one component of device 300) to perform one or more processes as described herein. In some embodiments, modules are implemented in software, firmware, and / or hardware, etc.

[0062] supply Figure 3 The number and arrangement of components are illustrated as examples. In some embodiments, with Figure 3Compared to the illustrated components, device 300 may include additional components, fewer components, different components, or components arranged differently. Additionally or alternatively, a group of components of device 300 (e.g., one or more components) may perform one or more functions described as being performed by another component or another group of components of device 300.

[0063] Now for reference Figure 4 The diagram illustrates an example block diagram of an autonomous vehicle computing 400 (sometimes referred to as an "AV stack"). As illustrated, the autonomous vehicle computing 400 includes a perception system 402 (sometimes referred to as a perception module), a planning system 404 (sometimes referred to as a planning module), a positioning system 406 (sometimes referred to as a positioning module), a control system 408 (sometimes referred to as a control module), and a database 410. In some embodiments, the perception system 402, planning system 404, positioning system 406, control system 408, and database 410 are included in and / or implemented in the vehicle's automatic navigation system (e.g., the autonomous vehicle computing 202f of vehicle 200). Additionally or alternatively, in some embodiments, the perception system 402, planning system 404, positioning system 406, control system 408, and database 410 are included in one or more separate systems (e.g., one or more systems that are the same as or similar to the autonomous vehicle computing 400, etc.). In some examples, the perception system 402, planning system 404, positioning system 406, control system 408, and database 41 are included in one or more independent systems located within the vehicle and / or at least one remote system as described herein. In some embodiments, any and / or all of the systems included in the autonomous vehicle computing 400 are implemented in software (e.g., software instructions stored in memory), computer hardware (e.g., via microprocessors, microcontrollers, application-specific integrated circuits (ASICs), and / or field-programmable gate arrays (FPGAs), etc.), or a combination of computer software and computer hardware. It will also be understood that in some embodiments, the autonomous vehicle computing 400 is configured to communicate with remote systems (e.g., autonomous vehicle systems identical or similar to remote AV system 114, queue management systems identical or similar to queue management systems 116, and / or V2I systems identical or similar to V2I system 118, etc.).

[0064] In some embodiments, the perception system 402 receives data associated with at least one physical object in the environment (e.g., data used by the perception system 402 to detect at least one physical object) and classifies the at least one physical object. In some examples, the perception system 402 receives image data captured by at least one camera (e.g., camera 202a) that is associated with one or more physical objects within the field of view of the at least one camera (e.g., representing the one or more physical objects). In such examples, the perception system 402 classifies at least one physical object based on one or more groups of physical objects (e.g., bicycles, vehicles, traffic signs, and / or pedestrians, etc.). In some embodiments, based on the classification of physical objects by the perception system 402, the perception system 402 transmits data associated with the classification of the physical objects to the planning system 404.

[0065] In some embodiments, the planning system 404 receives data associated with a destination and generates data associated with at least one route (e.g., route 106) along which a vehicle (e.g., vehicle 102) can travel toward the destination. In some embodiments, the planning system 404 periodically or continuously receives data from the sensing system 402 (e.g., the data associated with the classification of physical objects described above), and the planning system 404 updates at least one trajectory or generates at least one different trajectory based on the data generated by the sensing system 402. In some embodiments, the planning system 404 receives data associated with the updated location of the vehicle (e.g., vehicle 102) from the positioning system 406, and the planning system 404 updates at least one trajectory or generates at least one different trajectory based on the data generated by the positioning system 406.

[0066] In some embodiments, positioning system 406 receives data associated with (e.g., representing) a location of a vehicle (e.g., vehicle 102) in an area. In some examples, positioning system 406 receives LiDAR data associated with at least one point cloud generated by at least one LiDAR sensor (e.g., LiDAR sensor 202b). In some examples, positioning system 406 receives data associated with at least one point cloud from multiple LiDAR sensors, and positioning system 406 generates a composite point cloud based on the individual point clouds. In these examples, positioning system 406 compares the at least one point cloud or composite point cloud with a two-dimensional (2D) and / or three-dimensional (3D) map of the area stored in database 410. Then, based on the comparison of the at least one point cloud or composite point cloud with the map, positioning system 406 determines the location of the vehicle in the area. In some embodiments, the map includes a composite point cloud of the area generated prior to navigation of the vehicle. In some embodiments, the map includes, but is not limited to, a high-precision map of the geometry of the roadway, a map describing the connectivity of the road network, a map describing the physical properties of the roadway (such as traffic speed, traffic flow, the number of vehicle and bicycle lanes, lane width, lane traffic direction, or the type and location of lane markings, or combinations thereof), and a map describing the spatial locations of road features (such as pedestrian crossings, traffic signs, or various types of other traffic lights). In some embodiments, the map is generated in real time based on data received by the sensing system.

[0067] In another example, positioning system 406 receives Global Navigation Satellite System (GNSS) data generated by a Global Positioning System (GPS) receiver. In some examples, positioning system 406 receives GNSS data associated with the location of a vehicle in an area, and positioning system 406 determines the latitude and longitude of the vehicle in the area. In such examples, positioning system 406 determines the location of the vehicle in the area based on the latitude and longitude of the vehicle. In some embodiments, positioning system 406 generates data associated with the location of the vehicle. In some examples, based on the location of the vehicle determined by positioning system 406, positioning system 406 generates data associated with the location of the vehicle. In such examples, the data associated with the location of the vehicle includes data associated with one or more semantic properties corresponding to the location of the vehicle.

[0068] In some embodiments, the control system 408 receives data associated with at least one trajectory from the planning system 404, and the control system 408 controls the operation of the vehicle. In some examples, the control system 408 receives data associated with at least one trajectory from the planning system 404, and the control system 408 controls the operation of the vehicle by generating and transmitting control signals to operate the powertrain control system (e.g., DBW system 202h and / or powertrain control system 204, etc.), the steering control system (e.g., steering control system 206), and / or the braking system (e.g., braking system 208). In an example, where the trajectory includes a left turn, the control system 408 transmits control signals to cause the steering control system 206 to adjust the steering angle of the vehicle 200, thereby causing the vehicle 200 to turn left. Additionally or alternatively, the control system 408 generates and transmits control signals to change the state of other devices of the vehicle 200 (e.g., headlights, turn signals, door locks, and / or windshield wipers, etc.).

[0069] In some embodiments, the perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model (e.g., at least one multilayer perceptron (MLP), at least one convolutional neural network (CNN), at least one recurrent neural network (RNN), at least one autoencoder, and / or at least one transformer, etc.). In some examples, the perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model individually or in combination with one or more of the aforementioned systems. In some examples, the perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model as part of a pipeline (e.g., a pipeline for identifying one or more objects located in the environment, etc.).

[0070] Database 410 stores data transmitted to, received from, and / or updated by the sensing system 402, planning system 404, positioning system 406, and / or control system 408. In some examples, database 410 includes storage components for storing operation-related data and / or software, and for computing 400 using autonomous vehicles (e.g., with...). Figure 3(The storage component 308 is the same as or similar to the storage component 308). In some embodiments, database 410 stores data associated with 2D and / or 3D maps of at least one area. In some examples, database 410 stores data associated with 2D and / or 3D maps of a part of a city, multiple parts of multiple cities, multiple cities, counties, states, and / or countries (e.g., countries). In such examples, a vehicle (e.g., the same as or similar to vehicle 102 and / or vehicle 200) can drive along one or more drivable areas (e.g., single-lane roads, multi-lane roads, highways, remote roads, and / or off-road roads, etc.) and causes at least one LiDAR sensor (e.g., the same as or similar to LiDAR sensor 202b) to generate data associated with images representing objects included in the field of view of the at least one LiDAR sensor.

[0071] In some embodiments, database 410 may be implemented across multiple devices. In some examples, database 410 includes a vehicle (e.g., a vehicle identical or similar to vehicle 102 and / or vehicle 200), an autonomous vehicle system (e.g., an autonomous vehicle system identical or similar to remote AV system 114), and a queue management system (e.g., with...). Figure 1 Queue management system 116 (same as or similar to queue management system) and / or V2I system (e.g., with Figure 1 Among the V2I systems (118 similar to or similar V2I systems), etc.

[0072] Now for reference Figure 5 An implementation 500 for a process for device control based on acoustic emission is illustrated. In some embodiments, implementation 500 includes a device assembly 510. In embodiments, the device assembly is a wiper blade assembly (e.g., a wiper blade) positioned abutting or in physical contact with a windshield (e.g., a surface). Figure 6 Wiper blade assembly 600). In the example, the assembly is a wiper blade assembly positioned adjacent to or in physical contact with the LiDAR sensor housing (e.g., Figure 9 Wiper blade assembly 900). In the example, the device assembly is a wiper blade assembly located adjacent to or in physical contact with the camera sensor housing (e.g., Figure 9 Wiper blade assembly 900). In the example, the assembly is a wiper blade assembly positioned adjacent to or in physical contact with the headlight lens (e.g., Figure 9 Wiper blade assembly 900).

[0073] Typically, a device assembly includes a support member (e.g., an arm) and a cleaning member (e.g., a wiper blade). In an example of a windshield wiper blade assembly, the support member is one or more arms that rotate in a repetitive motion. The cleaning member is one or more wiper blades attached to one or more swinging arms. The wiper blades are located near or in physical contact with a surface. During operation, the swinging arms cause the wiper blades to wipe or otherwise clean the surface.

[0074] In an embodiment, sensors associated with the device assembly capture acoustic emissions occurring on surfaces, the device assembly, or any combination thereof. Typically, acoustic emissions are vibrations captured in response to a collision event. Parameters associated with environmental conditions are calculated. Figure 5 In the example, acoustic emissions captured at device assembly 510 are used to generate control signal 518 at control system 504b. The control signal is transmitted to the drive-by-wire system at reference numeral 520. In this embodiment, the drive-by-wire system is used to control the operation of vehicle 502. In this embodiment, the control signal transmitted to device assembly 510 controls the operation of the device assembly.

[0075] Typically, this technology improves the resolution of rainfall detection. It mitigates the impact of parameters such as droplet size, droplet spacing, rainfall rate, and droplet removal rate on various types of sensors, such as cameras and LiDAR. This technology enables the detection of these parameters and, consequently, the determination of whether a vehicle is safe to operate or whether the environment is unsafe and thus beyond the vehicle's operational limits. Furthermore, this technology eliminates abrasive damage caused by wiper-based cleaning of vehicle surfaces, such as optical surfaces and optical films (e.g., anti-reflective (AR) coatings). Traditionally, abrasive particles adhere to optical surfaces and move along them when the wiper blade assembly is activated, resulting in wear on the optical surfaces. This technology detects and measures abrasive contamination on surfaces and enables alternative cleaning techniques in response to potential wear during wiper-based cleaning. In embodiments, alternative cleaning techniques include fluid-based cleaning methods. For example, a cleaning fluid is sprayed onto the surface using a nozzle. Compressed air is blown onto or across the surface at high speed to remove cleaning fluid droplets (if any).

[0076] Figure 6 This is an example of a wiper blade assembly 600 with multiple sensors. Figure 6 In the example, the wiper blade assembly 600 includes a motor (not shown) that is controlled at least in part based on acoustic emission. Figure 6In the example, a first wiper arm 602A and a second wiper arm 602B are illustrated. The second wiper arm 602B includes a first sensor 604A, a second sensor 604B, and a third sensor 604C (collectively referred to as sensor 604). In an embodiment, sensor 604 is a piezoelectric sensor. Sensor 604 is mounted in the arm 602 of the wiper blade assembly. In the example, sensor 604 is embedded or otherwise coupled to the arm 602 such that vibrations impacting the arm 602, the wiper blade, or a surface are detected by sensor 604.

[0077] In an embodiment, particles (e.g., rain, debris, other environmental pollutants) collide with one or more surfaces and components of a vehicle. Sensor 604 captures vibrations on the surfaces and components in response to the particle collisions. In an example, vibrations are generated in response to raindrops impacting the windshield of the vehicle. The vibrations are ultimately captured by the sensor. In an embodiment, parameters vary according to the intensity of the vibrations detected by sensor 604. Therefore, the detected vibrations are used to calculate parameters associated with the environment, such as rainfall rate, rainfall frequency, droplet size, and raindrop spread. In an embodiment, a database stores a mapping of vibration patterns detected in acoustic emissions to one or more parameters associated with the environment. In an embodiment, wiper blade components are controlled based on the calculated parameters. In an embodiment, the vehicle is controlled based on the calculated parameters. In an embodiment, the database of vibration information is training data used to train a neural network that predicts environmental parameters based on vibration information.

[0078] Figure 7 This is an example of acoustic triangulation used for location positioning in a multi-source situation. Figure 7 In the example, acoustic triangulation is used to calculate the rainfall rate. Sensors are mounted at different locations on the same surface. In this embodiment, the sensors are mounted at different locations resembling pyramids or triangles, depending on the geometry of the surface where they are mounted. The sensors are communicatively coupled to a timing system that measures the precise time when each sensor (e.g., sensor 604) detects a valid event. In this embodiment, the timing system uses a timer to integrate several operations of the vehicle, thereby achieving time-based synchronization of vehicle operations. In this embodiment, triangulation is used for measurement and calibration purposes. Triangulation software uses both the surface geometry and the arrangement and arrival times of the events to calculate a unit vector in the direction of the origin of the events. The absolute coordinates of the acoustic emission source corresponding to the event are then calculated.

[0079] In this embodiment, a valid event is the generation of acoustic emission in response to precipitation. During the operation of the vehicle, water or debris unrelated to rainfall may collide with the vehicle. This can happen, for example, when water and mud splash onto the vehicle from a puddle, such as a puddle. It can also happen, for example, when a sprinkler system causes water to impact the vehicle. Events originating from puddles and sprinklers are not valid events. In this embodiment, the technology enables the implementation of a backup system for verifying that an event represents rainfall (e.g., is a valid event).

[0080] exist Figure 7 In the example, several equations associated with triangulation are illustrated at reference numeral 702A. A first circle 706A has a center S1, which represents the source location of an event detected by a first sensor. A second circle 708A has a center S2, which represents the source location of an event detected by a second sensor. A third circle 710A has a center S3, which represents the source location of an event detected by a third sensor. Triangulation techniques are used to process each source location. Specifically, the following equation is used to determine possible acoustic emission source locations:

[0081] Ri=tiVg

[0082] Where i refers to the i-th sensor, t represents the time associated with the event detected at the i-th sensor, and V g This indicates wave velocity.

[0083] exist Figure 7 In the example, the calculation of possible acoustic emission source locations 712 is illustrated at reference numeral 702B. Various sensors are used to calculate multiple possible locations where the detected event has occurred. These possible locations are illustrated as loops 716B, 718B, and 720B. The intersection of possible locations 716B, 718B, and 720B occurs at possible acoustic emission source locations 712. Typically, the actual acoustic emission (AE) source location 704B lies within the possible acoustic emission source locations 712 calculated based on the illustrated triangulation. The calculation of the location associated with the detected event enables the determination of environmental parameters as described above.

[0084] In this embodiment, the wiper blade assembly is controlled based on calculated parameters. The wiper blades of the wiper assembly wipe away water, snow, wash fluid, and other liquids or debris from a surface. In this example, the wiper assembly has one or more speeds (e.g., low, medium, or high speeds) at which a motor drives the wiper arm to oscillate across the surface of the vehicle, causing the blades to wipe the surface. This technique enables control of the wiper blade assembly speed based on vibrations detected by sensors. Specifically, the wiper blade assembly speed is set based on the actual rainfall rate and droplet size calculated via acoustic emission.

[0085] In an embodiment, acoustic emission is used to determine the degradation of the wiper blade assembly. In the example, the autonomous vehicle operates for an extended period without a human driver. In some cases, physical wear and tear on the wiper blade assembly may go unnoticed until complete obstruction of the surface occurs. This technique enables the detection of wear on a wiper blade assembly, including blades used to wipe the corresponding surface. In an embodiment, this technique estimates the amount of degradation (e.g., wear) experienced by the assembly. The estimate of degradation associated with one or more wiper blades is transmitted as a control signal to indicate the condition of one or more wiper blades.

[0086] Figure 8 This is a flowchart of the process for controlling wiper blade assembly based on acoustic emission. In some aspects or embodiments, regarding... Figure 8 The processing of one or more steps (e.g., entirely and / or partially, etc.) is performed by the autonomous system of the vehicle (e.g., with...) Figure 2 One or more systems (such as autonomous systems 202 or similar to autonomous systems) may be used. Additionally or alternatively, systems different from the autonomous system (e.g., those similar to autonomous systems 202) may be used. Figure 2 The same or similar wired control system as 202h, or Figure 6 In addition to the wiper blade assembly 600, or in combination with the system (e.g., completely and / or partially, etc.) Figure 8 One or more steps in the processing.

[0087] In box 802, a sensor (e.g., piezoelectric sensor 604) mounted in the wiper blade assembly detects acoustic emissions. In this example, the acoustic emissions captured by one or more sensors are used to calculate parameters such as rainfall rate. One or more acoustic emission sensors (typically acoustic emission piezoelectric sensors) are mounted in the structure (e.g., arm) of the wiper blade assembly such that vibrations propagate to the rubber cleaning blade mounted on the structure and then to the acoustic emission sensors within the structure. When vibrations occur on the windshield or optical surface due to rainfall, these vibrations propagate throughout the windshield to the wiper blade and are subsequently sensed by the acoustic emission sensors. By using multiple (e.g., three or more) acoustic emission sensors, acoustic triangulation is used to map the vibrations on the windshield to determine the rainfall rate and amount (droplet size, frequency, and diffusion).

[0088] Therefore, in box 804, one or more parameters associated with environmental conditions are calculated. Figure 8 In the example, the rainfall rate is calculated. Based on the calculated rainfall rate, it is determined whether to activate the wiper blades based on the rainfall rate. A backup system is used to determine whether the detected vibration is a result of rainfall, in order to eliminate several false alarms detected as rain. In an embodiment, the backup system (e.g., Figure 2 The camera 202a, LiDAR sensor 202b, radar sensor 202c, and microphone 202d are typically located on the vehicle. Figure 8 In the example, the backup system is a camera system (e.g., camera 202a). In an embodiment, the camera system includes a camera located outside the vehicle and a camera located inside the vehicle. The backup system is responsible for detecting whether it is raining. Therefore, at block 806, weather data is used to verify and confirm the forecast of rain. In an embodiment, the weather data is calculated from sources such as GPS data, AV (Automatic Viewership). Figure 4 Third-party weather data extracted from other data sources such as AV Calculator 400 or any other computing system.

[0089] In block 808, the wiper rate is determined based on detected vibrations associated with rainfall, verified and confirmed by the backup system. In block 810, acoustic vibrations are measured by a sensor integrated with the wiper blade assembly, and the processing flow returns to block 804 for continuous calculation of the rainfall rate and the corresponding wiper activation rate. In this embodiment, the rainfall rate is used to guide the navigation of the vehicle. For example, under adverse or heavy rainfall rates, the operation of the autonomous vehicle is hindered. Additionally, LiDAR sensor cleaning and camera sensor cleaning can be based on the calculated rainfall rate. In this way, the technology can maintain sensor visibility in the presence of contaminants and debris in the environment.

[0090] Typically, during acoustic vibration measurements of wiper blade assemblies, the acoustic vibrations are compared to a calibrated threshold to determine the level of degradation in wiper blade performance. In an embodiment, acoustic emission is measured upon activation of the wiper blade assembly to detect wiper blade wear or degradation through detected acoustic emission events. When wear exceeding a threshold is detected, the wiper blade can be returned to its storage position, and other cleaning methods can be relied upon, such as water- and air-based sprays for removing contaminants including abrasive particles (such as sand or dust). In an embodiment, this technique is used in manually driven vehicles to prevent wiper-based removal of dust clumps that may abrade the windshield.

[0091] Therefore, vibration is also used to determine wiper blade degradation. At block 820, it is determined whether the detected vibration exceeds a calibration threshold. In an embodiment, the calibration threshold depends on the type of contaminants or debris known to be present in the collision of particles with the surface. For example, a contaminant detection system may be used to determine the composition of the current particle collision with the surface. If the acoustic vibration is determined to be less than the calibration threshold, the process continues to block 822. At block 822, it is determined that the wiper is in good condition. Specifically, the captured vibration does not indicate the presence of abrasive material on the surface.

[0092] In box 820, if it is determined that the captured acoustic vibration is greater than a calibration threshold, the process continues to box 824. In box 824, a dirt detection system is used to determine the presence of abrasive obstructions. In an embodiment, the dirt detection system calculates the type of contaminants present on the surface of the vehicle. Additionally, in an embodiment, the dirt detection system triggers a sensor cleaning cycle in response to contaminants or debris on the surface. The dirt detection system outputs whether any contamination is present on the surface and also identifies the type of contamination on the sensor. If no abrasive obstructions are found on the surface, the process continues to box 826. In box 826, it is confirmed that the obstruction is water, and the process continues to box 832, where a degradation message is generated.

[0093] If an abrasive obstruction is found on the surface, the process continues to box 828. In box 828, the abrasive obstruction is confirmed and an alternative cleaning method is initiated. In an embodiment, the alternative cleaning method includes using fluid or air to clean the abrasive obstruction from the surface. When an abrasive obstruction is present, wiping the obstruction with a wiper blade assembly may damage the wiper blade, the surface, or any combination thereof. This alternative cleaning method cleans abrasive obstructions that could damage the wiper blade and the surface without causing further damage during cleaning.

[0094] This technology detects the presence of abrasive material and measures wiper blade wear. A wear event is detected when the captured vibration corresponds to a predetermined vibration pattern associated with the level of wear on the wiper blade. In this example, the wiper blade deteriorates over time due to use. This technology applies acoustic measurements and uses alternative cleaning methods such as fluids and air to restore wiper blade functionality. If the indication of wear persists (e.g., vibration exceeds a calibration threshold), the vibration is attributed to a permanently worn wiper blade that requires replacement.

[0095] Therefore, in box 830, the wipers are activated and acoustic vibration is sensed. If the acoustic vibration is below a threshold after an alternative cleaning method, the process continues to box 810, where acoustic vibration is measured via the wiper blades since the wiper blade assembly has regained functionality. If the acoustic vibration remains above the calibration threshold, the process continues to box 832. In box 832, a degradation message is generated. In this example, the degradation message is an indicator or message displayed inside the vehicle. In this example, the degradation message is transmitted to a remote location.

[0096] For ease of description, the present technology has been described using a wiper blade assembly configured for cleaning the surface of a vehicle's windshield. However, the present technology is also used to enable an automatic cleaning cycle applied to windshields, sensor covers, or other vehicle components through which light or other signals pass. For example, windshields, headlights, taillights, and sensors include large surfaces that require periodic cleaning to increase visibility.

[0097] Figure 9 This is an example of a LiDAR assembly 900 with multiple sensors.

[0098] exist Figure 9 In the example, a LiDAR surface 902 is illustrated. A wiper blade rubber 904 contacts a housing or shield 906 containing the LiDAR electronics. In this example, the LiDAR electronics include signal leads 908, electrodes 910, and piezoelectric ceramic elements 912. A couplant layer 914 and a damping material 916 are also illustrated.

[0099] exist Figure 9In the example, the acoustic sensor is mounted in the wiper blade assembly such that the acoustic element is located directly behind the wiper blade rubber 904, which receives vibrations from the LiDAR surface and deteriorates over time. In this embodiment, the technology uses the wiper blade to remove dirt or other contaminants from the housing 906 (e.g., the surface). In this embodiment, the technology can also determine the level of contamination associated with the LiDAR housing 906. Conventionally, for a given sensor, a cleaning system senses when the sensor's field of view becomes obstructed or otherwise contaminated. The cleaning system is activated in response to contamination. The cleaning system may include, for example, washing or spraying. This technology is able to calculate the rainfall rate or the percentage of obstruction or contamination on the surface associated with the sensor and trigger a cleaning cycle for that particular sensor.

[0100] Figure 10 This is a block diagram of a process controlled by a device based on acoustic emission. In the example, the device is a cleaning device such as a wiper blade assembly, an air injector, and a heating / cooling / defrosting device. In some aspects or embodiments, one or more steps (e.g., completely and / or partially, etc.) of the process 1000 are controlled by an autonomous system of the vehicle (e.g., with...). Figure 2 The autonomous system 202 is the same as or similar to the autonomous system. Additionally or alternatively, in addition to systems different from the autonomous system (e.g., with...), Figure 2 The same or similar wired control system as 202h, or Figure 6 One or more steps of processing 1000 may be performed in addition to (e.g., completely and / or partially, etc.) the wiper blade assembly 600.

[0101] In block 1002, information associated with the first, second, and third acoustic emission sensors is received. In an embodiment, this information corresponds to an event (e.g., a collision between a particle and a surface). Typically, the acoustic emissions output by the first, second, and third acoustic emission sensors are used to determine the degradation of the wiper blade assembly.

[0102] In block 1004, a first timestamp associated with first acoustic emission sensor information (e.g., the time when a valid event (collision between a particle and the surface of a vehicle) is detected from the timing system) is received, a second timestamp associated with second acoustic emission sensor information is received, and a third timestamp associated with third acoustic emission sensor information is received.

[0103] In box 1006, a unit vector is calculated from each acoustic emission sensor in the direction of the origin of the event (e.g., the location where the particle collides with the surface) based on the information from the first acoustic emission sensor, the information from the second acoustic emission sensor, the information from the third acoustic emission sensor, the first timestamp, the second timestamp, the third timestamp, and the geometry of the surface.

[0104] In block 1008, parameters associated with environmental conditions are calculated based on unit vectors (e.g., droplet size, droplet spacing, rainfall rate, droplet removal rate, debris). In this embodiment, acoustic emission sensor information and unit vectors are used to estimate the amount of degradation present at the wiper blade assembly.

[0105] In box 1010, the control system (e.g., Figure 4 408 control system Figure 5 The control system 504b) is configured to operate the vehicle based on parameters. In the example, an AV including a wiper blade assembly, as described herein, operates for an extended period without physical human visual inspection. During these periods, surface damage or obstruction may occur. This technique detects surface damage or obstruction and controls the operation of the wiper blade assembly based on the detected surface damage or obstruction.

[0106] In some examples, the wiper blade assembly is operated based on an estimated amount of degradation indicating the condition of the wiper blades. In response to the wiper blade degradation exceeding a predetermined threshold, an alternative cleaning method is used to operate the AV. In embodiments, the detected acoustic emission information allows the information captured at the AV to be used to determine environmental conditions and the degradation of the wiper blade assembly. In particular, the determination of environmental conditions at the AV enables real-time, fine-grained control of the AV system in response to the determined environmental conditions. Furthermore, determining the degradation of the wiper blade assembly allows mitigation techniques to be applied even when the degradation is not noticed by humans.

[0107] In the preceding description, aspects and embodiments of this disclosure have been described with reference to numerous specific details, which may vary from implementation to implementation. Therefore, the specification and drawings should be considered illustrative rather than restrictive. The sole and exclusive indication of the scope of this invention, and what the applicant expects to be the scope of this invention, is the literal and equivalent scope of the claims published from this application in the specific form of the published claims, including any subsequent amendments. Any definitions of terms expressly set forth herein for inclusion in such claims should be taken as meaning as such terms are used in the claims. Furthermore, when the term “comprising” is used in the preceding specification or appended claims, what follows that phrase may be an additional step or entity, or a sub-step / sub-entity of a previously stated step or entity.

Claims

1. A method for a vehicle, comprising: detecting, with at least one processor, vibrations generated from particles colliding with a surface of the vehicle, wherein a first acoustic emission sensor is located at a first location on an arm of a wiper blade assembly, a second acoustic emission sensor is located at a second location on the arm, and a third acoustic emission sensor is located at a third location on the arm; receiving, with the at least one processor, first acoustic emission sensor information from the first acoustic emission sensor, second acoustic emission sensor information from the second acoustic emission sensor, and third acoustic emission sensor information from the third acoustic emission sensor corresponding to an event; receiving, with the at least one processor, a first timestamp associated with the first acoustic emission sensor information, a second timestamp associated with the second acoustic emission sensor information, and a third timestamp associated with the third acoustic emission sensor information; calculating, with the at least one processor, unit vectors from each acoustic emission sensor in a direction of an origin of the event based on the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information, the first timestamp, the second timestamp, the third timestamp, and a geometry of the surface; calculating, with the at least one processor, a parameter associated with an environmental condition based on the unit vectors; and operating, with the at least one processor, the vehicle based on the parameter.

2. The method of claim 1, comprising: determining an abrasive wear state of a wiper blade based on the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information.

3. The method of claim 1 or 2, wherein, calculating a parameter based at least in part on triangulation applied to the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information.

4. The method of any one of claims 1 to 3, wherein, the first timestamp, the second timestamp, and the third timestamp are obtained from a timing system.

5. The method of any one of claims 1 to 4, comprising: controlling a rate of the wiper blade assembly based on the parameter associated with an environmental condition.

6. The method of any one of claims 1 to 5, comprising: verifying the calculated parameter via comparison with weather data.

7. The method of any one of claims 1 to 6, comprising: determining a particle type based on the first acoustic emission sensor information, the second acoustic emission sensor information, and the third acoustic emission sensor information.

8. The method of claim 7, comprising: using an alternative cleaning method in response to determining that an abrasive particle type has collided with the surface.

9. A vehicle, comprising: at least three acoustic emission sensors configured to detect vibrations generated from particles colliding with a surface of the vehicle, wherein a first acoustic emission sensor is located at a first location on an arm of a wiper blade assembly, a second acoustic emission sensor is located at a second location on the arm, and a third acoustic emission sensor is located at a third location on the arm; at least one computer-readable medium storing computer-executable instructions; and at least one processor communicatively coupled to the at least three acoustic emission sensors and configured to execute the computer-executable instructions, the execution performing the method of any of claims 1-8.

10. The vehicle of claim 9, further comprising another control system that operates the wiper blade assembly based on the parameter.

11. A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a first device, the at least one program comprising instructions, which, when executed by the at least one processor, cause the first device to perform the method of any one of claims 1 to 8.

Images