Systems and methods for a sensor protection system

Abstract

This teaching relates to methods and systems for sensor protection. The sensor protection system includes an assembly having a first structure embedded in a second structure, wherein a sensor is firmly placed in the first structure for sensing information through the assembly and being protected from the negative impact of the external environment, and one or more devices mounted on the second structure for providing means to clean an inclined surface of the second structure through which the sensor senses surrounding information, wherein the one or more devices can be activated individually or jointly as needed to clean the inclined surface so as to prevent degradation of the information sensed by the sensor through the assembly.

Description

Technical Field

[0001] The present teachings generally relate to sensors. More specifically, the present teachings relate to sensor protection assemblies. Background Art

[0002] With the advancement of sensing technology, automation in different industries relies on advanced sensing technology to provide information about the surrounding environment of an automation site, which forms the basis for various computerized decisions. For example, various sensors are deployed on different automated assembly lines in different manufacturing sites to provide key information for the robots working in these sites to work correctly. As another example, driverless vehicles are an emerging field where sensing technology is necessary to help the computer system in a moving vehicle make correct vehicle control decisions in dynamic situations. In such applications, multiple modalities of sensors can be deployed in different parts of the vehicle to continuously provide observations of the surrounding environment of the moving vehicle. Such observations can include visual, auditory, and 3D depth information. For example, a moving vehicle needs to clearly and accurately "see" what obstacles are in its field of view and needs to determine various relevant parameters associated with each observed obstacle. For example, an autonomous vehicle needs to determine the pose of each obstacle in its field of view, whether each such obstacle is in motion, the speed of each such obstacle, and how far each such obstacle is from the moving vehicle at each moment. These parameters need to be obtained based on continuous, clear, and accurate information from sensors so that the computer system can successfully implement obstacle avoidance in real time. Thus, it is necessary to ensure that the sensors deployed on an autonomous driving vehicle continuously provide accurate and uninterrupted performance. Summary of the Invention

[0003] The teachings disclosed herein relate to methods, systems, and programming for data processing. More specifically, the present teachings relate to methods, systems, and programming related to modeling a scene to generate scene modeling information and utilizing the same.

[0004] In one example, a method for creating a sensor protection system is disclosed. An assembly having a first structure embedded in a second structure is provided, where a sensor is securely placed in the first structure for sensing information through the assembly and being protected from the negative effects of the external environment. One or more devices are mounted on the second structure for providing means to clean an inclined surface of the second structure, through which the sensor senses surrounding information, where the one or more devices can be activated individually or jointly as needed to clean the inclined surface so as to prevent degradation of the information sensed by the sensor through the assembly.

[0005] In various examples, the present teachings disclose a sensor protection system that includes an assembly having a first structure embedded in a second structure, where a sensor is firmly placed in the first structure for sensing information through the assembly and being protected from the negative effects of the external environment, and one or more devices mounted on the second structure for providing means to clean an inclined surface of the second structure, through which the sensor senses surrounding information, where the one or more devices can be activated individually or jointly as needed to clean the inclined surface so as to prevent degradation of the information sensed by the sensor through the assembly.

[0006] Additional advantages and novel features will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following description and the accompanying drawings, or may be learned by the production or operation of the examples. The advantages of the present teachings may be realized and obtained by practicing or using various aspects of the methods, means, and combinations set forth in the detailed examples discussed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The methods, systems, and / or programming described herein will be further illustrated by way of example embodiments. These example embodiments are described in detail with reference to the accompanying drawings. These are non-limiting example embodiments, where in the multiple figures of the drawings, like reference numerals represent similar structures, and in the drawings:

[0008] Figure 1 Illustrates an autonomous vehicle on which a sensor is deployed according to an embodiment of the present teachings;

[0009] Figure 2A Represents an example type of sensor deployed on an autonomous vehicle according to an embodiment of the present teachings.

[0010] Figure 2B Describes example different sensor mounts installed on different parts of a vehicle to provide surrounding information to assist in autonomous driving according to an embodiment of the present teachings;

[0011] Figures 3A - 3B Represents an example configuration of attaching an inertial measurement unit (IMU) to a sensor according to an embodiment of the present teachings;

[0012] Figures 4A - 4B Represents an example embodiment of deploying a sensor with an attached IMU on an autonomous vehicle according to an embodiment of the present teachings;

[0013] Figure 5 Represents a fleet of autonomous vehicles each having sensors mounted thereon to assist in autonomous driving according to an embodiment of the present teachings;

[0014] Figure 6Describe an exemplary sensor bracket on which different types of sensors are mounted in accordance with an embodiment of the present teachings;

[0015] Figure 7A Describe an exemplary conceptual sensor bracket in which sensors of different modalities are mounted in accordance with an embodiment of the present teachings;

[0016] Figure 7B Describe an exemplary conceptual sensor bracket having sensors mounted therein, with one or more IMUs attached to the sensor bracket in accordance with an embodiment of the present teachings;

[0017] Figure 8A Describe an anti-tampering mechanism of a sensor bracket having an IMU attached thereto and connected to an in-vehicle anti-tampering system in accordance with an embodiment of the present teachings;

[0018] Figure 8B Is a flowchart of an exemplary process of an anti-tampering mechanism in accordance with an embodiment of the present teachings;

[0019] Figure 9A Describe an exemplary high-level system diagram of an IMU in accordance with an embodiment of the present teachings;

[0020] Figure 9B Is a flowchart of an exemplary process of an IMU in accordance with an embodiment of the present teachings;

[0021] Figure 10A Describe an exemplary high-level system diagram of an in-vehicle anti-tampering system in accordance with an embodiment of the present teachings;

[0022] Figure 10B Is a flowchart of an exemplary process of an in-vehicle anti-tampering system in accordance with an embodiment of the present teachings;

[0023] Figure 11A Show a camera lens with dirt or debris thereon and the resulting misleading image;

[0024] Figure 11B Show an exemplary environmental protection sensor housing assembly in accordance with an embodiment of the present teachings;

[0025] Figures 11C - 11F Show different aspects of an environmental protection sensor housing assembly in accordance with an embodiment of the present teachings;

[0026] Figure 12A Illustrate an exemplary embodiment for delivering or spraying a cleaning fluid to a part of a housing assembly in accordance with an embodiment of the present teachings;

[0027] Figure 12B Illustrate an exemplary embodiment of a wiper assembly having a wiper blade for cleaning a part of a housing assembly in accordance with an embodiment of the present teachings;

[0028] Figure 12C Illustrative embodiments in accordance with the teachings herein for delivering air to a portion of a housing assembly through a hair dryer assembly having an air hose are illustrated;

[0029] Figure 13A Illustrative mechanisms in accordance with the teachings herein related to an environmental protection sensor housing assembly and an environmental protection sensor system controller are described;

[0030] Figure 13B is a flowchart of an illustrative process of an environmental protection sensor system controller in accordance with an embodiment of the teachings herein;

[0031] Figure 14A Illustrative reasons leading to a need to clean a sensor assembly and an illustrative configuration for such cleaning in accordance with an embodiment of the teachings herein are shown.

[0032] Figure 14B An illustrative high-level system diagram of a sensing quality control unit in accordance with an embodiment of the teachings herein is described;

[0033] Figure 14C is a flowchart of an illustrative process of a sensing quality control unit in accordance with an embodiment of the teachings herein;

[0034] Figure 15 The illustrative structure of an environmental protection sensor bracket assembly in accordance with an embodiment of the teachings herein is described;

[0035] Figure 16A Illustrates an arcuate frame-like structure containing different cleaning tools in accordance with an embodiment of the teachings herein;

[0036] Figure 16B An illustrative embodiment of an environmental protection sensor bracket in accordance with an embodiment of the teachings herein having individual sensors contained therein and protected using an environmental protection assembly based on separate segments is described;

[0037] Figures 17A - 17B Illustrative embodiments of glare protection with respect to a housing assembly and / or a sensor bracket in accordance with an embodiment of the teachings herein are illustrated;

[0038] Figure 18 is an illustrative diagram of an architecture of a mobile device that can be used to implement a dedicated system for practicing the teachings herein in accordance with various embodiments;

[0039] Figure 19 is an illustrative diagram of an architecture of a computing device that can be used to implement a dedicated system for practicing the teachings herein in accordance with various embodiments. Detailed Description

[0040] In the following detailed description, numerous specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to those skilled in the art that the teachings may be practiced without these details. In other instances, well-known methods, procedures, components, and / or circuits have been described at a relatively high level without detail in order to avoid unnecessarily obscuring aspects of the teachings.

[0041] The present teachings are directed to solving deficiencies in the prior art with respect to the sensing quality and accuracy of autonomous vehicles. As described herein, in order to ensure safe autonomous driving, it is necessary for sensors to reliably provide accurate information about the surroundings of the vehicle. The reliability requirement ensures the presence of sensors installed in their intended locations. Since a vehicle may be exposed to the public (e.g., when parked on a street or in a parking lot), it is important to include means to prevent tampering with the sensors on the vehicle. The accuracy of the information obtained requires such sensors to be in a state capable of truly and unambiguously acquiring information about the surrounding environment. In order to sense the intended information, most sensors are installed such that they are exposed to an open space, subject to rain, snow, dust, sunlight, and debris often from different sources. The present invention discloses different means and their embodiments for solving the reliability and accuracy problems.

[0042] Figure 1 FIG. illustrates an autonomous vehicle 100 in accordance with an embodiment of the present teachings, on which different sensors are deployed. In this illustrative embodiment, the autonomous vehicle is a truck (without a trailer) having sensors mounted on the top and front of the vehicle. For example, as Figure 1 illustrated, the exemplary sensors include stereo cameras 110 and 130 on the top of the truck 100 for observing obstacles located in front of the vehicle, a LiDAR sensor 120 for providing depth information of the forward field of view, a radar sensor 150 at the front of the truck, and additional cameras 140 and 150 for observing road conditions. These exemplary sensors are installed in their designated locations and are each responsible for a specific role to collect a specific type of information beneficial for the vehicle to make decisions related to autonomous driving. For example, the stereo camera pair 110 and 130 may be designated to not only view the scene (objects such as trees, road signs, buildings, other obstacle vehicles, and lane markings), but also estimate the depth of the observed objects. The LiDAR sensor 120 may be designated to measure the depth information of the scene in front of the vehicle 100 and provide a depth map. The radar sensor 150 may be used to sense obstacles at a low height to detect any obstacles on the road surface in front of the vehicle. The cameras 140 and 160 may be used to detect obstacles approaching the road surface, such as lane markings immediately in front of the vehicle.

[0043] Different types of sensors may be deployed on an autonomous vehicle. Figure 2AIllustrative types of sensors that can be deployed on an autonomous vehicle in accordance with embodiments of the present teachings. The sensors can be of multiple modalities (200). Some sensors may be oriented for active sensing (210), some may be oriented for environmental sensing (220), and some may be for passive sensing (230). Examples of active sensors can include radar sensors (210-1),..., LiDAR sensors (210-2) that actively act (transmit and measure) to collect the required information. Examples of passive sensors include photographic sensors, such as cameras (230-2) or thermal sensors (230-1), that collect data on whatever is printed on them. Environmental sensing can be based on information from one or more sensors, and what is sensed is information about the environment, such as lighting conditions (220-1), weather conditions such as rain or snow (220-2), road conditions such as wet or snow-covered (220-3), driving-related parameters such as road grade (220-4),..., and traffic conditions such as road congestion (220-5).

[0044] Different sensors responsible for collecting specified information can be strategically installed in appropriate parts of the vehicle. This is shown in Figure 2B where, in accordance with embodiments of the present teachings, different sensor mounts are installed in different parts of the vehicle to provide sensing information to assist in autonomous driving. Each sensor mount can have one or more sensors installed therein. In this illustrative embodiment, there are four visible sensor mounts 250, 260, 270, and 280, which are installed at different locations on the truck 240, at the top (250), front lower part (260), front side (270), and right side (280) of the truck, respectively. There may be parallel or corresponding sensor mounts 270 and 280 installed on opposite sides of the truck (not shown). Each mount can also be installed with different types of sensors. For example, the sensors in mount 250 can be used for long-range obstacle observation in terms of visually observable objects and their depth measurements. Sensor mount 260 can include sensors responsible for detecting obstacles closer to the vehicle 240. The sensor mounts installed on the sides of the truck can be used to observe obstacles on both sides of the vehicle 240. The rear of the truck can also have a mount with sensors that are designated to observe objects behind the vehicle. In this way, the truck is made aware of its surroundings so that any autonomous driving decisions can be made in accordance with the surrounding situation.

[0045] When these sensor brackets are installed on a vehicle in their respective roles to collect the required information, it is important that these sensors are always present when the vehicle is moving. Since the sensors and sensor brackets are installed externally, there is reason to worry that these sensors and sensor brackets may be illegally tampered with or even stolen, rendering the vehicle unable to drive itself. According to some embodiments of the present teachings, an inertial measurement unit or IMU can be used to implement an anti-tampering mechanism. Such an anti-tampering mechanism can be implemented for both individual sensors and sensor brackets. Figures 3A - 3B An exemplary anti-tampering mechanism for a single sensor using an IMU is shown according to an embodiment of the present teachings. Figure 3A It is shown that the IMU 310 is attached to the sensor 110 such that the sensor 110 is located on top of the IMU 310. Figure 3B A different configuration is shown in which the IMU 320 is attached to the top of the sensor 110. Such an IMU unit is capable of detecting various types of states associated with it, including its own attitude, the vibrations it experiences, and other types of information that the IMU detects. The IMU is also configured to be able to report such detected information such that any interference that causes it to deviate from its initial state can be used to detect a tampering event. In some embodiments, the IMU can also be attached to other parts of the sensor, for example, attached to its side (not shown). Generally, the IMU can be attached to any position of the sensor and continuously report information related to the state of the sensor.

[0046] In both of these configurations, the IMU and the sensor are attached to each other in such a way that when the sensor is moved from a fixed attitude, the structure (sensor plus IMU) will necessarily interfere with the attitude of the IMU. That is, the movement of the sensor causes the movement of the IMU. The attachment between the IMU and the sensor, or between the IMU and the sensor housing assembly, can be achieved directly or indirectly by different means. In an embodiment, a mechanical and / or structural mechanism can be implemented for attachment, such as but not limited to strong glue, adhesives, one or more mounting frames, one or more magnets, one or more plates, fasteners, connectors (including lugs, snaps, and corresponding mating devices (e.g., male-female connectors)), and / or combinations thereof. In an embodiment, the IMU and the sensor can be firmly mounted within a housing or box so as to be attached to each other. Once attached to each other, the structure containing the sensor and the IMU can be mounted on the vehicle as a unit, as Figure 4A and 4B shown respectively. Figure 4A It is shown that the sensor 110 with the IMU 310 attached to it in the configuration as shown in Figure 3A is a structure to be subsequently mounted on top of the vehicle 100. Figure 4B It is shown that the sensor 110 and the IMU 320 are in the configuration as shown in Figure 3BThe components shown in [description] are attached together as a structure that is subsequently installed on top of vehicle 100. To move sensor 110 from vehicle 100, the entire structure can be moved because each of sensor 110 and the IMU is not separately detachable. With this mechanism, when the vehicle is in a parked state, as shown in [reference], if the sensor installed on such a parked vehicle is disturbed, the attached IMU will sense the disturbance in attitude, and subsequent actions such as sounding an alarm can be taken. Details will be discussed with reference to [reference]. Figure 5 As shown in [reference], if the sensor installed on such a parked vehicle is disturbed, the attached IMU will sense the disturbance in attitude, and subsequent actions such as sounding an alarm can be taken. Details will be discussed with reference to [reference]. Figures 8A - 10B discussed.

[0047] As described in the text, sensors deployed on autonomous vehicles can also be installed on sensor brackets. Figure 6 Describing an exemplary sensor bracket according to an embodiment of the present teachings, on which different types of sensors are installed. As shown in the figure, the sensor bracket 600 has different types of sensors installed thereon, such as including a stereo pair camera (left and right stereo cameras), a LiDAR sensor, a wide dynamic range camera, a long-range camera, a night vision camera, and a rearview camera, etc. When such a sensor bracket is installed on a vehicle, each of the sensors installed therein can be deployed to perform their respective sensing functions. Such a sensor bracket may also be subject to tampering or theft. To prevent such tampering, the disclosed method of using an IMU unit can also be applied to the sensor bracket.

[0048] Figure 7A Describing an exemplary conceptual sensor bracket 700 according to an embodiment of the present teachings, in which multiple sensors of different modalities are installed. Although only three sensors of two different modalities are shown in the figure, just two types of sensors, this is only for illustration and not as a limitation. As shown in [reference], an increasing number of types of sensors can be installed in a single sensor bracket. In the embodiment illustrated in [reference], there are two cameras (110 and 130, which can be a stereo pair) and one LiDAR sensor (120). Such a sensor bracket can be installed on a vehicle, and measures need to be taken to prevent tampering. Figure 6 As shown in [reference], an increasing number of types of sensors can be installed in a single sensor bracket. In the embodiment illustrated in [reference], there are two cameras (110 and 130, which can be a stereo pair) and one LiDAR sensor (120). Such a sensor bracket can be installed on a vehicle, and measures need to be taken to prevent tampering. Figure 7A In the embodiment illustrated in the figure, there are two cameras (110 and 130, which can be a stereo pair) and one LiDAR sensor (120). Such a sensor bracket can be installed on a vehicle, and measures need to be taken to prevent tampering. Figure 7B Describing an exemplary conceptual sensor bracket 700 according to an embodiment of the present teachings, having sensors installed therein, while one or more IMUs are attached to the bracket for tampering detection. In this exemplary embodiment, multiple IMUs are attached to the sensor bracket 700. Generally, one or more IMUs can be attached to different positions of the sensor bracket 700. In this embodiment, two IMUs (720, 730) are placed on top of the sensor bracket 700, and one IMU is attached to either end of the sensor bracket 700 (710 and 740). In this exemplary embodiment, the IMUs with respect to the sensor bracket 700 function in the same way as in [reference]. Figures 3A - 4BThe IMUs attached to the respective sensors shown have the same function. That is, if the sensor bracket 700 is tampered with, this will cause the IMUs to detect a deviation from their initial attitude, and this detected deviation can be used to detect tampering.

[0049] To utilize the detected attitude deviation of the IMUs to detect tampering, the IMUs attached to the sensors or sensor brackets can communicate with a tamper - proof mechanism, which can be configured to react to any signals from the IMUs and take certain actions to prevent tampering. Figure 8A Describe such a tamper - proof mechanism for a sensor bracket according to an embodiment of the present teachings. The sensor bracket has an IMU attached thereto and connected to an in - vehicle tamper - proof system 800. In this illustrative embodiment, once in operation, each exemplary IMU attached to the sensor bracket 700 continuously checks its attitude, status, and vibration, and reports the detected data. The checks can be carried out continuously at certain configured intervals. The IMU communicates with the in - vehicle tamper - proof system 800 so that the IMU reports the detected attitude, status, and vibration to the in - vehicle tamper - proof system 800.

[0050] Depending on the information received from the IMUs, the in - vehicle tamper - proof system 800 determines whether there are any tampering events. Such events can be associated with the severity of the detected data (attitude changes, vibrations) from the IMUs. For example, if the reported attitude deviation and / or vibration exceed a certain specified range, then the in - vehicle tamper - proof system 800 can consider that there is a tampering event. To react to the detected tampering event, the in - vehicle tamper - proof system 800 can activate its internal mechanism to trigger an alarm. In some embodiments, the in - vehicle tamper - proof system 800 can also send an alarm signal to some remote sites according to certain configurations. In some embodiments, once the alarm is triggered, the in - vehicle tamper - proof system 800 can not only trigger a loud alarm in the vehicle, but also activate the appropriate (which can be all) sensors to start recording the vehicle's surrounding environment, so as to capture any information associated with the detected tampering event. This recorded data from the activated sensors can also be sent to the remote site via the connection to the remote site.

[0051] In some embodiments, the data reported from the IMU (attitude, status, and vibration) can also be used to initiate other tasks. For example, in order to rely on sensor data to control autonomous driving, the sensors on the vehicle need to be properly calibrated. For example, in order to detect the depth of an observed obstacle, a stereo camera can be deployed and calibrated such that images from the left and right cameras in the stereo pair can be used to compute a depth map. The accuracy of the estimated depth information can be a function of proper calibration. Due to driving, the attitude of the sensors may change due to, for example, vibrations of the vehicle. Such changes may lead to inaccurate stereo-based depth estimation, thus posing problems for autonomous driving. In such cases, a recalibration based on the new (shifted) attitude of the two cameras may be required. The need for such recalibration can be detected based on the detected attitude information from the IMU and the expected attitude of the calibrated cameras. In some cases, if the attitude of the camera has changed to the extent that the sensor needs to be reinstalled, then the information from the IMU enables the on-vehicle system 800 to detect this situation and alert the driver of the vehicle to take action regarding the installation of the sensor. Thus, the IMU can be used not only for tampering detection purposes but also for sensing the need for reinstallation and / or recalibration of some sensors.

[0052] Figure 8B is an illustrative process flow of the anti-tampering mechanism according to an embodiment of the present teachings. At step 810, the IMU detects attitude, status, and vibration respectively and sends this information to the on-vehicle anti-tampering system 800. When receiving the information from the IMU, at step 820, the on-vehicle anti-tampering system 800 determines whether the detected vibration from any IMU is greater than a threshold. When the vibration level exceeds the threshold, the on-vehicle system 800 regards it as a sensor tampering event. Thus, the first test to be conducted is the level of vibration. The vibration threshold for detecting a tampering event can be pre-determined or dynamically determined. For example, the threshold can be set based on the average level of vibration that the vehicle has experienced in the last fixed time period. If the vehicle has been driving on a bumpy road, then the average level of vibration may be high, such that the threshold indicating the vibration level for tampering can be set high. In one embodiment, the system can be configured such that when the vehicle is in motion, the threshold is set so high that no vibration will trigger a detected tampering event. In one embodiment, when the vehicle is in motion, the on-vehicle anti-tampering system 800 can be configured not to act on the detected vibration, thus effectively not detecting a tampering event. That is, the on-vehicle anti-tampering system will only operate when the vehicle is in the parking mode. In this mode, the threshold for vibration can be set quite low such that any unexpected vibration can be regarded as a tampering event. Figure 8A

[0053] ​If, at step 820, it is determined that the vibration exceeds the threshold, then, in process 860, the vehicle anti-tampering system 800 triggers an alarm inside the vehicle. Such an alarm may include certain loud siren-like sounds emitted via, for example, a speaker in the vehicle. Additionally, at step 870, the vehicle anti-tampering system 800 may also determine that some sensors are activated to record surrounding information. For example, based on which IMU reported a high level of vibration, the sensors located near that IMU may be activated. In some embodiments, the vehicle anti-tampering system 800 may activate more sensors to record information, such as all sensors on the same side of the vehicle where the IMU is located, or even all sensors deployed on the vehicle to record the observed information. In some embodiments, the sensors to be activated to record information may be of certain types, such as visual sensors and acoustic sensors (without LiDAR or radar sensors). At step 880, the determined sensors are activated to collect surrounding information. In some embodiments, the vehicle anti-tampering system 800 may be configured to notify a remote site of the detected tampering event by sending the alarm (of the tampering event) and the information collected from the activated sensors to the remote site.

[0054] If the vibration levels from all IMUs are below the threshold, then it may be indicated that there is no tampering event. In such a case, the vehicle system may proceed to detect whether there are any sensors that need to be reinstalled or recalibrated. As described herein, the sensors installed on the vehicle may have been previously calibrated based on their installation positions. During driving and / or over time, the installation may become loose due to, for example, the vibration of the vehicle, resulting in a change in its attitude and thus causing inaccuracies in detecting obstacles. When the change in the sensor attitude is small, recalibration can be performed, which can be done on-the-fly. In some cases, if the change in the sensor attitude is too large, then on-the-fly calibration may not be able to restore the accuracy (e.g., when the left camera in a stereo pair becomes severely tilted). In such a case, reinstallation may be required.

[0055] To detect different situations, at step 830, the information received from the IMUs, i.e., the attitude / status, can be compared against the known or expected (correct) attitude / status of each sensor. Based on the comparison result, at step 840, the vehicle system 800 determines whether any sensor needs to be reinstalled. If any sensor needs to be reinstalled, then at step 895, the vehicle system 800 sends an alarm to the driver indicating which sensor needs to be reinstalled. Such an alarm can be sent to an interface with the driver, such as the dashboard, or it can also be communicated via voice, or entered into a log of all maintenance tasks required for the vehicle. After reminding the driver of the required reinstallation, at step 850, the vehicle system 800 continues to check the need to recalibrate any sensors based on the comparison result.

[0056] If it is determined in step 840 that the comparison result indicates that reinstallation is not required, then the vehicle system 800 directly proceeds to step 850 to check whether recalibration is needed. The need for recalibration can be determined based on some predetermined criteria. In some embodiments, such criteria can be defined based on the deviation of the currently detected pose from the known and expected (correct) poses of different types of sensors. For example, for a pair of stereo cameras, if the transformation matrix in the stereo calculation assumes that the left and right cameras are horizontal, then a slight decrease in the height of one camera may result in a significant error in the estimated depth. In such a case, even if the deviation is not significant, recalibration may be required. On the other hand, if the height of the LiDAR sensor is decreased, this may not have much impact as long as the field of view of the LiDAR camera can still cover the critical areas in front of the vehicle. If no sensor requires recalibration, then the vehicle system 800 proceeds to step 810 to receive information (pose / status / vibration) from the IMU. If any sensor requires recalibration, then in step 855, the vehicle system 800 initiates the corresponding recalibration process and then proceeds to step 810 to continuously receive information from the IMU.

[0057] Figure 9A Illustrative high-level system diagram of an IMU according to an embodiment of the present teachings. In some embodiments, the IMU (such as 310) is a reporting device configured to merely detect self-awareness information and send out such collected information. In this illustrative embodiment, the IMU 310 includes a vibration detector 910, a status detector 930, an attitude detector 940, and a communication unit 920. The vibration detector 910 is configured to detect the vibrations experienced by the IMU. The status detector 930 is configured to detect the current status of the IMU, such as the current operating status of the IMU. The attitude detector 940 is configured to detect the current attitude of the IMU, which may include the latitude, longitude, and orientation of the IMU. The detection by each detector can be performed based on a certain pre-configured schedule stored in the detection schedule 960. The detection information from each detector can then be sent to the communication unit 920, and the communication unit 920 then forwards the received detection results to the vehicle anti-tampering system 800.

[0058] Figure 9BFIG. 0 is a flow chart of an illustrative process of an IMU according to an embodiment of the present teachings. The IMU may be configured to detect different types of information based on a schedule having certain predetermined detection intervals. To control the detection activities, the vibration unit 910, the status detector 930, and the attitude detector 940 may access the detection schedule stored in 960 at step 905 and then determine at step 915 whether it is time for each detector to detect the corresponding information based on the schedule. In some embodiments, the schedule for each detector may be configured differently. For example, the predetermined time interval for detecting vibration may be less than the time interval for detecting status and / or attitude. The predetermined time interval for detecting each type of information may be set according to, for example, what the corresponding information means and the importance of the information for tampering activities. For example, vibrations above a certain threshold may indicate an ongoing tampering event, making it very important to detect vibrations in a timely manner. On the other hand, since the attitude of the IMU may change with the movement of the vehicle and / or the condition of the road on which the vehicle is located, the information related to the change in attitude may or may not be associated with a tampering event.

[0059] In some embodiments, the schedule for each detector may also be configured dynamically. For example, when the vehicle is in motion, based on the assumption that the likelihood of tampering is lower when the vehicle is in motion, the predetermined intervals for different detectors may be set longer. The intervals may also be set based on the speed range of the vehicle movement. For example, the higher the speed, the larger the interval, and the lower the speed, the smaller the interval. In an extreme case, when the speed of the vehicle reaches zero (parking state), the predetermined interval may be reset to a smaller value because tampering is more likely to occur when the vehicle is in the parking state.

[0060] Once it is determined according to the corresponding detection schedule that a particular type of information needs to be detected, the relevant detector proceeds to detect the specified information. For example, if it is determined according to the schedule associated with vibration that it is time to detect vibration, then the vibration detector 910 proceeds to detect the level of vibration that the IMU is experiencing at step 925. Similarly, if it is determined according to the schedule associated with attitude information that it is time to detect the attitude of the IMU, then the attitude detector 940 proceeds to detect the current attitude of the IMU at step 935. If it is determined according to the schedule associated with vibration that it is time to detect the status of the IMU, then the status detector 930 proceeds to detect the status associated with the IMU at step 935. The information detected by one or more detectors in the IMU (whether vibration, attitude, or status) is then sent to the communication unit 920, which is responsible for transmitting the information so detected to the on-vehicle anti-tampering system 800 at step 955.

[0061] Figure 10AIllustrative high - level system diagram of an in - vehicle anti - tampering system 800 according to an embodiment of the present teachings. As described herein, the in - vehicle anti - tampering system 800 can be deployed inside a vehicle and is configured to interface with an IMU to receive vibration / attitude / state - related detection information sensed by the IMU. Subsequently, based on the received information, it determines whether a tampering event exists. If the in - vehicle anti - tampering system 800 estimates that an ongoing tampering event is taking place, then the in - vehicle anti - tampering system 800 can then trigger an alarm, emit a loud alarm sound, activate sensors for collecting / storing observations from sensors around the IMU, the reported data of which leads to the determination of the tampering event, and / or forward an alarm about the sensed tampering event and the observed surrounding information from the activated sensors to a remote location (e.g., the vehicle's owner or a fleet control center).

[0062] To achieve the desired functionality, the in - vehicle anti - tampering system 800 includes an update controller 1000, a sensor - specific operation controller 1020, an alarm communication unit 1030, a sensor installation determiner 1010, a sensor calibration determiner 1050, and a sensor calibration scheduler 1040. Figure 10B Is a flowchart of an illustrative process of the in - vehicle anti - tampering system 800 according to an embodiment of the present teachings. In operation, the update controller 1000 first receives IMU readings from all IMUs at step 1005. To detect any event, the update controller 1000 can check the vehicle status information stored at 1060 at step 1015. Such stored information can include various information about the status of the vehicle (e.g., whether the vehicle is currently in motion, the speed of motion, etc.), configuring the functionality of the in - vehicle anti - tampering system 800. In some embodiments, through such configuration information, the in - vehicle anti - tampering system 800 can be configured in such a way that, for example, based on the assumption that no tampering activity is possible when the vehicle is in motion, IMU readings can be ignored whenever the vehicle is in motion. In some embodiments, an alternative configuration can also be provided to allow the in - vehicle anti - tampering system 800 to continue monitoring IMU readings even when the vehicle is in motion, so as to detect different events caused by tampering and disposal of any sensor, which may make it necessary to alarm, reinstall, or recalibrate certain sensors.

[0063] If the configuration indicates that update controller 1000 is to ignore the IMU readings, then the processing loop returns to step 1005 to continue receiving IMU readings from the IMU and to check whether such received data should be further processed. When it is determined at step 1025 that the IMU data needs to be processed (or cannot be ignored), update controller 1000 compares the received vibration information associated with each sensor to which the IMU is attached with the vibration thresholds stored at 1002 and determines at step 1017 whether there is a tampering event for that sensor. As described herein, such vibration thresholds may be preset as static parameters or may be adaptively adjusted based on dynamic conditions. For example, when the vehicle is in motion, the threshold may be set higher, while when the vehicle is in a parked state, the threshold may be set lower so that slight vibrations of the IMU associated with a sensor or sensor mount can be detected as a tampering event.

[0064] When the vibration level reported by the IMU associated with a sensor exceeds the vibration threshold 1002, update controller 1000 may trigger an alarm 1007 at step 1025, thereby generating some loud alarm sound. Additionally, update controller 1000 may also call the sensor-specific operation controller 1020 to activate selected appropriate sensors at step 1027 to start recording their respective surrounding information. The sensors selected to record the information around the sensor that is believed to be tampered with may correspond to sensors that can be used to capture relevant information (visual information, auditory information, etc.) associated with the tampering event. To identify such sensors, sensor-specific operation controller 1020 may access the sensor configuration information stored in sensor information storage device 1080 to determine the physically proximate sensors, the nature of each such proximate sensor, the field of view of such sensors, etc., and select appropriate such sensors to record the surrounding information.

[0065] Based on the selected sensors, sensor-specific operation controller 1020 sends activation signals to the selected sensors, receives sensing signals from such sensors, and provides the received sensing information to alarm communication unit 1030. When the vehicle is in a parked state, typically the sensors are off and do not actively record the surrounding information. In the operating mode discussed herein, when a tampering event is detected, the respective sensors around the detected tampering location can be activated in the manner described to capture multi-modal sensor data associated with the tampering event. Such captured surrounding information may be stored in the on-vehicle anti-tampering system 800 (not shown) or may be sent to a remote location at step 1035 for evidence preservation. The recorded information may be sent to a remote location together with the alarm, and the remote location may be the communication device of the vehicle's driver or a remote control center responsible for monitoring the operation of the vehicle (e.g., a truck fleet).

[0066] When it is determined in step 1017 that no tampering event is detected, the IMU readings can be used to detect other maintenance-related requirements. As described herein, the IMU data associated with the sensor can be used by the sensor installation determiner 1010 and the sensor calibration determiner 1050 in steps 1045 and 1075, respectively, to determine whether reinstallation or recalibration is required. In some embodiments, reinstallation may be required when the handling of the sensor's attitude is relatively severe and may render the sensor unreliable. The sensor installation determiner 1010 can determine in step 1055 whether reinstallation is required based on the vehicle state information from 1060. For example, if the vehicle is moving at high speed and the LiDAR sensor in front of the vehicle is severely misaligned (e.g., due to rough road conditions), then if the vehicle is operating in autonomous mode, this situation may pose a serious safety hazard. In such a case, once the sensor installation determiner 1010 detects this situation, it can immediately call the alarm notification unit 1030 to alarm the driver in step 1065, so that the driver can switch to the manual driving mode and stop the vehicle to reinstall the sensor.

[0067] If reinstallation is not required, then in step 1075, the sensor calibration determiner 1050 can determine whether any of the sensors associated with the IMU readings need to be recalibrated. This determination can be made based on information related to the calibration state recorded in 1070. For example, if a pair of stereo cameras have been previously calibrated to establish a transformation matrix, and the calibration assumptions and the camera positions can also be recorded. For example, the assumption may be that the two cameras are horizontal, and what is recorded may also include the attitudes of the two cameras. When the in-vehicle anti-tampering system 800 receives information from the IMU, it can compare the reported attitudes of the cameras with the recorded attitudes to see, for example, whether the assumption of horizontal alignment still holds, and so on. If the current attitude indicates that the two stereo cameras are no longer horizontally aligned, then the previously calibrated transformation matrix is no longer applicable for estimating depth based on the two images obtained from the two stereo cameras. In such a case, considering the current attitude of the cameras, it may be necessary to recalibrate the stereo cameras to adjust the transformation matrix. To this end, the sensor calibration determiner 1050 can call the sensor calibration scheduler 1040 to schedule an upcoming task for recalibrating the sensor in question in step 1095.

[0068] In some embodiments, recalibration can be performed on-the-fly while the vehicle is in motion. In some embodiments, recalibration may need to be performed offline while the vehicle is not in motion. A schedule for recalibration can be set according to the urgency of the problem. For example, if the attitude deviation is large and / or the sensor in question is very important for autonomous driving, then recalibration can be scheduled to occur soon. For any sensor, if on-the-fly recalibration is not possible and the vehicle is in motion, then the sensor calibration decision maker 1050 can also call the alert communication unit 1030 to send an alert to an appropriate party, such as the driver or a remote monitoring center ( Figure 10B not shown).

[0069] Through the IMU and the vehicle anti-tampering system 800, tampering events can be detected, alerted, and reported. At the same time, the detection of such an event can trigger the sensor to record information around the event, thus providing useful evidence associated with the tampering. Additionally, IMU readings can also be used to detect the need for adjustment of sensors that are detected as misaligned for any reason. The detection of such a need may be crucial for ensuring the correct operation of sensors in any autonomous vehicle and is key to safety.

[0070] In addition to the installation and proper orientation of sensors, other aspects of the sensors can also affect their operation, thus affecting the safe operation of the autonomous vehicle. For example, sensors are typically installed outdoors and are vulnerable to wind, dust, debris, or other problems caused by the environment and / or wear and tear. Certain types of sensors may be more susceptible to such environment-related problems. For example, if raindrops are present on the lens of a camera, then it prevents the camera from obtaining a true picture of the scene in its field of view. Debris deposited on the lens of the camera will cause the same problem. Even without such deposits on the lens, exposing the camera to the outdoors for a long time will cause enough dust to accumulate on the lens, affecting the ability to sense precise information in the field of view.

[0071] Figure 11AShows a camera 1110 with a lens 1102 and the resulting image, where an environment-induced object is deposited on the lens 1102. As shown, the camera lens 1102 is flashing with environmental particles, which prevents the camera from capturing accurate information. Image 1107 illustrates an example image obtained by the camera 1110, where the obtained image includes many circular sub-objects that may correspond to debris deposited on the lens 1102, and these debris effectively obscure the view of the scene that the camera 1110 is intended to capture. Image 1110 is another example image obtained by the camera 1100. As can be seen from Image 1110, there are various seemingly spurious objects identified by the edges detected in the image. Such spurious objects may correspond to, for example, raindrops deposited on the lens of the camera. In some cases, some events such as light / glare from the sun may also cause degradation of the image.

[0072] In some cases, a sensor such as a camera may be enclosed in a housing so that the lens of the camera is not subject to different adverse objects / events. However, the camera may still be affected by the environment through the medium by which it senses visual information, such as a transparent cover on the housing in front of the camera lens, because such adverse objects / events may still occur on the transparent cover. Various known signal processing techniques can be used, such as a combination of edge detection, texture analysis, and artificial intelligence (AI)-based inference / argumentation / learning algorithms, to detect spurious objects / events from the environment that appear in the image of the scene captured by the camera. When using AI-based techniques, heuristics can be applied to define the characteristics of different types of spurious objects / events in order to facilitate the detection of such spurious objects / events that may exist in the sensor data due to negative environmental impacts and thus represent the degradation of the obtained image.

[0073] Through such processing, the degradation of the image can be identified, and in some cases, the detected degradation information can also be used to estimate the cause of the degradation. For example, if many small sub-objects are continuously detected in multiple frames of pictures even when other objects undergo different motions (e.g., other cars on the road, passing buildings, etc.), then it can be estimated that such sub-objects may correspond to environmental deposits on the camera lens or its cover. The problem with such degraded images is that it prevents an autonomous vehicle from obtaining a true representation of its surrounding environment and inhibits its ability to make safe and reliable autonomous driving decisions. Therefore, a mechanism is needed to protect the sensor or the sensor mount from such environment-related impacts.

[0074] Figures 11B - 11F Shows different aspects of an environmental protection sensor housing assembly according to an embodiment of the present teachings. Figure 11BDescribe an exemplary environmental protection sensor housing assembly 1120 according to an embodiment of the present invention. The environmental protection sensor housing assembly 1120 includes means for protecting a sensor housed therein (optionally in its own housing placed in the assembly 1120) from the negative effects of the environment. In this exemplary embodiment, the environmental protection sensor housing assembly 1120 includes six sides, one of which is an inclined surface 1170, as shown in more detail in Figure 11B as shown in more detail below. The inclined surface 1170 of the housing serves as a window for the sensor within the environmental protection sensor housing assembly 1120. For example, if the sensor is a camera 1125, the lens of the camera "sees" the scene in the field of view through the inclined surface 1170 and collects visual information about the scene. To this end, the inclined surface 1170 can be made of a transparent material to enable the camera to see through.

[0075] According to an embodiment, the inclined surface 1170 can be arranged at an angle with respect to the longitudinal axis L-L of the housing assembly 1120. In one embodiment, the inclined surface 1170 can be arranged at an acute angle α with respect to the longitudinal axis L-L and / or a plane or surface extending parallel to the longitudinal axis L-L (e.g., the bottom surface of the housing assembly 1120). In such a configuration, for example, the inclined surface 1170 can make it more difficult for unwanted objects deposited thereon to remain. In yet another embodiment, the inclined surface 1170 can be arranged at an obtuse angle with respect to the longitudinal axis L-L and / or a plane or surface extending parallel to the longitudinal axis L-L (e.g., the bottom surface of the housing assembly 1120). In another embodiment, the inclined surface 1170 can be arranged at a right angle or substantially perpendicular with respect to the longitudinal axis L-L and / or a plane or surface extending parallel to the longitudinal axis L-L (e.g., the bottom surface of the housing assembly 1120).

[0076] Within the environmental protection sensor housing assembly 1120, according to an embodiment, there can be an additional inner housing 1105. The inner housing 1105 can be located at the position where a sensor such as a camera is present in the assembly. In an embodiment, the inner housing 1105 can be embedded within the housing assembly 1120. In one embodiment, the inner housing 1105 can be in the form of a rectangular prism having a longitudinal axis along the L-L axis. In an embodiment, the housing assembly 1120 can be in the form of a trapezoidal prism, where its longitudinal axis L-L is parallel to the longitudinal axis of the inner housing 1105 (and vice versa).

[0077] In addition, according to an embodiment, a protruding portion 1165 can be provided that extends between the inner housing 1105 and the housing assembly 1120. The protruding portion 1165 also has a longitudinal axis parallel to the longitudinal axis L-L. As shown in Figures 11C - 11EAs shown, the protruding portion 1165 abuts against the inclined surface 1170. The protruding portion 1165 can be in the form of a tube that is truncated or ends at the interface with the inclined surface 1170. One end of the protruding portion 1165 is firmly connected to the additional inner housing 1105, and the other end of the protruding portion 1165 can be open, having an inclined cross-section. The inclined cross-section has the same angle as the inclined surface 1170 such that the two can intersect (see Figures 11D - 11E ), thereby forming a combined closed structure as illustrated in Figure 11B and 11F . In this exemplary embodiment, the inclined surface 1170 corresponds to the front side of the housing assembly 1120, and the front side intersects the cross-section of the protruding portion 1165 and has the normal of this cross-section. In one embodiment, the sensor is a camera, and its lens faces the cross-section of the protruding portion 1165 such that the camera senses visual information through the transparent inclined surface 1170 of the housing assembly 1120.

[0078] Around the inclined surface 1170, there may be a frame-like structure 1115 (in Figure 11B and 11C both), and on the frame-like structure 1115, one or more mechanisms or devices can be deployed to enable different devices to clean the inclined surface 1170. Each mechanism or device can be associated with a specified cleaning task. The one or more mechanisms can be activated individually or jointly as needed to clean the inclined surface 1170 in order to prevent deterioration of the information sensed by the sensor. In some embodiments, the one or more mechanisms or devices may include a fluid cleaning device, a wiper assembly, and a blower assembly.

[0079] In the exemplary embodiment as shown in Figure 11B , at the top of the frame-like structure 1115, there are horizontal and evenly spaced hydrophobic spray holes 1130 that allow the cleaning fluid to flow out of the spray holes and flow downward along the inclined surface 1170 to rinse the inclined surface 1170. The spray holes can be connected to a certain liquid storage tank or reservoir (see Figure 12A ), and the liquid storage tank or reservoir can be pressurized to control the hydrodynamics. There may also be one or more valves connected to each spray hole, and each valve can be controlled to be opened, partially opened, or closed to allow the fluid to flow out of the spray hole.

[0080] Figure 12AIllustrative embodiments are diagrammatically shown of using a fluid cleaning device to deliver or spray a cleaning fluid onto an inclined surface 1170 to remove debris and / or particles from the inclined surface 1170. As shown, the orifice 1130 can be part of a spray manifold 1200. That is, the fluid cleaning device includes a spray manifold 1200, an orifice 1130, and a source 1215 of cleaning fluid such that the cleaning fluid is delivered from the source 1215 through the manifold 1200 and the orifice 1130 to the inclined surface 1170. The orifice delivers or sprays the cleaning fluid, such as a hydrophobic fluid or liquid, onto the inclined surface 1170. The manifold 1200 can be an object having channels therein that are adapted to receive the cleaning fluid (from a source, such as a storage tank 1215) for delivery (simultaneously) to the orifice 1130 for outputting the cleaning fluid. Connected to the manifold 1200 is an optional delivery device 1205 that is configured to deliver the cleaning fluid from a source (such as a reservoir or storage tank 1215) to the manifold 1200. The delivery device 1205 can be in the form of a structure disposed on the housing assembly 1120, such as a conduit, tube, pipe, or hose, although this is not limiting.

[0081] The source, reservoir, or storage tank 1215 is a container configured to store a cleaning fluid or rinse fluid, such as a hydrophobic fluid or liquid or other types of cleaning fluids. An optional pump 1210 can be connected to the storage tank 1215 and to the (optional) delivery device 1205. The pump 1210 can be adapted to pump out the cleaning fluid from the storage tank 1215 and supply the cleaning fluid to the delivery device 1205 (optionally) and the manifold 1200 and its orifice 1130. In an embodiment, the pump 1210 can be designed to pressurize the cleaning fluid for output from the manifold 1200 and the orifice 1130. A controller 1330 (see Figure 13A ) can be implemented to, in response to an input command signal (e.g., based on sensed information (such as visual information observed by a camera present in the environmental protection sensor housing assembly 1120), the sensed information being sent to a sensed quality control unit 1310 for sensed quality assessment, as described in more detail below), activate or drive the pump 1210 (and / or the fluid cleaning device) such that the pump 1210 withdraws fluid from the storage tank 1215, pressurizes the fluid, and outputs the pressurized cleaning fluid through the delivery device 1205 (such as a pipe) (if applicable), and through the orifice 1130 of the manifold 1200 to the inclined surface 1170.

[0082] In an embodiment, as previously described, each orifice 1130 can have an associated valve or nozzle (not shown). Each nozzle can be configured such that its output opening is aligned with the inclined surface 1170. When the cleaning fluid is pumped into the channels of the manifold 1200, the cleaning fluid can be pushed to each nozzle and then sprayed from the orifice 1130 onto and onto the inclined surface 1170.

[0083] Return to reference Figure 11B , along the left vertical side of the frame-like structure 1115, a wiper blade 1140 that is part of a wiper assembly 1220 (see Figure 12B ) can be attached in a manner that allows the wiper blade to move up and down along the inclined surface 1170. The wiper blade can be in slight contact with or very close to the inclined surface such that any object protruding from the inclined surface 1170 will be wiped off if the wiper blade moves. Such protruding objects can correspond to dust, dirt, debris, particles, or any other type of residue deposited on the inclined surface from the environment, and / or cleaning fluid provided on the inclined surface 1170 from a fluid cleaning device. The wiper blade 1140 can be motorized, and a motor (e.g., see 1235 described below Figure 12B ) can be controlled to move the wiper blade in either direction along the longitudinal direction of the inclined surface 1170. A micro motor can be installed within, above, or near the track 1135 (e.g., embedded within the left vertical side) of the frame-like structure 1115, and the micro motor can be controlled to achieve the desired movement of the wiper blade. For example, when the cleaning fluid is ejected from the hydrophobic nozzle 1130 onto the inclined surface 1170 and flows downward along the longitudinal direction of the inclined surface, the movement of the wiper blade 1140 can enhance the cleaning of the wet inclined surface.

[0084] Figure 12BThe illustration depicts an exemplary embodiment of a wiper assembly 1220 for cleaning an inclined surface 1170. The wiper assembly 1220 includes a wiper blade 1140 disposed on a wiper arm 1225, an actuator 1235 (e.g., an electric motor), and an optional transmission system 1230 for the movement of the wiper arm 1225 and the wiper blade 1140, such as translational movement. In an embodiment, the wiper blade 1140 is configured to translate or articulate between the bottom and the top of the inclined surface 1170 by the movement of the wiper arm 1225. In an embodiment, the wiper assembly 1220 includes a flexible wiper blade 1140 mounted on a rigid wiper arm 1225 such that when the arm 1225 moves, the flexible wiper blade 1140 moves along the inclined surface 1170 (e.g., translates in an upward or downward direction). According to an embodiment, the wiper arm 1225 may be connected to the transmission system 1230. The transmission system 1230 is configured to transfer power from the actuator 1235 into motion to move the wiper assembly 1220. For example, the transmission system 1230 may include a mechanical mechanism for moving the wiper arm 1225 to move the wiper blade 1140. In an embodiment, the transmission system 1230 includes a conveyance device for pushing and pulling the wiper arm 1225 along and / or within a track 1135. The end of the wiper arm 1225 may be inserted and fixed within the track 1135 such that the wiper arm 1225 is still configured to move relative to the track 1135 when pushed or pulled. In yet another embodiment, the transmission system 1230 includes at least one gear mechanism configured to be rotated by the actuator 1235. For example, such a gear mechanism may be configured to move along teeth within the track 1135. In another embodiment, the transmission system 1230 includes a belt and / or pulley system for moving the wiper arm 1225 relative to the track 1135. For example, a gear may be utilized to rotate the belt or pulley to move the wiper arm 1225. Thus, although examples of the transmission portion are presented herein, the type of mechanism and / or transmission system for moving the wiper arm 1225 within and relative to the track 1135 to move the wiper blade 1140 along the inclined surface 1170 is not limiting. Additionally, the transmission system 1230 is not required. In one embodiment, the actuator or electric motor 1235 may be connected to the wiper arm 1224 and the wiper blade 1140.

[0085] A controller 1340 may be implemented (see Figure 13A) in response to an input command signal (e.g., based on detection of the implementation of fluid delivery from a fluid cleaning device, and / or based on sensing information sent to the sensing quality control unit 1310 for sensing quality assessment, described in more detail below), start or drive the wiper assembly 1220 such that the wiper arm 1225 translates or oscillates between the bottom and top of the inclined surface 1170. In one embodiment, the wiper assembly 1220 can be controlled in conjunction with the use of the injection manifold 1200; for example, the wiper arm 1225 can be used during and / or after spraying cleaning fluid onto the inclined surface 1170.

[0086] According to another embodiment, the wiper arm 1225 and the wiper blade 1140 are configured to oscillate about a pivot point, much like a standard windshield wiper system for a vehicle windshield. For example, a motor or actuator 1235 can be connected to the wiper arm 1225, and the wiper arm 1225 is configured to reciprocate or oscillate about a pivot point (e.g., at a center point) along a track 1135. The wiper arm 1225 can be connected to a rotatable shaft at the pivot point and can be configured to rotate about its pivot such that the wiper arm 1225 pivots back and forth between two positions.

[0087] In addition to the spray holes 1130 and the wiper blade 1140, additional mechanisms for cleaning the inclined surface 1170 can be introduced. For example, as Figure 11B shown, an air hose 1150 that is part of a blower assembly 1240 (see Figure 12C ) can also be attached to the frame-like structure 1115, making it possible to blow away solid deposits (e.g., debris, particles) from the inclined surface 1170. As Figure 11B shown, the air hose 1150 is disposed at the bottom of the frame-like structure 1115. Through the air hose, air can be blown upward from the bottom of the inclined surface 1170 to blow away any unwanted debris and residues deposited on the inclined surface. Figure 11F Illustrates more details associated with the air hose 1150 in accordance with an embodiment of the present teachings. As shown, the air hose 1150 can correspond to a structure 1155 or manifold that houses a blowing slot 1157. The manifold or structure 1155 can be an object having channels therein that are adapted to receive pressurized air for delivery to the inclined surface 1170 through the air holes 1150. The air hose structure 1155 can be attached to the bottom edge of the frame-like structure 1115 and connected to a delivery device (in Figure 12C(schematically illustrated in), the conveying device is an air supply pipe from a supply source or air source 1255, where when it is necessary to convey air through the device at a certain speed, the air from the supply source can be pressurized. The conveying device 1245 can be in the form of, for example, a conduit, tube, pipe, or hose, although this is not restrictive. The supply source or air source 1255 can be a container configured to store air, or the air can also be sourced from the surrounding environment. An optional compressor 1250 or pump can be connected to the air source 1225 and the conveying device 1245. The compressor 1250 can be suitable for pressurizing and pumping the air from the air source 1255 (or the environment) and supplying the pressurized air to the conveying device 1245, and thus to the manifold / structure 1155 and the air holes 1150.

[0088] A blowing controller 1320 can be implemented (see Figure 13A ), in response to an input command signal (e.g., based on sensing information sent to the sensing quality control unit 1310 for sensing quality assessment, which will be described in more detail below), to start or drive the compressor 1250, so that the compressor 1250 pressurizes the air and outputs it through the output device 1245 (such as a conduit) (if provided) and through the air holes 1150. In an embodiment, each air hole 1150 can be configured such that its output opening is aligned with the inclined surface 1170, so as to push debris upward and / or outward away from the inclined surface 1170. In one embodiment, a blower assembly 1240 can be started to blow air toward the inclined surface 1170, so as to remove cleaning fluid and / or debris from the inclined surface 1170. According to an embodiment, in addition to or instead of the wiper assembly 1220, air can be used.

[0089] Although the drawing depicts the blowing slot 1157 as part of the air hose 1150, it should be noted that a number of air holes or slots can be provided for conveying pressurized air to the inclined surface 1170.

[0090] In addition, although the above drawings and description relate to the fluid cleaning device and the manifold 1200 disposed at the top or above the inclined surface 1170, and the blower assembly 1240 at the bottom or below the inclined surface 1170, their positions and the described directions are not restrictive. That is, the positions of the manifold 1200 and the blower assembly 1240 can be swapped, i.e., placed at the bottom and top of the inclined surface 1170 of the sensor housing assembly 1120 respectively. Thus, any description regarding the fluid (cleaning fluid, air) and its directional movement (e.g., downward, upward) is illustrative rather than restrictive.

[0091] Accordingly, any of the above-described cleaning mechanisms can be provided on the housing assembly 1120 individually or in combination. Thus, in addition to providing the sensor housing assembly 1120 on an autonomous vehicle, the present disclosure also includes a system for using a controller to determine a negative impact on at least one sensor and, in response to the determined negative impact, activating, via the controller, one or more cleaning devices mounted on the sensor housing assembly and configured to assist in cleaning and / or removing debris and / or particles from the sensor housing assembly 1120, particularly from the inclined surface 1170.

[0092] For an environmental protection sensor housing assembly 1120 on which various cleaning mechanisms are deployed, it is the inclined surface 1170 of the environmental protection sensor housing assembly 1120 that is cleaned rather than the sensors housed therein. The cleaning operation can be controlled based on, for example, observed quality issues that occur. More details related to the implementation, control, and operation of the cleaning mechanisms described herein will be provided with reference to Figures 13A - 16B provided.

[0093] For alternative implementations of different mechanisms deployed on the environmental protection sensor housing assembly 1120, such mechanisms are controlled to operate when cleaning is required. In some embodiments, the control of the operation can be based on the quality of the sensed information. For example, the sensed information can be analyzed in terms of sensing quality to determine whether the inclined surface 1170 needs to be cleaned. Figure 13A In accordance with an embodiment of the present teachings, an exemplary embodiment of a system configuration for environmental protection sensing is described. In this exemplary system configuration, a sensor (e.g., a camera) 1125 disposed in the environmental protection sensor housing assembly 1120 is connected to an environmental protection sensor system controller 1300, which controls the sensing quality of the sensor. In this exemplary embodiment, the environmental protection sensor system controller 1300 includes a sensing quality control unit 1310, a blowing controller 1320, a hydrophobic jet controller 1330, and a wiper motor controller 1340.

[0094] For facilitating environmental protection quality sensing, a sensor provides sensed information (such as visual information observed by a camera present in an environmental protection sensor housing assembly 1120) to a sensing quality control unit 1310 for sensing quality assessment. Based on the quality of the sensed information, the sensing quality control unit 1310 can control the activation of one or more controllers associated with different cleaning devices to clean an inclined surface 1170. For example, when raindrops are detected in an image acquired by the camera present in the assembly 1120, the sensing quality control unit 1310 can call a wiper motor controller 1340 to activate a motor to drive a wiper blade 1140 to wipe off the raindrops on the inclined surface 1170. In some cases, the sensing quality control unit 1310 can also activate a hydrophobic spray controller 1330 to initiate the spraying of a hydrophobic cleaning liquid while the wiper wipes the inclined surface 1170 to enhance the cleaning effect. To this end, the hydrophobic spray controller 1130 can control the state of a valve associated with a hydrophobic hole and the fluid supply pressure. If debris is observed in an image from the camera (e.g., an object that is always detected without any change or movement in different frames), then the sensing quality control unit 1310 can call a blowing controller 1320 to activate an air hose 1150 to blow off the debris deposited on the inclined surface 1170.

[0095] For operation, the sensing quality control unit 1310 is connected to the blowing controller 1320, the hydrophobic spray controller 1330, and the wiper motor controller 1340 and calls the appropriate controller in case of a need for cleaning. Once called, the controller can then control the associated cleaning mechanism to clean the inclined surface 1170. Figure 13B It is a flowchart of an illustrative process of an environmental protection sensor system controller 1300 according to an embodiment of the present teachings. To evaluate the quality of sensed information, the environmental protection sensor system controller 1300 first receives, at step 1350, the information sensed by a sensor present in a housing 1200. Subsequently, the received sensed information is analyzed at step 1355 to determine whether the quality of the sensed information makes it necessary to clean the inclined surface 1170. If it is determined at step 1360 that cleaning is not required, then the environmental protection sensor system controller 1300 loops back to step 1350 to continuously receive sensed information from the sensor.

[0096] If it is determined in step 1360 that cleaning is required, then the environmental protection sensor system controller 1300 determines in step 1365 what type of cleaning needs to be applied. This determination can be made according to some pre-determined rules, based on the type and severity of the quality degradation in the received sensor observations. For example, if the degradation is related to raindrops on an inclined surface, then wiping with the wiper blade 1140 can solve the problem. If the degree or severity of such degradation is significant, then both the wiper blade 1140 and the hydrophobic spray 1130 may be required. If the degradation seems to be caused by dust / debris deposited on the inclined surface 1170, then the air hose 1150 may be needed to blow off the dust / debris. In some embodiments, after using the air hose 1150 to blow off the dust / debris, other devices may be employed to clean the inclined surface because the dust / debris deposition may leave annoying marks on the surface, affecting the quality of the observations made through the inclined surface 1170. In some cases, when dust / debris is detected, the environmental protection system controller 1300 can first apply the air hose to blow off the dust / debris and then monitor whether it has satisfactorily eliminated the degradation without further cleaning. If it further observes that the degradation still exists (even though the degradation may have been alleviated by the air hose 1150), then the environmental protection system controller 1300 can subsequently call for additional cleaning using the wiper blade 1140 and / or the hydrophobic spray 1130. Thus, the control can be set in a situation where intelligent decisions are made adaptively based on the circumstances.

[0097] Depending on the type of cleaning required to mitigate the degradation, the environmental protection system controller 1300 can then proceed to design a cleaning configuration or schedule in step 1370, e.g., when to apply which cleaning device and for how long. Based on the configuration / schedule for applying the required cleaning, the environmental protection system controller 1300 then generates a control signal in step 1375 to implement the cleaning schedule and, in step 1380, sends the control signal to the controller responsible for controlling the cleaning devices to perform the predetermined cleaning.

[0098] Figure 14AIllustrative reasons that lead to the need to clean the sensor assembly and illustrative cleaning configurations in accordance with embodiments of the present teachings are shown. In this illustration, illustrative reasons that lead to the need to clean the inclined surface include raindrops, deposited debris, …, and dust accumulation. For each reason, a series of cleaning activities can be configured. For example, to remove raindrops, there may be two cleaning activities configured as Step 1 and Step 2, where Step 1 involves activating the hydrophobic jet 1130 of the fluid, and Step 2 involves activating the wiper blade 1140 to wipe the wet surface to remove the raindrops. These two steps can also be separated by an appropriate delay in time. Similarly, to remove debris deposited on the inclined surface 1170, as Step 1, an air hose 1150 can first be applied to blow off the debris. This can be followed by a second operation to remove the stains left on the surface by the debris, which can include applying the hydrophobic jet as a first sub-step, and then as a second sub-step of the second operation, wiping with the wiper blade. The same cleaning steps can also be used for dust removal, as Figure 14A shown in

[0099] Figures 14B - 14C An illustrative implementation of the environmental protection system controller 1300 is described. Figure 14B An illustrative high-level system diagram of the environmental protection system controller 1300 in accordance with embodiments of the present teachings is described. To implement the desired functions disclosed herein, the environmental protection system controller 1300 includes a sensor data feature extractor 1400, a degradation cause classifier 1410, a cleaning requirement determiner 1420, a cleaning activity configuration unit 1430, a cleaning activity scheduler 1440, a cleaning tool parameter analyzer 1450, and a cleaning control signal generator 1460. Figure 14C A flowchart of an illustrative process of the environmental protection system controller 1300 in accordance with embodiments of the present teachings is shown. Figure 14B The different components included in the environmental protection system controller 1300 disclosed in Figure 14C perform the processes described in

[0100] During operation, the sensing quality control unit 1310 is used to apply the necessary cleaning to the environmental protection sensor housing assembly to minimize the negative impact of the environment on the sensing quality. It can perform a certain process to achieve the cleaning. For example, it can first detect the existence of such a need, and then determine what types of cleaning are required (e.g., wiping or blowing). After that, it can be determined which cleaning tools are to be activated to perform the required cleaning. Optionally, to achieve the cleaning, the sensing quality control unit 1310 can also proceed to schedule different cleaning activities (e.g., when to apply which cleaning tool and for how long) for the cleaning. Finally, to actually control the cleaning tools to perform the predetermined cleaning, appropriate control signals for different cleaning tools can be generated and used to activate the selected cleaning tools.

[0101] There can be different ways to determine whether cleaning is needed. In some embodiments, cleaning can be performed routinely according to a schedule. In some embodiments, the need for cleaning can be determined based on the situation. For example, as described herein with respect to Figure 11A it can be detected that the acquired image has deteriorated, and when it is estimated that the deterioration is caused by bad objects (e.g., raindrops, small particles, etc.) deposited on the inclined surface from the environment, it can be determined that cleaning is needed. In some embodiments, a hybrid operation mode can also be adopted. Although there is a predetermined schedule, the dynamically detected deterioration can also trigger the cleaning. For example, it can be set that under normal circumstances, cleaning can be performed according to the schedule (e.g., once every 24 hours when the vehicle is moving and once a week when the vehicle is not moving). This setting can be implemented in parallel with the dynamic setting, through which whenever deterioration is observed and evaluated as being related to the deposition of bad objects on the assembly, cleaning can also be triggered. In this case, the dynamically determined cleaning decision overrides the normal schedule, enabling the assembly to be cleaned promptly when needed. In Figure 14B a cleaning requirement determiner 1420 is provided to identify the need for cleaning. As shown in the figure, it can retrieve the settings for regular / normal cleaning from the cleaning activity schedule storage device 1470 and determine whether to activate the cleaning operation based on, for example, the reading from a timer. It can also call the sensor data feature extractor 1400 and the deterioration cause classifier 1410 to identify possible deterioration present in the sensing information and the cause estimated based on, for example, the deterioration category specifications stored in 1415. If the cause of the detected deterioration makes cleaning necessary, then the cleaning requirement determiner 1420 then proceeds to activate the cleaning operation.

[0102] When the cleaning need determiner is to make a cleaning decision based on the detected degradation, the sensor data feature extractor 1400 receives sensing information from sensors present within the environmental protection sensor housing assembly at step 1435, and at step 1445, it extracts relevant features from the sensing information. The relevant features to be extracted can be determined based on the type of sensor providing the sensing information. For example, if the sensor in question is a camera, then the relevant features to be extracted may include edges and their characteristics, as well as characteristics of objects, such as the shape, size, and motion of individual objects. These extracted features can be used to evaluate the possible degradation of the sensing information caused by, for example, environmental effects. For example, when an edge is detected in an image, the features associated with the edge can be analyzed and used to determine whether the image is blurry.

[0103] The characteristics of an object detected from an image can also be used to determine whether the object corresponds to a real object or a deposit on an inclined surface. If an object is considered not to correspond to an actual object, then that object has caused degradation of the sensing information and thus needs to be removed. There can be different ways to determine whether an object is a real object. For example, if an object of the same shape and size is continuously detected at the same location in all frames of a video, then that object may be debris on the inclined surface 1170. If it is observed that such an object is split into multiple parts that move from one frame to another in different directions inconsistent with the known motion of real objects observable in the field of view of the autonomous vehicle, for example, raindrops may drip vertically (raindrops may travel along the inclined surface in the longitudinal direction), then such information can be used to evaluate whether certain objects correspond to raindrops.

[0104] Based on the features extracted from the sensed information, at step 1455, the degradation cause classifier 1410 determines and classifies the degradation and its cause. Thus, the cleaning need determiner 1420 determines the required cleaning at step 1465 based on, for example, a cleaning standard profile stored in 1425 (an example of which is provided in Figure 14A ). For the required cleaning, the cleaning activity configuration unit 1430 generates a configuration for the cleaning activity involved at step 1467. As shown in Figure 14A , to reduce the degradation caused by debris, the air hose 115 will be applied to physically remove the debris, followed by a stain removal step that involves two sub-steps of hydrophobic spraying and wiping the inclined surface using the wiper blade 1140. The configuration generated (by the cleaning activity configuration unit 1430) may include the specifications of the specific tools to be applied and the chronological order of applying different tools.

[0105] The generated configuration can then be used by the cleaning activity scheduler 1440 to generate a schedule for the next cleaning event at step 1495. Such a schedule can be generated based on an understanding of how each cleaning device (spraying, wiping, and blowing) works and parameters associated with the specific devices deployed on the environmental protection sensor housing assembly 1120. For example, the time it takes for the cleaning fluid to drip from the holes to cover the inclined surface can be used to determine how many seconds the hydrophobic spray holes need to be kept open. Such information is stored in the cleaning tool profile storage device 1460 and accessed by the cleaning tool parameter analyzer 1450 to analyze and design recommendations on how to activate each cleaning tool to be deployed at step 1475 according to certain operating parameters to achieve the desired goal. The recommendations designed by the cleaning tool parameter analyzer 1450 can then be provided to the cleaning activity scheduler 1440 so that the cleaning activity scheduler 1440 can generate an appropriate schedule at step 1485, which specifies how to activate each cleaning tool to be called for cleaning.

[0106] The cleaning activity schedule originating from the cleaning activity scheduler 1440 can specify the chronological order of sub-events, where each sub-event can correspond to the application of a cleaning tool to be used for cleaning. The schedule can also specify the specific time or duration for each cleaning tool to operate. The designed schedule can be stored in the cleaning activity schedule storage device 1470 after being generated. Each cleaning activity schedule stored in 1470 can be accessed by the cleaning control signal generator 1460 to first optionally check at step 1495 if it is a predetermined cleaning time. If so, then the cleaning control signal generator 1460 generates a corresponding control signal at step 1497 that can be used to implement the predetermined cleaning activity. Such control signals are then sent from the cleaning control signal generator 1460 to the corresponding controllers at step 1499, and the corresponding controllers then control their respective cleaning tools to perform the predetermined cleaning. This is illustrated in Figure 13A Figure. To perform the sub-cleaning events in a predetermined chronological order, control signals can be sent to different controllers at different times according to the schedule to ensure the correct chronological order of different cleaning steps.

[0107] For example, to remove debris deposited on the inclined surface 1170, such as Figure 14AAs shown, the cleaning activity to be applied may involve blowing off debris, wetting the inclined surface 1170, and then wiping the wet inclined surface to remove any stains resulting from the debris. To form such a cleaning activity, a configuration can be generated that specifies first applying the air hose 1150 to blow off the debris, second applying the hydrophobic jet 1130 to wet the inclined surface 1170, and finally applying the wiper blade 1140 to the wet inclined surface to wipe off any stains that the debris may leave. A schedule corresponding to this configuration can specifically indicate the chronological order of the individual events. For example, the air hose 1150 is applied to blow for 5 seconds, followed by the cleaning fluid being released from the nozzle 1130 for 3 seconds, and finally the motor of the wiper blade 1140 is activated in slow motion for 15 seconds.

[0108] According to this schedule, the cleaning control signal generator 1460 then generates and sends different control signals for the corresponding controllers at different times. In this example, the cleaning control signal generator 1460 then generates three control signals. The first control signal is for the blowing controller 1320; the second control signal is for the hydrophobic jet controller 1330; the third control signal is for the wiper motor controller 1340. The first control signal includes an instruction to control the air hose of the blowing controller 1320 to blow for 5 seconds. The second control signal includes an instruction to control the nozzle (and associated valves and pressure) of the hydrophobic jet controller 1330 to release the cleaning fluid for 3 seconds. The third control signal includes an instruction to control the wiper motor of the wiper motor controller 1340 to drive the wiper blade 1140 at a certain speed to wipe the inclined surface 1170 for 15 seconds. To control the chronological order, the cleaning control signal generator 1460 can first send the first control signal to the blowing controller 1320, send the second control signal to the hydrophobic jet controller 1330 5 seconds later, and finally send the third control signal to the wiper motor controller 1340 3 seconds later.

[0109] As described herein, in some embodiments, the sensing quality control unit 1310 may alternatively select a cleaning schedule to enforce regardless of what the sensing information reveals. For example, the cleaning activity schedule storage device 1470 may store a default cleaning schedule that specifies a fixed-interval cleaning schedule, e.g., a series of cleaning steps every 2 hours while the vehicle is in motion and the same series of cleaning steps each time the vehicle is turned on and begins to move. The sensing quality control unit 1310 may be configured to automatically execute such a default cleaning schedule under normal circumstances, unless the operating mode is switched to an adaptive cleaning mode, in which, as described above, cleaning is only activated when degradation is detected. Steps 1495 - 1499 form a loop corresponding to the fixed-interval cleaning schedule. As shown, in this loop, the cleaning control signal generator 1460 may access the default schedule stored in the storage device 1470 and check in step 1495 against a timer whether it is time to perform the cleaning. When at the specified fixed time interval, the cleaning control signal generator 1460 generates in step 1497 control signals corresponding to the predetermined cleaning activities in the order specified (as described above), and subsequently sends in step 1499 the generated control signals to the appropriate controllers (1320, 1330, …, 1340) in the specified order for the predetermined cleaning activities. This process continues in a loop such that the environmental protection sensor housing assembly can be cleaned regularly.

[0110] In some embodiments, the fixed-interval cleaning mode and the adaptive cleaning mode may be used in combination. For example, the fixed-interval cleaning mode may be the default mode. Meanwhile, the components (sensor data feature extractor 1400, degradation cause classifier 1410, cleaning requirement determiner 1420, cleaning activity scheduler 1440, cleaning tool parameter analyzer 1450) provided for performing the adaptive cleaning operation may continue to operate to determine whether any cleaning is required. If so, an adaptive cleaning schedule is created and executed. This adaptive cleaning schedule may or may not replace the default cleaning schedule. Through the mechanisms discussed herein, any negative impacts from the environment can be countered and the quality of the sensing information can be improved.

[0111] As described herein, in some embodiments, multiple sensors may be mounted on a sensor bracket, which may then be mounted on a vehicle to provide the vehicle with different types of sensor information to facilitate autonomous driving. The above discussion pertains to the sensor housing assembly for a single sensor. Similar concepts regarding environmental protection sensing quality control described herein may be applicable to a sensor bracket with appropriate modifications. Figure 15 An exemplary sensor bracket 1510 having a sensor bracket cover 1520 thereon is described in accordance with embodiments of the present teachings. WithFigure 6 Compared with the sensor bracket on which a multi-modal sensor is installed in [[reference]], the sensor bracket 1510 has a cover 1520 to prevent, for example, debris, rain, snow, dust, etc. from directly depositing on the sensor. As described herein, although the sensor 1530 is protected by the cover, the cover itself may also be negatively affected by the environment. Thus, there is a need to clean the cover.

[0112] As Figure 15 The sensor bracket cover 1520 illustrated in the figure is different from the environmental protection sensor housing assembly 1120 in several aspects. First, the illustrated sensor bracket cover 1520 has an extended length with turns, such as 1540-1 and 1540-2. Such turns are necessary when the bracket turns, and the bracket turning has the advantage of installing sensors that can observe information from different directions. Additionally, the sensor bracket cover 1520 has a curved surface. Although a flat surface can be used, such as the environmental protection sensor housing assembly 1120, there may be certain advantages to having a curved surface. For example, in the case of a curved surface, debris is less likely to remain on the cover, especially when the vehicle is moving. Although a cover with a curved surface can reduce the chance of debris remaining, it may still be necessary to introduce a mechanism that allows the cover to be cleaned when needed. The environmental protection sensor housing assembly disclosed herein can be applied to the sensor bracket cover 1520.

[0113] Figure 16A Illustrates an exemplary structure 1600 with an arcuate structure embedded with a cleaning mechanism according to an embodiment of the present teachings. Except that the structure 1600 is curved along the longitudinal direction, the structure 1600 is the same as the combination of 1170 and 1115 with an embedded cleaning tool. The structure 1600 has a frame-like structure 1620. The two parallel sides of the frame 1620 in the longitudinal direction are curved in the same way. Between these two parallel sides of the frame 1620, there may or may not be a curved surface. As shown, different tools / devices that can be controlled to perform cleaning-related tasks are attached to the frame-like structure 1620. This includes hydrophobic spray holes 1630 at the top of 1620, a wiper blade 1640 that can be motorized to move up and down along a track in the frame, and an air hose 1650 at the bottom of the structure 1620 that blows air upward. To motorize the wiper blade 1640, a motor can be embedded in one of the parallel sides in the longitudinal direction to drive the wiper blade to move. When there is a curved surface 1610 between the two parallel sides of 1620 in the longitudinal direction, when the wiper blade 1640 moves, it wipes and cleans the curved surface 1610. In some embodiments, there may be no curved surface 1610. In these embodiments, the structure 1600 can still be used to clean whatever curved surface the structure 1600 is placed on. This is shown in Figure 16B in.

[0114] Figure 16B Describing embodiments in accordance with this teaching, how to use the Figure 16A cleaning mechanism shown in to provide an environmentally protected sensor bracket assembly in combination with a sensor bracket lid 1520. In this illustrative embodiment, as Figure 15 shown in, the sensor bracket 1510 has a lid 1520 to protect the respective sensors 1530 (e.g., 1530-1 and 1530-2). Although the sensors are covered to prevent environmental-related objects (rain, debris, dust, etc.), the lid 1520 is not covered. The sensor bracket lid 1520 may still be negatively affected by the environment. To address this issue, a mechanism as shown in Figure 16A can be used to provide a segmented environmental protection cleaning mechanism for each part of the sensor bracket lid 1520, with each part associated with the respective sensors covered by that part. For example, two illustrative cleaning mechanisms (e.g., 1600-1 and 1600-2) are shown attached to parts of the sensor bracket lid 1520 to provide cleaning means for sensors 1530-1 and 1530-2, respectively. Each cleaning mechanism has a corresponding frame-like structure (i.e., 1620-1 and 1620-2), and the corresponding frame-like structures mate with the parts of the sensor bracket lid 1520 that are aligned with the respective sensors (e.g., 1530-1 and 1530-2) (e.g., the curvature of the frame of each cleaning mechanism is the same as the curvature of the sensor bracket lid 1520).

[0115] In some embodiments, the cleaning mechanism (1600-1 or 1600-2) may have a curved surface (e.g., 1610-1 and 1610-2) that closely mates with the curved surface of the sensor bracket lid 1520. In such cases, each cleaning mechanism is deployed to ensure the quality of sensing performed by the corresponding sensor by cleaning its own curved surface. Due to the mating between the cleaning mechanism and the sensor bracket lid, this configuration achieves an environmental protection effect. In some embodiments, the cleaning mechanism may not have its own curved surface. When attaching the cleaning mechanism by mating with the curved surface of the sensor bracket lid 1520, the cleaning mechanism can be used to clean the part of the sensor bracket lid that is directly within its frame.

[0116] The cleaning mechanism 1600 operates in a similar manner as described herein with reference to Figures 12A - 14C as shown in Figure 16BAs shown, a cleaning mechanism 1600-1 is deployed to ensure the environmental protection sensing quality of the sensor 1530-1. The cleaning mechanism 1600-1 includes a hydrophobic nozzle 1630-1, a wiper blade 1640-1, and an air hose 1650-1. These tools can be similarly embedded in the frame-like structure 1600-1 and controlled in a similar manner as described herein to clean the enclosed curved surface (whether or not it is directly part of the curved surface of the sensor bracket cover). Similarly, a cleaning mechanism 1600-2 is deployed to ensure the environmental protection sensing quality of 1530-2. The cleaning mechanism 1600-2 includes a hydrophobic nozzle 1630-2, a wiper blade 1640-2, and an air hose 1650-2, all of which can be controlled in a similar manner as described herein. Depending on the level of the sensor, for each sensor, the wiper blade can be controlled to clean within an appropriate range. In this configuration, the air hose can effectively blow off residues along a substantially vertical portion of the curved surface.

[0117] In some cases, dazzling sunlight (or glaring light) may be a problem that causes a decline in the ability to perceive accurate surrounding information. Figure 17A and 17B An illustrative embodiment of a glare shielding mechanism 1700 that solves this problem in the case of an environmental protection sensor or sensor bracket assembly associated with the aforementioned housing assembly 1120 is shown. Figure 17A -B illustrates an example of a glare shielding mechanism 1700 that can be mounted to and / or installed on a sensor housing assembly and is configured to assist in shielding glare for at least one of the sensors therein. According to an embodiment, for example, the glare shielding mechanism 1700 can be disposed above the air hose and above the housing assembly 1120 and / or the sensor bracket assembly. According to an embodiment, in the case where the sensor housing assembly includes an inclined surface, the glare shielding mechanism can be mounted above the inclined surface. In an embodiment, as Figure 17A shown in -B, the sensor in the housing assembly may include a camera whose lens faces the inclined surface, such as Figure 11B and 11F shown in (see housing assembly 1120, surface 1170, and camera 1125). The glare shielding mechanism is configured to shield dazzling sunlight from the sun or other glare, such as glaring light from streetlights, highway overhead lights or lighting, and other lights, signs, vehicles, etc. that may negatively affect at least one sensor (such as a camera) within the sensor housing assembly in the form of glare. A controller for determining the negative impact of the environment (including glare) on the at least one sensor ( Figure 17A or Figure 17B(not explicitly shown in the figure) can be used to activate the glare shielding mechanism 1700 in response to any such determined negative impacts, especially in response to the determination of glare that may affect the readings of the at least one sensor and / or the autonomous driving of the vehicle.

[0118] According to an embodiment, the glare shielding mechanism is configured to move between a first retracted position where it does not inhibit any observation of the sensor and / or does not inhibit any glare, and at least one second extended position to shield the glare while allowing the at least one sensor to capture accurate information. In one embodiment, the glare shielding mechanism includes a light-shielding curtain mounted to the sensor housing assembly. The light-shielding curtain can be configured to move between the first retracted position and any one of the at least one second extended positions. In one embodiment, the light-shielding curtain includes a body in a winding assembly, the body including a first end and a second end, the first end being fixed relative to the sensor housing assembly, wherein the second end is configured to move towards and away from the winding assembly such that the body is configured to wind around and unwind from an axis. In the first retracted position, the light-shielding curtain is wound into the winding assembly while the second end is disposed adjacent to the winding assembly. In the at least one second extended position, the light-shielding curtain is at least partially unwound from the winding assembly such that the second end is disposed away from both the winding assembly and the sensor housing assembly and extends therefrom, so that the body of the light-shielding curtain assists in shielding the glare relative to the at least one sensor. In another embodiment, the light-shielding curtain is mounted on an axle such that its body is configured to flip and rotate about the axis of the axle, the axle being located at one end of the sensor housing assembly. In the first retracted position, the body of the light-shielding curtain is positioned against the sensor housing assembly, and in the at least one second extended position, the body of the light-shielding curtain extends from the sensor housing assembly to assist in shielding the glare relative to the at least one sensor.

[0119] Specifically, Figure 17A -B represents an example of the light-shielding curtain 1720, which can be electrically operated such that its body is designed to extend a length L from the housing assembly (above the inclined surface 1170) and / or the edge of the sensor bracket assembly, while its second end 1725 is placed away from the axle and the axis A-A and extends away from the axle and the axis A-A, so that any sunlight or glare from above the sun (or the environment, e.g., overhead lighting) can be shielded by the light-shielding curtain 1720. In one embodiment, the light-shielding curtain 1720 is a collapsible light-shielding curtain that is configured to move between a retracted position (a stowed or undeployed position where the light-shielding curtain is not activated) and an extended position (or a deployed position), in which the light-shielding curtain is designed to shield the blinding sunlight (or other glare) from the sun while allowing the sensor to capture information. For example, it can be in a retractable (winding) assembly (see Figure 17A) in the form of a retractable (winding) assembly that winds (mounted on the sensor housing assembly) a reel 1730 around an axis A-A and unfolds from the reel 1730 between a retracted position and an extended position by an actuator 1510 or a motor. The actuator 1510 or the motor is controlled by a controller and is activated, for example, when it is determined that glare has a negative impact on the sensor / camera and / or the driving of the vehicle.

[0120] As described above, in another embodiment, the light-shielding curtain 1720 can be designed to be mounted on the sensor housing assembly such that its body is configured to rotate and flip back and forth around an axle providing the axis A-A. These features and this embodiment are also shown in Figure 17B i.e., the light-shielding curtain 1720 can be configured to move from a first storage (retracted) position (e.g., at the top of the housing assembly 1720 or behind the bracket cover 1520, as Figure 17B shown by the dashed line in) to a second extended (in-use) position (e.g., as Figure 17B shown in, where the length of the light-shielding curtain 1720 extends from the edge and the second end 1725 is placed away from the axle and the axis A-A and extends away from the axle and the axis A-A) by an actuator 1510 or a motor.

[0121] Figure 18 is an illustrative diagram of a mobile device architecture that can be used to implement a dedicated system for implementing the method of protecting sensors and sensor assemblies disclosed in this teaching according to various embodiments. In this example, the device on which this teaching is implemented corresponds to a mobile device 1800, including but not limited to a smart phone, a tablet computer, a music player, a handheld game console, a positioning system (GPS) receiver, and a wearable computing device (e.g., glasses, a wrist watch, etc.), or any other form of device. The mobile device 1800 may include one or more central processing units ("CPUs") 1840, one or more graphics processing units ("GPUs") 1830, a display 1820, a memory 1860, a communication platform 1810 such as a wireless communication module, a storage device 1890, and one or more input / output (I / O) devices 1840. Any other appropriate components, including (but not limited to) a system bus or a controller (not shown) may also be included in the mobile device 1800. As Figure 18 shown in, a mobile operating system 1870 (e.g., iOS, Android, Windows Phone, etc.) and one or more applications 1880 can be loaded from the storage device 1890 into the memory 1860 for execution by the CPU 1840. The application 1880 may include an appropriate mobile application for managing tasks related to this teaching on the mobile device 1800. User interaction can be achieved through the I / O device 1840.

[0122] To implement the various modules, units, and their functions described in this disclosure, a computer hardware platform can be used as the hardware platform for one or more of the elements described herein. The hardware elements, operating systems, and programming languages of such computers are conventional in nature, and it is assumed that those skilled in the art are sufficiently familiar with these technologies to adapt them to the appropriate settings as described herein. A computer with user interface elements can be used to implement a personal computer (PC) or other types of workstations or terminal devices. However, if appropriately programmed, a computer can also act as a server. It is believed that those skilled in the art are familiar with the structure, programming, and general operation of such computer devices, and thus the accompanying drawings should be self-explanatory.

[0123] Figure 19 FIG. is an illustrative diagram of an exemplary computing device architecture of a dedicated system that can be used to implement the various functions related to the implementation of the present teachings in accordance with various embodiments. Such a dedicated system incorporating the present teachings has a functional block diagram of a hardware platform that includes user interface elements. The computer can be a general-purpose computer or a dedicated computer. Both can be used to implement the dedicated system for the present teachings. Computer 1900 can be used to implement any component of the session or dialogue management system as described herein. For example, the various functions associated with the present teachings can be implemented on a computer, such as computer 1900, through the hardware, software programs, firmware, or a combination thereof. Although for convenience only one such computer is shown, the computer functions related to the session management system described herein can be implemented in a distributed manner on many similar platforms to distribute the processing load.

[0124] Computer 1900 includes, for example, a COM port 1950 connected to a network to which it is connected to facilitate data communication. Computer 1900 also includes a central processing unit (CPU) 1920 in the form of one or more processors for executing program instructions. The exemplary computer platform includes an internal communication bus 1910 for transferring various data files processed and / or communicated by computer 1900, as well as different forms of program storage devices and data storage devices (e.g., disks, read-only memory (ROM) 1930, or random access memory (RAM) 1940) that may be used to store program instructions to be executed by CPU 1920. Computer 1900 also includes I / O components 1960 for supporting the input / output streams between the computer and other components therein, such as user interface component 1980. Computer 1900 can also receive programs and data through network communication.

[0125] Thus, aspects of the methods of session management and / or other processing outlined above can be embodied programmatically. The program aspects of the present technology can be regarded as a "product" or "article of manufacture", which generally takes the form of executable code and / or associated data carried on or contained in some type of machine-readable medium. Tangible non-transitory "storage" type media include any or all memories or other storage devices that can provide storage for software programming at any time for a computer, a processor, etc., or their associated modules, such as various semiconductor memories, tape drives, disk drives, and the like.

[0126] All or part of the software can sometimes be transmitted through a network such as the Internet or various other telecommunications networks. For example, such transmission can cause the software to be loaded from one computer or processor into another computer or processor related to session management, for example. Thus, another type of medium that can carry software elements includes light waves, radio waves, and electromagnetic waves, such as those used across physical interfaces between local devices, through wired and fiber-optic fixed networks, and through various air links. Physical elements that carry these waves, such as wired or wireless links, fiber-optic links, etc., can also be regarded as media that carry software. Unless restricted to tangible "storage" media, terms such as computer or machine "readable medium" as used herein refer to any medium that participates in providing instructions to a processor for execution.

[0127] Thus, machine-readable media can take many forms, including but not limited to tangible storage media, carrier media, or physical transmission media. For example, non-volatile storage media include optical discs or magnetic disks, such as any storage device in any computer that can be used to implement the system or any of its components as shown in the accompanying drawings. Volatile storage media include dynamic memories, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wires and optical fibers, including the wires that form a bus within a computer system. Carrier transmission media can take the form of electrical signals or electromagnetic signals, or sound waves or light waves, such as those generated in radio frequency (RF) and infrared (IR) data communications. Thus, common forms of computer-readable media include, for example: floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROM, DVD or DVD-ROM, any other optical media, punched cards, paper tapes, any other physical storage media with a punched pattern, RAM, PROM, and EPROM, FLASH-EPROM, any other memory chip or cartridge, carriers that transmit data or instructions, cables or links that transmit such carriers, or any other medium from which a computer can read programming code and / or data. Many of these forms of computer-readable media may participate in conveying one or more sequences of one or more instructions to a physical processor for execution.

[0128] Those skilled in the art will recognize that various modifications and / or enhancements can be made to this teaching. For example, although the implementation of the various components described above can be embodied in hardware devices, it can also be implemented as a pure software solution - for example, an installation on an existing server. Additionally, the network fraud detection techniques disclosed herein can be implemented as firmware, a firmware / software combination, a firmware / hardware combination, or a hardware / firmware / software combination.

[0129] Although the content and / or other examples that are considered to constitute this teaching are described above, it should be understood that various modifications can be made to it, the subject matter disclosed herein can be implemented in various forms and examples, and this teaching can be applied in numerous applications, only some of which are described herein. The following claims are intended to claim any applications, modifications, and variations within the true scope of this teaching.

Claims

1. A sensor protection system, comprising: An assembly having a first structure and a second structure mounted on the outer surface of a vehicle, wherein The first structure is embedded in the second structure, The first structure includes a connecting tube truncated to produce an inclined cross-section, The second structure has a plurality of surfaces, including an inclined surface that intersects the inclined cross-section of the tube to form a closed structure, A sensor is placed in the first structure for sensing information about the surrounding environment through the assembly and being protected from the negative effects of the external environment; And One or more devices mounted on the second structure for cleaning the inclined surface of the second structure, through which the sensor senses the information, wherein the one or more devices are configured to be activated individually or jointly as needed to clean the inclined surface so as to prevent deterioration of the information sensed by the sensor.

2. The sensor protection system according to claim 1, wherein The first structure includes a rectangular prism, the rectangular prism and the truncated tube have parallel longitudinal axes, and the tube is truncated to produce a cross-section whose normal forms an angle with the longitudinal axis of the truncated tube; The second structure is a trapezoidal prism whose longitudinal axis is parallel to the longitudinal axis, wherein the inclined surface corresponds to the front side of the second structure, and the front side intersects the cross-section of the truncated tube and has the normal of the cross-section; And The one or more devices are mounted relative to the inclined surface.

3. The sensor protection system according to claim 1, wherein the inclined surface of the second structure is transparent.

4. The sensor protection system according to claim 3, wherein the sensor is a camera whose lens faces the cross-section of the truncated tube, such that the camera senses visual information through the transparent inclined surface of the second structure.

5. The sensor protection system according to claim 1, wherein the first structure is embedded therein and the second structure that houses the sensor is deployed on the vehicle to provide sensor information to the vehicle, thereby facilitating autonomous driving.

6. The sensor protection system according to claim 2, wherein the inclined surface of the second structure is set at an acute or obtuse angle relative to the longitudinal axis so that it is more difficult for foreign objects to deposit and remain on the inclined surface.

7. The sensor protection system according to claim 1, wherein each of the one or more devices is associated with a specified cleaning task.

8. The sensor protection system according to claim 1, wherein the one or more devices include a fluid cleaning device, a wiper assembly, and a blower assembly.

9. The sensor protection system according to claim 8, wherein The fluid cleaning device includes a spray manifold having spray holes thereon, The wiper assembly includes a wiper blade and a motor; and The blower assembly includes an air hose and an air source.

10. The sensor protection system according to claim 9, wherein: The manifold and the spray holes are configured to convey cleaning fluid from a source to the inclined surface; The wiper blade is configured to move along the inclined surface when the motor is activated; And The air hose includes at least one groove for guiding air from the air source to the inclined surface to remove any deposits on the inclined surface.

11. A method for a sensor protection system, comprising: providing an assembly having a first structure and a second structure mounted on an outer surface of a vehicle, wherein the first structure is embedded in the second structure, the first structure includes a connecting tube truncated to produce an inclined cross-section, the second structure has a plurality of surfaces, including an inclined surface that intersects the inclined cross-section of the tube to form a closed structure, a sensor is placed in the first structure for sensing information about the surrounding environment through the assembly and being protected from the negative effects of the external environment; and mounting one or more devices on the second structure to clean the inclined surface of the second structure, the sensor sensing the information through the inclined surface; and when a need to clean the inclined surface is detected, activating at least one of the one or more devices to clean the inclined surface so as to prevent deterioration of the information sensed by the sensor.

12. The method according to claim 11, wherein the first structure includes a rectangular prism, the rectangular prism and the truncated tube have parallel longitudinal axes, and the tube is truncated so as to produce a cross-section whose normal forms an angle with the longitudinal axis of the truncated tube; the second structure is a trapezoidal prism whose longitudinal axis is parallel to the longitudinal axis, wherein the inclined surface corresponds to the front side of the second structure, the front side intersects the cross-section of the truncated tube and has the normal of the cross-section; and the one or more devices are mounted relative to the inclined surface.

13. The method according to claim 11, wherein the inclined surface of the second structure is transparent, and wherein the sensor is a camera whose lens faces the cross-section of the truncated tube such that the camera senses visual information through the transparent inclined surface of the second structure.

14. The method according to claim 11, wherein the one or more devices include a fluid cleaning device, a wiper assembly, and a blower assembly; wherein the fluid cleaning device includes a spray manifold having spray holes thereon, the wiper assembly includes a wiper blade and a motor; and the blower assembly includes an air hose and an air source, and wherein the method comprises: delivering cleaning fluid from a source to the inclined surface through the manifold and the spray holes; moving the wiper blade along the inclined surface when the motor is activated; and guiding air from the air source to the inclined surface through at least one groove of the air hose to remove any deposits on the inclined surface.

15. A method for a sensor protection system, comprising: providing a sensor housing assembly mounted on an outer surface of an autonomous vehicle, the sensor housing assembly accommodating at least one sensor, the sensor housing assembly having a first structure and a second structure, wherein the first structure is embedded in the second structure, the first structure includes a connecting tube truncated to produce an inclined cross-section, the second structure has a plurality of surfaces, including an inclined surface that intersects the inclined cross-section of the tube to form a closed structure, The sensor housing assembly protects the at least one sensor from the negative effects of the environment, including debris and / or particles, which impede the at least one sensor from capturing accurate information related to the field of view with respect to the environment to facilitate autonomous driving of an autonomous vehicle; Using a controller, determining the negative effects on the at least one sensor; and In response to the determined negative effects, activating, by the controller, one or more cleaning devices mounted on the sensor housing assembly and configured to assist in removing debris and / or particles from the sensor housing assembly.

16. The method according to claim 15, wherein the sensor housing assembly includes an inclined surface serving as a window for a sensor within the sensor housing assembly, wherein the one or more cleaning devices are configured to clean the inclined surface, and wherein the method further includes: Removing debris and / or particles from the inclined surface of the sensor housing assembly.

17. The method according to claim 16, wherein the one or more cleaning devices include a fluid cleaning device, the fluid cleaning device including a spray manifold having spray holes thereon, wherein the manifold and the spray holes are configured to convey cleaning fluid from a source to the inclined surface, and wherein activating the one or more cleaning devices includes: Conveying, through the manifold and the spray holes, cleaning fluid from the source to the inclined surface.

18. The method according to claim 17, wherein the one or more cleaning devices include a wiper assembly, the wiper assembly including a wiper blade and a motor, wherein the wiper blade is configured to move along the inclined surface relative to a frame disposed on the sensor housing assembly, and wherein activating the one or more cleaning devices includes: Moving the wiper blade along the inclined surface relative to the frame.

19. The method according to claim 17, wherein the one or more cleaning devices include a blower assembly, the blower assembly including an air hose and an air source, wherein the air hose includes at least one slot for directing air from the air source to the inclined surface for cleaning, and wherein activating the one or more cleaning devices includes: Conveying, through the at least one slot of the air hose, air from the air source to the inclined surface.

20. The method according to claim 17, wherein the sensor housing assembly includes three cleaning devices, and wherein the three cleaning devices include: A fluid cleaning device including a spray manifold having spray holes thereon, wherein the manifold and the spray holes are configured to convey cleaning fluid from a source to the inclined surface, a wiper assembly including a wiper blade and a motor, wherein the wiper blade is configured to move along the inclined surface relative to a frame disposed on the sensor housing assembly, and a blower assembly including an air hose and an air source, wherein the air hose includes at least one slot for directing air from the air source to the inclined surface for cleaning, and wherein activating the one or more cleaning devices includes: Air is delivered from the air source to the inclined surface through the at least one groove of the air hose; Cleaning fluid is delivered from a source to the inclined surface through the manifold and the spray holes; and the wiper blade is moved relative to the frame along the inclined surface.

Images