Computer-aided driving

Abstract

A method for operating a vehicle may include sensing, by at least one sensor of the vehicle, an environment of the vehicle, the environment including a dynamic object; estimating an estimated impact of the dynamic object on future travel of the vehicle; wherein the estimation is in response to information stored in a dynamic database, and the information is about an estimated behavior of the dynamic object; and performing a driving-related operation of the vehicle based on the estimated impact.

Description

[0001] This application is a divisional application of a patent application with an application date of January 30, 2018, application number CN201880023864.1, and invention name “Computer Assisted Driving”.

[0002] Cross-references

[0003] This application claims priority to U.S. Provisional Patent Serial No. 62 / 457821, filed on February 11, 2017, and U.S. Provisional Patent Serial No. 62 / 455,656, filed on February 6, 2017, both of which are incorporated herein in their entirety. Technical Field

[0004] The present invention relates to computer assisted driving. Background Art

[0005] Autonomous vehicles and driver assistance systems are expected to be used to relieve the burden on human drivers. There is an increasing need to improve autonomous vehicles and driver assistance systems. Summary of the invention

[0006] A system, method and computer program product may be provided as shown in the description, claims and drawings.

[0007] A method for operating a vehicle may be provided, the method comprising sensing an environment of the vehicle, which may include dynamic objects, through at least one sensor of the vehicle; estimating an estimated impact of the dynamic objects on future propagation of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic objects; and performing driving-related operations on the vehicle based on the estimated impact.

[0008] A method for a vehicle may be provided, the method comprising repeating the following steps for each of a plurality of locations: selecting which type of vehicle sensor from a plurality of types of vehicle sensors to use for sensing the vehicle's environment when the vehicle is at the location; and using at least one selected type of vehicle sensor to sense the vehicle's environment when the vehicle is at the location; wherein the use may include sensing signals associated with a plurality of points of interest in the environment.

[0009] A method for maintaining a dynamic database may be provided, the method may include receiving information about different locations of the plurality of vehicles from a first plurality of vehicles; maintaining the dynamic database, wherein the dynamic database may include statistical data related to the behavior of dynamic objects within the different locations; and distributing relevant portions of the dynamic database to a second plurality of vehicles.

[0010] A method for facilitating updating a dynamic database may be provided, the method comprising receiving, via a vehicle, a portion of a dynamic database; wherein the dynamic database may include statistical data related to the behavior of dynamic objects within different locations; searching for a mismatch between the contents of the portion of the dynamic database and sensory information sensed by the vehicle; and reporting the mismatch to a computerized entity involved in maintaining the dynamic database.

[0011] A computer program product may be provided that may store instructions that, once executed by a computerized system installed in a vehicle, cause the computerized system to: sense an environment of the vehicle, which may include dynamic objects, via at least one sensor of the computerized system; estimate an estimated impact of the dynamic objects on future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic objects; and perform driving-related operations on the vehicle based on the estimated impact.

[0012] A computer program product may be provided that may store instructions that, once executed by a computerized system installed in a vehicle, cause the computerized system to: repeat the following steps for each of a plurality of locations: select which type of vehicle sensor from a plurality of types of vehicle sensors to use to sense the vehicle's environment when the vehicle is at the location; and use at least one selected type of vehicle sensor to sense the vehicle's environment when the vehicle is at the location; wherein the use may include sensing signals associated with a plurality of points of interest in the environment.

[0013] A computer program product may be provided that may store instructions that, once executed by a computerized system positioned outside a vehicle, cause the computerized system to: receive information about different locations of the plurality of vehicles from a first plurality of vehicles; maintain a dynamic database, wherein the dynamic database may include statistical data related to the behavior of dynamic objects within the different locations; and assign relevant portions of the dynamic database to a second plurality of vehicles.

[0014] A computer program product may be provided that may store instructions that, once executed by a computerized system located outside a vehicle, cause the computerized system to: receive a portion of a dynamic database; wherein the dynamic database may include statistical data related to the behavior of dynamic objects within different locations; search for a mismatch between the contents of the portion of the dynamic database and sensory information sensed by the vehicle; and report the mismatch to a computerized entity involved in maintaining the dynamic database.

[0015] A computerized system installed in a vehicle may be provided, wherein the computerized system may include at least one sensor configured to sense an environment of the vehicle, which environment may include dynamic objects; and a processor configured to (a) estimate an estimated impact of the dynamic objects on future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about estimated behavior of the dynamic objects; and (b) perform driving-related operations on the vehicle based on the estimated impact.

[0016] A computerized system installed in a vehicle can be provided, wherein the computerized system can include a processor and multiple types of sensors; wherein the computerized system is configured to repeat the following steps for each of multiple locations: selecting, by the processor, which type of sensor from the multiple types of sensors to use to sense the vehicle's environment when the vehicle is located at the location; and using at least one selected type of sensor to sense the vehicle's environment when the vehicle is located at the location; wherein the use can include sensing signals associated with multiple points of interest in the environment.

[0017] A computerized system positioned outside a vehicle may be provided, the computerized system may include a communication unit and a processor; wherein the communication unit is configured to receive information about different locations of the plurality of vehicles from a first plurality of vehicles; wherein the processor is configured to maintain a dynamic database, wherein the dynamic database may include statistical data related to the behavior of dynamic objects within different locations; wherein the communication unit is configured to distribute relevant portions of the dynamic database to a second plurality of vehicles.

[0018] A computerized system positioned outside a vehicle may be provided, the computerized system may include a communication unit and a processor; wherein the communication unit is configured to receive a portion of a dynamic database; wherein the dynamic database may include statistical data related to the behavior of dynamic objects in different locations; wherein the processor is configured to search for a mismatch between the contents of the portion of the dynamic database and sensory information sensed by the vehicle; and wherein the communication unit is configured to report the mismatch to a computerized entity involved in maintaining the dynamic database.

[0019] A method for monitoring a vehicle and operating another vehicle may be provided, the method may include monitoring the movement of the vehicle through at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle; estimating, through a computer of the other vehicle, an estimated interaction between a driver of the vehicle and the vehicle based on the movement of the first vehicle and the model of the vehicle; determining the state of the driver based on the estimated interaction; estimating the estimated impact of the vehicle on the future travel of the other vehicle; and performing driving-related operations on the other vehicle based on the estimated impact.

[0020] A method for monitoring a vehicle and operating another vehicle may be provided, the method may include monitoring the dynamic behavior of the vehicle through at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle; sensing the dynamic behavior of the vehicle through a computer; estimating the state of the vehicle; estimating the interaction between the vehicle and the other vehicle, and performing driving-related operations on the other vehicle based on the estimated state and the estimated interaction.

[0021] A method for estimating the future behavior of a vehicle and operating another vehicle can be provided, the method may include monitoring the movement of the vehicle during a monitoring period and through at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle; attempting to predict the future behavior of the vehicle based on a combination of at least two of the following elements: (a) light signals generated by the other vehicle during the monitoring period; (b) vehicle speed and acceleration during the monitoring period; (c) the spatial relationship between the vehicle and the lane during the monitoring period; (d) the environment of the vehicle during the monitoring period; estimating the estimated impact of the vehicle on the future travel of the other vehicle; and performing driving-related operations on the other vehicle based on the estimated impact.

[0022] A method for operating a vehicle may be provided, the method comprising receiving, via the vehicle, a request from at least one entity outside the vehicle to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode; determining, via a vehicle computer, whether to change the operating mode; and when it is determined to change the operating mode, changing the operating mode of the vehicle from the non-autonomous driving mode to the autonomous driving mode.

[0023] A computer program product may be provided that may store instructions that, once executed by a computerized system located within another vehicle, cause the computerized system to: monitor the movement of a vehicle; wherein the vehicle is different from the other vehicle; estimate an estimated interaction between a driver and the vehicle based on the movement of the vehicle and the model of the vehicle; determine the state of the driver based on the estimated interaction; estimate an estimated impact of the vehicle on the future travel of the other vehicle; and perform driving-related operations on the other vehicle based on the estimated impact.

[0024] A computer program product may be provided that may store instructions that, once executed by a computerized system located within another vehicle, cause the computerized system to: monitor the dynamic behavior of a vehicle; wherein the vehicle is different from the other vehicle; perceive the dynamic behavior of the vehicle; estimate the state of the vehicle; estimate the interaction between the vehicle and the other vehicle, and based on the estimated state and the estimated interaction, perform driving-related operations on the other vehicle.

[0025] A computer program product may be provided which may store instructions which, once executed by a computerized system located within another vehicle, cause the computerized system to: monitor the movement of a vehicle during a monitoring period; wherein the vehicle is different from the other vehicle; attempt to predict the future behavior of the vehicle based on a combination of at least two of the following elements: (a) light signals generated by the other vehicle during the monitoring period; (b) vehicle speed and acceleration during the monitoring period; (c) the spatial relationship between the vehicle and the lane during the monitoring period; (d) the environment of the vehicle during the monitoring period; estimate the estimated impact of the vehicle on the future progress of the other vehicle; and perform driving-related operations on the other vehicle based on the estimated impact.

[0026] A computer program product may be provided that may store instructions that, once executed by a computerized system located within a vehicle, cause the computerized system to: receive a request from at least one entity outside the vehicle to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode; determine whether the operating mode is to be changed; and when it is determined that the operating mode is to be changed, change the operating mode of the vehicle from the non-autonomous driving mode to the autonomous driving mode.

[0027] A computerized system installed in another vehicle can be provided, wherein the computerized system may include a processor and one or more sensors; wherein the one or more sensors are configured to monitor the movement of a vehicle; wherein the vehicle is different from the other vehicle; wherein the processor is configured to: estimate an estimated interaction between a driver and the vehicle based on the movement of the vehicle and the model of the vehicle; determine the driver's state based on the estimated interaction; estimate the estimated impact of the vehicle on the future travel of the other vehicle; and help perform driving-related operations on the other vehicle based on the estimated impact.

[0028] A computerized system installed in another vehicle can be provided, wherein the computerized system may include a processor and one or more sensors; wherein the one or more sensors are configured to monitor the dynamic behavior of the vehicle; wherein the vehicle is different from the other vehicle; wherein the processor is configured to: perceive the dynamic behavior of the vehicle; estimate the state of the vehicle; estimate the interaction between the vehicle and the other vehicle, and based on the estimated state and the estimated interaction, help perform driving-related operations on the other vehicle.

[0029] A computerized system installed in another vehicle can be provided, wherein the computerized system can include a processor and one or more sensors; wherein the one or more sensors are configured to monitor the movement of a vehicle during a monitoring period; wherein the vehicle is different from the other vehicle; wherein the processor is configured to: attempt to predict the future behavior of the vehicle based on a combination of at least two of the following elements: (a) light signals generated by the other vehicle during the monitoring period, (b) vehicle speed and acceleration during the monitoring period, and (c) a spatial relationship between the vehicle and a lane during the monitoring period, (d) the vehicle's environment during the monitoring period; estimate an estimated impact of the vehicle on the future travel of the other vehicle; and help perform driving-related operations on the other vehicle based on the estimated impact.

[0030] A computerized system installed in a vehicle may be provided, wherein the computerized system may include a processor and a communication unit; wherein the communication unit is configured to receive a request from at least one entity outside the vehicle to change the vehicle's operating mode from a non-autonomous driving mode to an autonomous driving mode; wherein the processor is configured to determine whether to change the operating mode; and when it is determined to change the operating mode, change the vehicle's operating mode from the non-autonomous driving mode to the autonomous driving mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to understand the invention and to see how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings.

[0032] Figure 1 An example of a vehicle and its environment is shown;

[0033] Figure 2 An example of a vehicle and its computerized system is shown;

[0034] Figure 3 An example of a vehicle and its computerized system is shown;

[0035] Figure 4 Examples of points of interest and surroundings for a vehicle are shown;

[0036] Figure 5 Examples of points of interest and surroundings for a vehicle are shown;

[0037] Figure 6 Examples of points of interest and surroundings for a vehicle are shown;

[0038] Figure 7 Examples of points of interest and surroundings for a vehicle are shown;

[0039] Figure 8 Examples of points of interest and surroundings for a vehicle are shown;

[0040] Fig. 9 Examples of points of interest and surroundings for a vehicle are shown;

[0041] Fig.10 An example of a registration process is shown;

[0042] Fig.11 An example of a vehicle and its computerized system is shown;

[0043] Fig.12 An example of a method is shown;

[0044] Fig.13 An example of a method is shown;

[0045] Fig.14 An example of a method is shown;

[0046] Fig.15 An example of a method is shown;

[0047] Fig.16 An example of a vehicle and its environment is shown;

[0048] Fig.17 An example of a vehicle and its environment is shown;

[0049] Fig.18An example of a method is shown;

[0050] Fig.19 An example of a method is shown;

[0051] Fig. 20 An example of a method is shown;

[0052] Fig.21 An example of a method is shown; and

[0053] Fig. 22 An example of a vehicle and its environment is shown. DETAILED DESCRIPTION

[0054] Any reference to a system shall, mutatis mutandis, apply to a method performed by the system and / or to a computer program product storing instructions which, when executed by the system, will cause the system to perform the method.

[0055] Any reference to a method shall, mutatis mutandis, apply to a system configured to perform the method and / or a computer program product storing instructions which, once executed by a system, will cause the system to perform the method.

[0056] Any reference to a computer program product shall apply, mutatis mutandis, to a method performed by a system and / or a system configured to execute instructions stored in a non-transitory computer-readable medium.

[0057] Any reference to a communication channel or communication unit may include any type of communication link and / or channel, such as wireless or wired, direct link or indirect link, cellular communication, satellite communication, Wi-Fi communication, etc. Any reference to a computerized system refers to one or more computers including at least one hardware processor, hardware storage unit, etc.

[0058] The term "and / or" means additionally or alternatively.

[0059] The terms "self-driving" and "autonomous" are used interchangeably.

[0060] A dynamic object may be any non-static object. An object may be considered dynamic when it is moving, when it is expected to move in a short period of time (e.g., a vehicle temporarily stopped before an intersection), etc. Non-limiting examples of dynamic objects may be vehicles, people, temporarily moving objects (e.g., a tree moved by the wind), etc.

[0061] The driving-related operation of any vehicle may involve multiple units / components of the vehicle. Each unit / component may assist in executing driving-related parameters in various ways, including but not limited to instructing, triggering and / or requesting another unit or component to perform an action or prevent an action from being performed in advance. The assistance may also include performing an action or avoiding performing an action. The unit / component may or may not belong to a computerized system installed inside or outside the vehicle.

[0062] A dynamic database is a database that can store information about the expected behavior of dynamic objects (e.g., the expected acceleration of a vehicle of a particular model and at a particular intersection). The information can include statistical data related to previously monitored behavior. The dynamic database can be maintained by a computerized system located external to the vehicle.

[0063] The dynamic database may also include information about changes in environmental conditions.

[0064] The statistical data may or may not be time-sensitive. The statistical data may be provided based on at least one of the following (or a combination of at least two, three, four, five, six or more of the following): time window, location, direction of travel, similar locations, prototype locations, dynamic objects, vehicle models, drivers, roads, lanes, road sections, vehicle types, each intersection, traffic lights, buildings, risk levels, speeds, weather conditions, etc.

[0065] For example, the statistics may indicate at least one of the following:

[0066] a. Time-dependent distribution of vehicle types on the lane.

[0067] b. The behavior of dynamic objects within one or more intersections.

[0068] c. Relationships between the states of one or more traffic lights located at one or more intersections and the behaviors of dynamic objects within the one or more intersections.

[0069] d. Speeds in different directions within one or more intersections.

[0070] e. Different types of vehicles within one or more intersections.

[0071] f. The behavior of different types of vehicles and one or more persons at one or more intersections.

[0072] g. Behavior of dynamic objects selected from vehicles and people near one or more buildings.

[0073] Non-limiting examples of information that may be stored in the dynamic database or another database and required to be associated with information stored in the dynamic database may include, for example, the following:

[0074] The behavior of the car (based on road type conditions and lane markings.) For example:

[0075] a.Trucks tend to slow down at this location.

[0076] b. The car slows down near the accident (perhaps there is something interesting to see).

[0077] c. Cars sometimes turn left even when there is no legal turn.

[0078] d. Motorcycles tend to ignore white lines.

[0079] e. Police cars in this position tend to cause traffic to slow down.

[0080] f. Pepsi (or Coca-Cola) car slows down (close to the delivery location)

[0081] Car's behavior (according to the driver)

[0082] a. Young people tend to speed up

[0083] b. Young people (here) ignore stop signs 10%

[0084] c. Women turn left 45%

[0085] d. Military left turn 78%

[0086] Car behavior (according to time of day)

[0087] a.During the school term, many drivers tend to double park.

[0088] b. Drivers often turn left at this location to go to church on Sundays.

[0089] Pedestrian Behavior

[0090] a. Based on time of day, based on youth, adults, men, women, groups.

[0091] b. Based on what they do (looking at the phone, playing). Instead of talking about general information (the person looking at the phone can't see what's around him), talk about specific location information, where the kids are more likely to jump into the road than others.

[0092] Given common behaviors of a pedestrian and car combination, for example:

[0093] a.Beside the school, cars parked on the other side of the road increase the probability that the child will jump into the road and run towards the car. This is not true in every location and at all times, but only in certain combinations.

[0094] Animals or pets and their behavior

[0095] a. The dog got frightened here and jumped onto the road

[0096] b. Pets tend to run on the roads near dog parks.

[0097] c. Deer Crossing

[0098] Temporary traffic warning:

[0099] a. Rockfall warning.

[0100] b. A scrubbed road.

[0101] c. Narrow shoulders.

[0102] Glare and flare conditions:

[0103] a. Glare at sunset in certain road directions and at certain times

[0104] b. Glare from building windows at sunrise or sunset.

[0105] c. The glare of a flashlight on a wet road at night.

[0106] Wind conditions:

[0107] a. Strong wind

[0108] b. Related to the forecast, but for a specific road location. Where crosswinds are expected.

[0109] This affects expectations about the car, and also about other cars.

[0110] Road and surface conditions:

[0111] a. This location sometimes has oil on the road.

[0112] b. This road may be icy if (weather conditions, or information from other roads in the area).

[0113] Pattern of lights, pattern of road signs

[0114] a. Road signs and billboards can influence driver behavior. Generally speaking, if there is a road sign that is sexual in nature, men will slow down, especially in a movie that shows this. The reaction during this part of the movie is…

[0115] b. The red light lasts for 20 seconds.

[0116] c. Street lights start at 20:00 on weekdays.

[0117] Lane:

[0118] a. Distribution of car types on the lane (public transport / expensive / cheap cars).

[0119] b. Speed ​​distribution on lanes according to time of day. (Can be used to select lanes)

[0120] For example, in which lane is the approach light better?

[0121] c. Information about parking in driveways according to time of day.

[0122] d. Information about public transportation (public transportation, bus stops only).

[0123] Intersection:

[0124] a. Speed ​​distribution in each direction at the intersection according to time of day. (same as lanes)

[0125] b. The behavior of cars at intersections based on car type and time of day (e.g. a Lamborghini is more likely to accelerate at a yellow light than a Fiat).

[0126] Metadata for buildings:

[0127] a. Building type (kindergarten, elementary school, high school, police station, parking lot, store, construction site, etc.).

[0128] b. The distribution of vehicles around buildings (e.g., more cyclists around high schools, more trucks around construction sites).

[0129] c. The distribution of pedestrians around buildings (for example, there are more children around schools).

[0130] d. Vehicle behavior around buildings (e.g., slowing down near parking lots).

[0131] Vehicle metadata:

[0132] a. Acceleration capability and maximum speed of each vehicle type:

[0133] That is, the Lamborghini accelerates much faster than the Fiat.

[0134] Information regarding static objects, navigation information, points of interest, and / or any other type of information that may affect the driving of the vehicle may be maintained in another database or may be stored in the computerized system in any manner.

[0135] A dynamic database (also referred to as a dynamic database or DDB) and various vehicles for use together are provided.

[0136] The DDB can make a request to the car, asking for the information it needs. There can be various types of information that the DDB can request, and how to request that information.

[0137] Additionally, in certain circumstances, the DDB can send a movement request to the car in order to collect the required information.

[0138] A DDB may be used in addition to navigation. A static dynamic database may only include information related to static objects. One use is to add mapping information for dynamic items that are not present in the static dynamic database. These include statistics about behavior (e.g., a child running into the road) that are not part of the static database, but are information learned about the place.

[0139] The DDB may also include suggested actions - given an obstacle (which may also appear in the static dynamic database), there is a suggestion for how to get through it. Suggestions on how to navigate and get through it, as well as the temporary hazards themselves will be part of the DDB. The suggestions can be car-based, and different cars and different driver preferences will receive different suggestions based on their speed and location.

[0140] DDB may be different from static dynamic database because it is not only about static objects.

[0141] For this DDB, we need a variety of information sources, mainly cars and people, but also the internet, and many other sources. This is very useful for self-driving cars, giving them the right context.

[0142] Objects in the DDB help draw up maps and make it easier for the car to localize itself, even if those objects are not needed for other navigation tasks.

[0143] Draw maps to predict the behavior and condition of dynamic objects

[0144] Dynamic maps will be used to predict the future behavior of objects that are not part of the static map but are relevant to driving.

[0145] This helps in the process of predicting the likely position of the car at the next time period. Some cars can turn at higher speeds, so while we expect one type of car to probably not turn, another type of car may have a higher probability of turning.

[0146] These examples target dynamic properties of a location. In this section we are not talking about temporary properties that use other solutions, such as car-to-car communication. Temporary properties might include specific road hazards (rocks, oil in the road) or other situations that are not permanent properties of the place, like a parked car, but happens to be there.

[0147] To use this dynamic information, we introduce the following process:

[0148] a. Collect information.

[0149] b. Calculate statistical data.

[0150] c. Efficiently store information in DDB.

[0151] d. Get this information to move the car.

[0152] e. Use this information to improve driving.

[0153] Collecting Information

[0154] Information is collected over time using autonomous car sensors and other sensors. For example, each autonomous car will record the behavior of the vehicles around it and can send this information to a central database. It can record any of the above information and send it to the DDB. Sensors can be fixed sensors in the position sensors on the vehicle, or can be any sensor used to report to the DDB. Another source of information is car self-reporting: if we know the car's driving plan, we know, for example, whether it turns left at an intersection, and can use this for our statistics. We may also know the speed of the car during the journey and other more precise information (this information is collected for map statistics). In addition, in the near future, people will be walking around with cameras all the time (thanks to augmented reality applications). They can be used as dynamic sensors, sending data to many databases for processing, and will also collect some data from people (such as their speed and behavior at different locations).

[0155] Data can be sent as raw data or event data. If sent as raw data, what the sensor sees or slightly processed, the DDB will record the event of the data. If sent as events, the calculations will be done locally (or in another DB) and sent to the DDB as events. Any combination is of course possible. The DDB can also ask sensors for data, in which case the sensor can send raw data, or events, depending on the request.

[0156] This information can be gathered from the internet (e.g. stag migration, weather, school started late today), this information can be gathered from traffic control (lighting schedules). The information needs to get to the DDB, so it needs to be actively collected. Some of these data points are not currently being sent from the car to the DDB. For example, if a car currently crosses a white line, the passing car and the observing car may not share the white line with the DB.

[0157] There are multiple reasons why the car does not share every single information with the database. The first is technical, sharing the information from all sensors would bring a huge communication burden, which is not possible with our technology. Second, it may violate all privacy issues (such as tracking people).

[0158] We may be interested in specific events (car passing lane, car turning left, etc.). We can ask our sensors or 3D mapping system (carto) to report only those types of events, or at least those it is smart enough to recognize. It is likely that any event that the car can use for driving, it also knows how to recognize, so we assume that it is most of the events (an autonomous car will be able to recognize those types of events based on its cognitive and navigational abilities).

[0159] The car is informed that DDB needs information and sends DDB discreet events (not sensor information to be analyzed). This type of discrete event may look like- “Truck at location X travelling in direction Y around time T (can be randomized within five minutes to maintain anonymity) crossing white lane 50 cm at speed of about 80 km / hr.” Sending such high-level events, even thousands or even millions per hour, requires very limited bandwidth which is insignificant for cars. Also, to avoid privacy issues, we can add a random dT (up to 5 minutes) to the timestamp of the reported event and ensure the anonymity of the reporter.

[0160] Calculate statistics

[0161] The calculation of statistics can be done in a number of ways, some of which are listed below.

[0162] The naive approach is to take the data collected from the first phase for each location and use that. The problem with this approach is that the raw data is rare, and we won't have the same amount of information for each location. For example, the percentage of cars with male drivers crossing the white line might be based on a very small sample.

[0163] The second approach is to create prototype locations. Some examples of prototype locations are a section of a two-lane highway, a section of a three-lane highway with a divider, or a four-way stop intersection, or a four-way stop intersection with one of the roads being busier, a one-way street next to an elementary school, etc. Each location on the map corresponds to some prototype location that best describes it. In this approach, statistics are combined using the information we have collected about the specific location and general statistics related to the prototype. When using this approach, we prefer to report statistics to the user for each location, but if we have not collected enough data, we can report prototype location statistics.

[0164] A third approach is to create a similarity function between locations. Each location on a map is unique, but information from other similar places is often relevant as well, and the more similar they are, the more relevant they are. Locations may be more similar to other locations in terms of some given parameters, which can be learned by a learning algorithm. For example: a scene where a pedestrian is crossing the road next to a school. The important parameter may be the age of the pedestrian in order to predict his behavior. Another example of a car passing a white line on a curve, here the main parameters may be the road curvature, the speed of the car, or the road visibility. In this approach, the most influential features will be learned from the data. The similarities between locations are studied based on their parameters, and then used to estimate the behavior of cars or pedestrians about locations where our data is sparse. Since the similarity between locations is based on the parameters we want to estimate, it is likely that different features of each location will be enhanced.

[0165] When we divide the world into prototype locations, the more locations we have, the better the fit at any location. Better fit also means better performance. However, the more prototype locations we have, the fewer locations that match each prototype, which means less data sharing between locations. There is an obvious trade-off between the number of prototype locations and how descriptive they are of the prototype. On the one hand, we can describe all four-lane highway segments with a single prototype. On the other hand, we can describe a prototype location as a four-lane highway segment with an elevation of 500-540 feet, a northerly direction of 139-147 degrees, a border or 1.5-17 yards without a median, and 2-2.3 yards between opposing lanes. Having a more descriptive prototype location is very useful for predicting behavior based on other places, and is also useful for compressing information. But it reduces the number of places we can compare.

[0166] If we don't divide the world into prototypical locations, we can still use machine learning to calculate location similarities. Once we have that, we know what information learned about similar places might be relevant.

[0167] A basic algorithm for finding the correct values ​​for different attributes might include the following steps:

[0168] Step 1 - Use direct observations of a specific location to calculate the property value at that location

[0169] Step 2 - Given the attribute values ​​computed in 1, the physical properties of the location, and any additional information about it, compute similarity functions with other locations. Similarity can be general or per attribute. General means that locations A and B are similar and the information we learn about A is relevant to B. Attribute specific means that if locations A and B are similar in some attribute, then we can learn some properties about B from A.

[0170] Step 3 - For each attribute, calculate a value using the value calculated in step 1 and a weighted average of the values ​​at similar locations.

[0171] A second possible algorithm could be

[0172] Step 1 - Use direct observations for a specific location to calculate the value of the attribute about that location

[0173] Step 2 - Classify the location as a prototype location (i.e. rural road next to a school)

[0174] Step 3 - Given the attribute values ​​calculated in step 1, classify the prototype location from step 2 using a weighted average of the measured value and the value in the same prototype location.

[0175] Efficient storage of information in DB

[0176] We want to store information about each location and then send this information to the car. We propose several methods to efficiently store the database information.

[0177] We could store the information for each location individually into the DB, which is a very simple approach, but would likely result in a storage footprint that is too large. Note that we are not storing the entire map, but only the attributes extracted from it, so ours is orders of magnitude smaller than saving the entire exact map, but still very large.

[0178] If we use the prototype position approach, then each position is stored as the prototype position + the difference between the specific position and the prototype. This is fairly efficient both to store and to transmit to the car (which may have prototype position statistics in its own database).

[0179] An intermediate approach would be to use only prototype positions in DB to make it more efficient. For each position we would store the prototype position index + difference of the current position. When a car requests to get information about a specific position we would send her the statistics of the specific position, so basically the car is not aware of the prototype position approach.

[0180] Generally speaking, our DB holds one entry per location. We define a location as a geographic area of ​​a certain size such that all points within that area have similar attributes. So basically, the larger the location area, the less storage space we need to store all the location information and the less communication with the cars. But using a large area for each location can lead to reduced accuracy. For example, consider a 10 km long section of highway. We could take the entire section and associate it as a single location with a single set of attributes. On the other hand, we could divide the section into 10 or even 100 small sections of the road and save each of them as a different location with its own set of attributes. In this case, some attributes (such as the average speed of the car) may be very similar in all locations, while other attributes may be different (such as the density of trees along the road).

[0181] For kilometer-long segments, these parameters may be very similar, but for smaller segments, these parameters will be more precise. Parameters like pass probability and speed are unlikely to be exactly the same. If they are - we can treat it as one location. If the difference is not significant compared to one location. If the difference is important for the car to drive, then we need to split the location. Splitting the location affects the storage size and accuracy of the DB and the communication bandwidth between the DDB and the car.

[0182] Retrieve this information to move the car

[0183] Depending on the storage paradigm used (see previous section), every time the car arrives at a known location, it can query the DDB to obtain dynamic information relevant to it. It is also possible that the DDB will push this information. With this approach, when the car is recognized to be interested in the information, the DDB will send the information to the car based on the car's filters.

[0184] If the car has stored information about prototype position statistics, then the DDB can just send the index of the prototype position and the difference between the prototype and the current position. The delta takes up very little space, so the whole communication process is very fast and efficient. Since the DDB probably knows the car's driving algorithm, it only needs to send information about it to make a difference. It can calculate when the information sent is different from the default, and will make the difference. This is expected to be a very small amount of data.

[0185] Use of information to improve driving

[0186] This database will contain among other things the behavior of many cars at many locations, and this data will be used to predict the behavior of moving objects (cars, pedestrians, animals, etc.) in new events. The analysis will take into account the location and the type of scenario (intersection, type of intersection, one-way street, multi-lane road, connection to a highway, etc.).

[0187] For each intersection location on the road, we will know how each car type (e.g. private, track, or even car model - BMW Mazda) will accelerate when the light turns green (or behave when the light turns red). This may change at different intersections and in different countries (Italy vs. the United States). The behavior of children next to the school will deepen at the hour, what school is it (middle school, high school: for example, high schools will have electric bikes), with deep learning we can automatically find it and use this information to make driving decisions.

[0188] Figure 1 Autonomous vehicle 11 is shown approaching intersection 21 .

[0189] The vehicle 11 may include units / components, such as three sensors (or three types of sensors) VS131, VS232, VS333, a computer 34 that can execute any method related to the vehicle in the specification, a communication unit 35 that can communicate with entities outside the vehicle, and a vehicle communication unit 36 ​​that can communicate with vehicle units / systems (for example, a vehicle system 37 that may include an engine, an engine controller, a gear unit, an air conditioning unit, a brake, a multimedia unit, etc.).

[0190] The computer 34 and zero or more of the other components / units described above may form a computerized system 30 installed in a vehicle.

[0191] An autonomous car is driving down the street, approaching intersection 21. The autonomous car's 10 system detects that a blue Mercedes car 12 is about to cross the intersection in the other direction, and locates its position. The system checks the predicted behavior of the Mercedes model at a specific location at a specific time of day. The query is analyzed based on past events in the area, taking into account the time of day, location, and nearby buildings. Predictions are made about the car's future behavior (speed, acceleration, etc.) (e.g., the Mercedes will speed up and cross the intersection at a yellow light). The autonomous car uses this data to make its own driving decisions (e.g., slow down and let the Mercedes safely through the intersection).

[0192] The autonomous car 10 may notice that another car 14 is double parked on the other side of the street (parallel to the parked car 13). Normally this is not a problem. However, if the location is close to a school 23, we get a warning, which may mean that a child will want to cross the street to get into that car from our direction. The warning will not be displayed after school. Similar warnings will appear whenever we pass a place where children gather, such as a swimming pool, a play area, a movie theater, and this may depend on the age of the children. Of course, this warning is not limited to children.

[0193] Database and Storage

[0194] The following sections describe two implementation styles that determine how the database is structured and how it works:

[0195] Per-object metadata - The database will include additional metadata for locations and points of interest on the map. The additional metadata will be used in the driving decision process.

[0196] Per-point metadata - The database will contain all the available information for each location and let deep learning handle how to find logic in it. The idea is to mark out a grid of points on the road and track the behavior of cars passing by each point.

[0197] Per-object metadata

[0198] Our database will include additional metadata for locations and points of interest on the map. The additional metadata will be used in the driving decision process.

[0199] Remember that metadata is "static" and does not update every second, so it only contains roughly permanent information. For example, it cannot contain information about obstacles on the road, because these obstacles tend to appear and disappear frequently.

[0200] Metadata is stored per object and its location, making searching and extracting data from tables fast and efficient.

[0201] Example of metadata that will be stored for each object:

[0202] Lane metadata:

[0203] a. Distribution of car types (public transport / expensive / cheap cars) on the lanes.

[0204] b. Speed ​​distribution on the lanes according to time of day.

[0205] c. Information about parking in a driveway according to time of day or day of the week.

[0206] d. Information about public transportation (public transportation, bus stops only).

[0207] Metadata for intersections:

[0208] a. The basic rules of the intersection remain unchanged (U-turns are not allowed, right-turn pedestrians are not allowed to cross the street).

[0209] b. Speed ​​distribution at the intersection in each direction according to time of day.

[0210] c. The behavior of cars at the intersection based on car type and time of day (e.g., a Lamborghini is more likely to accelerate at a yellow light than a Fiat).

[0211] Metadata for buildings:

[0212] a. Type of building (kindergarten, elementary school, high school, police station, parking lot, store, construction site, etc.).

[0213] b. The distribution of vehicles around buildings (e.g., more cyclists near high schools, more trucks around construction sites).

[0214] c. The distribution of pedestrians around buildings (for example, there are more children around schools).

[0215] d. Vehicle behavior around buildings (e.g., slowing down near parking lots).

[0216] Vehicle metadata:

[0217] a. Distribution of vehicle models in each region (city, state).

[0218] b. The distribution of driving speeds for each model under different conditions (e.g., a Lamborghini may have a higher average speed on the highway than a Fiat).

[0219] c. The acceleration capability and maximum speed of each car (for example, a Lamborghini can accelerate much faster than a Fiat). This helps predict where the car will be in the next frame.

[0220] Per-point metadata

[0221] Another way to store metadata on a map is to save all the available information for each location and let deep learning handle how to find the logic in it. The idea is to mark out a grid of points on the road and track the behavior of cars passing by each point.

[0222] In this approach, our prediction algorithm is not based on analyzing the behavior of the car and understanding the logic behind the behavior. Instead, we let deep learning analyze training examples, where each training example is an event of a car passing a certain point on the road with a certain speed and acceleration. The trained neural network will embed the behavior of the car in different areas in different scenarios. Finally, given a test example of a car about to pass a certain point on the road, the neural network will predict its speed and acceleration.

[0223] train

[0224] The network training can be based on thousands of training examples. Each training example includes details of an event where a car passes a specific point at a certain point in time with a certain speed and acceleration. The training vector should include the following information:

[0225] a. Car type

[0226] b. Car coordinate positioning

[0227] c. Time of day

[0228] d. Heading.

[0229] The neural network will be trained to take in an input vector and predict velocity and acceleration as the network output.

[0230] The neural network will eventually learn the driving logic at each point. For example, if a certain POI is close to a school, most cars will slow down in that area. Another example, a Mercedes is more likely to speed up than a Fiat after an intersection. The neural network will learn these behaviors of the cars, and when a new car approaches this point, it will predict its speed and acceleration in that POI.

[0231] The neural network will be trained offline based on training examples reported to the cloud.

[0232] test

[0233] Testing of the neural network occurs in real time when the system wants to predict how a specific car will behave in a certain location. The system sends a query to the cloud including the car type / model, the car's location, and the time of day.

[0234] The neural network will use the given examples and predict the car's speed and acceleration. The predictions are sent back to the requesting car and used in its driving decisions.

[0235] Reduce data storage

[0236] Storing a complete neural network for each point would consume a lot of storage space. However, we understand that close points on the road are likely to have similar neural networks. We propose a method to fuse similar networks if their learned weights are similar enough.

[0237] How do I update the metadata map?

[0238] The metadata in your map is updated in two main ways:

[0239] Self-Statistics – An autonomous car connected to the cloud will send its own metadata and statistics about driving speed, acceleration.

[0240] Scene Statistics – Autonomous vehicles detect and measure information about other vehicles and pedestrians as they drive. This information is sent to the cloud as reports about a specific car, being at a certain location, traveling at a certain speed, etc.

[0241] External updates – Information from sources other than the autonomous vehicle’s systems. For example: updates from municipalities regarding public transportation or road closures for construction.

[0242] The acceleration of my car and other cars

[0243] Create maps to predict static obstacles and road conditions

[0244] The system will be able to control the speed of the vehicle, or suggest a speed, to navigate obstacles and road humps. The sensors will measure the 3D structure, and based on that it will set the direction and speed of the vehicle, which will be based on the vehicle suspension system and other vehicle characteristics.

[0245] collision

[0246] For each vehicle, we can prepare an optimal driving speed table based on experimental data for different road humps. The parameter space we should check can be shown in the following table or other tables. A table based on this parameter space will be created as described in the following experimental data table example:

[0247]

[0248] When the vehicle approaches a road hump, the system calculates the 3D structure of the road hump and finds its features. Based on this, it finds the road hump that is most like it (its parameters and 3D shape are closest to it) in the table. Then, based on this table, it chooses the best speed and approach method.

[0249] Additionally, in the conversion table, there will be known road crowns with specific criteria that can also be identified using other features such as color, pattern, shape, etc. If the system recognizes a known road crown, it will use its specifications.

[0250] The road humps will be dynamically mapped so in situations where we know our GPS location we can tell which specific road hump we are approaching and its characteristics. In these cases the system will still verify that it has not changed and report it to the dynamic database.

[0251] From the above it is clear that sometimes vehicle sensors are used to understand road humps. There are two problems with this, the first is that it requires a lot of computation and the second is that it is unlikely that the car sensors can inherently map the entire road humps / obstacles as some of them are hidden and the hidden parts may be important.

[0252] We also have solutions where the road humps are part of the mapping DB. The DB may not include a description of the road humps (this is not required) but directions on how to navigate around the road humps. The directions will include speeds and approach types for different vehicle classes.

[0253] Learning will be done by observing the behavior of passing vehicles as they pass by.

[0254] Use the car or phone accelerometer or other sensors to see the approach and speed they use, and how the car is affected by the road hump. In general, it may be advantageous to examine how locals treat road hump as they gather knowledge. Therefore, the correct approach may be that of locals with similar vehicles.

[0255] A road hump might be a permanent fixture, but the description applies to any obstacle, some of which are temporary. Oil in the road is another example where the map understands the hazard and communicates it as best it can to approaching cars.

[0256] The DB can ask the car to approach the hazard in a specific way, and it can also learn about the hazard.

[0257] Interaction between database and sensors.

[0258] If the car provides information to DDB, there may be a payment involved, especially if the car has undergone some changes during its run. That option needs to be mentioned.

[0259] The DDR can be stored in the cloud or in another central database (a version of it or part of it can be kept in the car). The DDB will communicate with many cars: it will send information to them and receive information from them. It will save information obtained from many cars. In addition, it can access other information such as weather, road construction information, traffic information, new road introduction GUI, etc.

[0260] Figure 2 Communication between the car and the map database will be two-way.

[0261] There may be a two-way communication link between the computerized system 100 and the vehicles 31, 32, 33, and 34. The computerized system may include a communication module 130, a dynamic database 110, a static database, and one or more processors / computers or servers 140 and a storage unit 180.

[0262] The car will get a portion of the database based on the car's location (e.g. we might want to always have a map of 100km around the car), or access updates and changes in the central database (e.g. 1. there was road construction and the database changed, 2. POIs changed). Additionally, the DDB can ask the car to send data based on different criteria.

[0263] The car itself will send data to DDB based on DDB's request, or newly discovered data encountered on the road. In addition, DDB can ask the car to send data based on some other criteria.

[0264] Figure 3 A computerized system 100 is shown in communication with a vehicle 31 .

[0265] Information from DDB to Cars

[0266] Update Database - Sends the car-related parts of the database.

[0267] This happens in different situations, e.g.

[0268] Changes in car position

[0269] Changes in the database

[0270] Request information. Needs updating. If the database needs more information based on car reports or external reports

[0271] Uncovered areas (e.g. introduction of new roads into the database)

[0272] Fall / New Traffic Signs / Lights

[0273] Regional Updates

[0274] Update road construction.

[0275] Update POI

[0276] Reporting the behavior of moving objects

[0277] News from Cars to DDB

[0278] Reports of mismatches between the database and the car's sensor data. For example

[0279] Dropped / New Traffic Signs / Lights

[0280] Regional Updates

[0281] Update road construction

[0282] Update POI

[0283] Reporting the behavior of moving objects

[0284] Request a database of map information based on my location (in the case where the car is at the end of its map)

[0285] Answer to information request. Requires update. Sends scene information. For example

[0286] a. Dropped / new traffic signs / lights

[0287] b. Regional Update

[0288] c. Update road construction

[0289] d. Update POI

[0290] Reports the behavior of moving objects. For example

[0291] a. Cars at the intersection

[0292] b. Pedestrians at a certain location

[0293] This two-way communication can be described as: messages from DDB to the car and messages from the car to DDB.

[0294] News from DDB to Automotive

[0295] Update database - send the car-related part of the database. This can happen in different situations, for example:

[0296] Changes in car position

[0297] Changes in the database.

[0298] Request Information: Updates are required. If the database requires more information based on vehicle reports or external reports:

[0299] Uncovered areas (e.g., introduction of new roads into the database)

[0300] Dropped / New Traffic Signs / Lights

[0301] Regional Updates

[0302] Update road construction

[0303] Update POI

[0304] Reports the behavior of moving objects.

[0305] Request a change to the car navigation system to move to a different location to collect information

[0306] News from Cars to DDB

[0307] Reports of mismatches between the database and the car's sensor data. For example

[0308] Dropped / New Traffic Signs / Lights

[0309] Regional Updates

[0310] Update road construction

[0311] Update POI

[0312] Reports the behavior of moving objects.

[0313] Request map information database based on my location (in case the car is at the end of the map).

[0314] Answer to a request for information. Requires an update. Sends information about the scenario. For example:

[0315] Dropped / New Traffic Signs / Lights

[0316] Regional Updates

[0317] Update road construction

[0318] Update POIs.

[0319] Reports the behavior of a moving object. For example:

[0320] Cars at intersection

[0321] Pedestrians at a certain location.

[0322] Examples could be a stop sign falling, road construction, etc. The flow in this case could be:

[0323] The car using the database discovered that the map database did not match the 3D space of the measurements acquired by its sensors. The car reported this mismatch to the database.

[0324] Therefore, the database is sending update requests to several nearby cars.

[0325] The cars arrive at the location, take measurements and capture images of the location, and then send their data back to the database.

[0326] After enough information is collected, the database is updated.

[0327] This update was sent to all cars.

[0328] The most important part of this is DDB asking the car for information. Currently, DDB has two-way communication with the car. The car is sending DDB the obvious information it needs, like where the map is wrong. All of this may be new, but it may not be.

[0329] The important thing is that the DDB asks the car for information, a location, an object, or anything else. By default, the car doesn't send information. It could be something that's not on the road, or something that's not relevant to the car, or something that's just not normal to send, like the color of the road, or a picture of a tree. The DDB can ask for such information by telling the car what it needs, and the car can send the information to the DDB. This non-standard case is what we envision is needed to implement such a DDB.

[0330] Additionally, DDB may need the car to move a bit to collect this information. It could move to better place its sensors. It could also move so it can check for hazards in a specific way. It might take a different route (not unlike a car) to help DDB. Take a different lane, etc. DDB's ability to have the car move in different ways to collect data for it is also an important part of the communication protocol (and perhaps the payment protocol).

[0331] So, DDB can ask the car for specific sensing information. It can ask the car to move in different ways to gather its information. These things are major new additions to the communication between DDB and the car.

[0332] Positioning and mapping

[0333] This section describes our approach to localization and mapping. One of the most fundamental problems for an autonomous vehicle is to keep track of its own position and localize itself in a map. This process is called SLAM (Simultaneous Localization and Mapping) or odometry.

[0334] In our discussion, we will use multiple coordinate systems as part of our terminology. The first coordinate system is the GPS coordinate system, in which each point is a triple measurement (latitude, longitude, altitude) relative to the center of the Earth. In this coordinate system, every point on Earth has a unique description. The second coordinate system we will use is the car's coordinate system, where the origin is the driver's starting position and each point is measured relative to that starting point as (x, y, z).

[0335] Map and POI structure

[0336] Map Structure

[0337] Storing a map of the driving environment is the process used to localize and pinpoint the exact position of the car in the environment. The process of storing the map in a database is a question of what you store and how you store it.

[0338] A naive approach would be to "save everything", which means saving a complete 3D map of voxels with the exact position and color of each voxel. Very inefficient, but you can render an image of that area from any viewpoint. This is not actually necessary, since we can easily know the color from the previous frame.

[0339] On the other hand, we can only save information about "points of interest" (POIs) and their locations in the 3D map (or simplified 3D map). POIs can be the corners of buildings, the center of traffic signs, etc. By storing this information, we can eliminate "points of interest" searches (such as SIFT) and save computing time. In addition, by putting POIs into a central database, we can collect many POIs from many cars (and many locations where cars have been in the past), and we can evaluate how good each POI is. The best POI for 3D mapping is then dynamically selected.

[0340] Figure 4 Examples of points of interest and vehicle surroundings are shown.

[0341] Points of interest 51, 52, 53, 54, 55, and 56 may be associated with distinguishable locations, such as one or more corners of buildings 41 and 44, a corner of traffic sign 45, and the middle of road 46. Figure 4 Some buildings (42 and 43) are not assigned POIs.

[0342] Select POI

[0343] The main challenge is how to select POIs. On the one hand, POIs should be easy to find and identify in the image. On the other hand, POIs should also be distinct in depth and detectable by LiDAR.

[0344] Furthermore, we want any given scene to have ~100 available POIs within 100 meters of the car. POIs should be spread across all distance ranges and in full view. We assume that an average scene will have ~100 available points, and only half of them will be visible to the system at a given point in time.

[0345] For example, a red dot in a window might be easily identifiable by a camera image, but the lidar cannot shoot the laser at that point due to reflections from the glass. A better choice would be a green dot in the corner of the window. Another example is a red stop sign, which is difficult to identify in an image due to the red background, but is easily identifiable using lidar points and shots.

[0346] Figure 5 and Figure 6 Examples of points of interest and vehicle surroundings are shown.

[0347] exist Figure 5 In the example shown in FIG. 6 , the building is located on the left side of the vehicle. The first POI 61 is located on the window 63, while the second POI 62 is located on the gap 64 between the windows. Under certain conditions, it may be better to select the second POI.

[0348] A good point should first be static and distinguishable by depth and color. In addition, it should be located in a position where small misalignments of the background of the measured object do not cause changes in the distance. Thus, a good POI (e.g. Figure 6 The POI 69) would be, for example, a sign (e.g. Figure 6 68), where we should point to the center of the sign rather than its edge.

[0349] Figure 7 Several good options (indicated by circles 72) and some bad options (indicated by + symbols 71) for the POIs are shown.

[0350] So far, we have discussed the distinction of POIs in a given scene, but our POIs must be detectable from different viewpoints and at different times of the day. We divide the possible POIs into four groups (see for example Figure 8 ):

[0351] Static points on static objects (81) - For example: points on buildings, road signs, traffic signs, telephone poles. These points are best suited for our purposes. They never move, and are easy to find as long as they are not hidden.

[0352] Static points on dynamic objects (82) - for example: points on a parked car. These points can be used to locate a given car at a given time, as they remain stationary during the time the car is driving through the area and are reliable for this process. The problem is that these points may move at some point in the future. We can use these POIs in the process of locating multiple cars passing through the area. However, we should be skeptical of this and its reliability. The map update process should be able to clear these POIs from the DDB once they are no longer reliable (e.g. the parked car has moved)

[0353] Dynamic points on static objects (83) - such as shadows on the road or clouds in the sky. This category of POIs is similar to the second category of POIs described above, except that in this category we know in advance when they become irrelevant and can automatically purge them from the DDB after a known period of time. For example, a point on the road may be indistinguishable due to a shadow of a building falling on the road being stationary for about 1 hour. Afterwards, the shadow moves and the point is no longer reliable. Another example is clouds in the sky, which are easily camouflaged and can be used to locate multiple cars driving in the area for about 1 minute (depending on the wind). This POI will enter the DB with an "expiration time" and will be automatically deleted after that time.

[0354] Dynamic points (84) on dynamic objects - such as moving cars or pedestrians. These points are the worst for the localization process and cannot be used under any circumstances. These POIs never enter the DDB and cannot be used for the localization process at any time.

[0355] Using the above definition, only "type 1" POIs are inserted into the DDB when building a map. During the map update process, when the car is driving and using the map, "type 2" and "type 3" points can enter and exit the DDB according to their relevance.

[0356] Storing POIs

[0357] So far, we discussed how to select POIs and categorize them. In this section, we will discuss what defines a POI, how to store it in a DB, and how to retrieve the data.

[0358] In image processing, there are many ways to save POIs. The most naive approach is to save the position (u, v) and color (R, G, B) of the pixel for each POI. This is a very poor description of the POI, and it is difficult to distinguish POIs with this method. A better solution is to save some neighborhood around the POI (e.g. a 10x10 patch around the point). This provides more information, but is very expensive (e.g. for a 10x10 neighborhood, we need to store 10*10*3=300 values ​​for each POI) and still gives bad results when trying to compare and match between POIs.

[0359] Therefore, it is common nowadays to use "feature descriptors" as representations of POIs. Feature descriptors encode the information of interest into a series of numbers and act as a kind of digital "fingerprint" that can be used to distinguish one feature from another. Ideally, this information is invariant under image transformations, so even if the image is transformed in some way, we can find the features again. There are many commonly used functions, each with their own pros and cons, to name a few: HOG, SURF, SIFT, Harris, BRISK, etc.

[0360] One major drawback of all the above descriptors is the inability to include 3D information in the descriptors. All descriptors are based only on images, so they do not contain any information about the depth or 3D structure of the POI.

[0361] Our descriptor can include a variant of a well-known image descriptor, as well as some descriptors of 3D information from our LiDAR measurements. By using such descriptors, we hope to detect and match POIs from close viewpoints more accurately than using pure image descriptors. We believe that incorporating 3D knowledge of the 3D structure around the POI will help detect and match POIs under certain scene transformations.

[0362] For example, two patches in an image may look very similar and have very similar descriptors (e.g. SIFT), but their 3D structures may be very different. Fig. 9 In , the two patches 85 and 86 share very similar color values, which may cause their image descriptors to be close. However, on closer inspection, we notice that the 3D structure of each patch is very different: the right patch 86 is a single plane, while the left patch 85 is only half a plane, with the other half at infinity. An approach that uses descriptors based solely on the image may fail and think that the two patches are a match. But descriptors that use the 3D structure of the scene will easily distinguish between the two. In this case, the range of different points within each patch is calculated and provides the above information.

[0363] Map Building

[0364] A major concern of the localization process is the construction of the initial map. In this chapter, we introduced several methods for performing the initialization process, and in the next chapter, we will discuss how to update the map after it has been initialized.

[0365] Use Mapping Vehicles

[0366] One way to build a map is to use a special mapping vehicle that will be equipped with a high-end, accurate and fast GPS+IMU system. Such a system is very expensive and cannot be part of every autonomous vehicle's sensor.

[0367] Such a vehicle would have to drive through all the streets of the city, perhaps even several times, and record in each frame what it sees.

[0368] During driving, the process of detecting POIs will pinpoint interesting and detectable points in the image and use LiDAR to accurately calculate the position of these points in the GPS coordinate system. Each POI will be stored in the DB with its corresponding descriptor (see the chapter on map structure) and its precise location.

[0369] Using the previous model

[0370] This approach is to initialize the DDB offline using a given city map and 3D known models of some static objects on the map. For example, we can use the city municipal plan to accurately locate the location of each manhole cover, fire hydrant, telephone pole, traffic sign, and traffic light on the map. We can also obtain the 3D structure of each of the above with high accuracy. We can use these inputs and generate (manually or automatically) the location of POIs. Please note that these POIs are "Type 1" POIs and can be reliable for the positioning process. Therefore, at the end of the initialization process, we have a complete city map with many reliable POIs and their precise locations in GPS world coordinates that can be used for the positioning process.

[0371] Map Updates

[0372] Once the map is initialized, the autonomous car can drive the area and use the POIs saved in the DB to localize itself. However, as mentioned before, the DB is dynamic and POIs are always coming in and out of the DB depending on their relevance.

[0373] Once a POI is not relevant, it can be removed from the DB. For example, a POI on a parked car will be removed from the DB once the car moves. Another example is a POI generated by a shadow of a building on the road, which will be automatically removed from the database after a known period of time. Another example is a static point on a building that is hidden by a car parked in front of it. The process of removing a point from the DB should be supported by reports from multiple cars in order to eliminate noise from random measurements or erroneous reports.

[0374] If a POI is relevant to other cars, it can also be added to the DB. For example, a car has just parked on the side of the road and a "type 2" POI can be used by other cars for localization. A new traffic sign is put up and from now on all cars can use it for localization. The process of adding a POI to the DB also requires support from multiple cars, because we don't want to add unreliable temporary POIs to the DB. Only after a POI is verified by reports from multiple cars will it be added to the database.

[0375] Additionally, the locations of POIs in our DB can be refined using measurements from autonomous cars. If a car finds a match to a given POI, we can use the car's measurements to update the location of the POI in our database. The update process must account for possible errors in the new measurements and update the POI with appropriate weights.

[0376] The above ideas are summarized in a possible workflow of the following algorithm:

[0377] The DB sends the predicted POI (type 1, 2, 3) locations and descriptors to the autonomous car.

[0378] The autonomous car tries to find a match with a given POI

[0379] a. Type 1 POI:

[0380] i. If a match is found - the measurement will be reported to the DB.

[0381] ii. If no match is found - the POI will be reported to the DB "Hidden".

[0382] iii. If a new point suspected to be type 1 is found - send to DB as "suspicious".

[0383] b. Type 2 POI:

[0384] i. If a match is found - the measurement will be reported to the DB.

[0385] ii. If no match is found - the POI is reported to the DB as "not relevant"

[0386] iii. If a new point suspected to be type 2 is found - send to DB as "suspicious".

[0387] c. Type 3 POI:

[0388] i. If a match is found - the measurement is reported to the database.

[0389] ii. If no match is found - the POI is reported to the DB as "Not Relevant".

[0390] iii. If a new point suspected of type 3 is found - send it to DB as "suspicious"

[0391] DB gets the report from Autonomous Cars:

[0392] a. Type 1 POI:

[0393] i. If measurements are reported - used to update the location of the POI in the DB.

[0394] ii. If reported as "Hidden" - reduce POI relevance by 1.

[0395] iii. If a new "suspicious" point is obtained - increase the POI relevance by 1 and add it to the "need verification list"

[0396] b. Type 2 POI:

[0397] i. If measurements are reported - used to update the location of the POI in the DB.

[0398] ii. If reported as "Not Relevant" - reduce POI relevance by 2.

[0399] iii. If a new "suspicious" point is obtained - increase the POI relevance by 1 and add it to the "verification required list".

[0400] c. Type 3 POI:

[0401] i. If measurements are reported - used to update the location of the POI in the DB.

[0402] ii. If reported as "Not Relevant" - reduce POI relevance by 3.

[0403] iii. If there are new "suspicious" points, add them one by one to the "verification required list".

[0404] When the relevance value > R (R-parameter), the interest point is added to the DB.

[0405] Once the relevance value of POIs == 0, they are removed from the DB.

[0406] (Optional) Send a request to the car to measure POIs in the "verification required list".

[0407] Positioning process

[0408] The localization of the car is the process of finding the transformation of the camera in the current frame relative to some origin. If we consider the "car's coordinate system", then the localization is relative to the starting position of the drive, and we only need local radar measurements of our POIs. If we consider the "world coordinate system", we also need the global GPS position of each POI.

[0409] Positioning (car coordinate system)

[0410] In order to find the car's position on the map while driving, the system will apply SLAM (self-localization and mapping).

[0411] Algorithms for SLAM are well known. Typically, they predict the car's position at time t using the predicted position at time t-1 and new sensor data sensed at time t-1. Sensor data may include, for example, camera frames, radar point clouds, and GPS measurements. The prediction of the car's position at time t may be a weighted average of the predicted position at time t and a prediction made from the sensor data sensed at time t-1. A well-known implementation of SLAM is based on an extended Kalman filter.

[0412] Fig.10 Shown are the measurement 91 in (t-1), the measurement in (t) 92, the mapped projections 93 and 94, and the matches found between the matches.

[0413] An important distinction is that we in DDB may have objects that are not strictly needed. They will be there because they help the car find its location on the map. So, for example, a unique and easy to find telephone pole could be added to the map just for this purpose.

[0414] Positioning (world coordinate system)

[0415] Given an image of a scene, the process of finding the precise position of the camera is called localization. We recommend the following process:

[0416] Given our map of POIs and the inaccurate position of the car (e.g., a position estimated using GPS or a previous position), we can estimate the approximate location of each POI in the image.

[0417] Find the POI in the image in a window around the point predicted location. If the POI is not occluded or hidden, it should be visible and can be identified.

[0418] A precise radar is pointed and photographed in the direction of the POI to determine its exact 3D position.

[0419] We repeat this process for every visible POI found in the image.

[0420] Next, we use a smart elimination algorithm (described below) to select a subset of POIs for the localization process.

[0421] We use reliable POIs to calculate the exact position of the car.

[0422] Update POI reliability in the map for the next iteration.

[0423] The goal of the elimination algorithm is to select only the most reliable POIs in each scene and base our localization on only these points. This will make the localization process much more robust to outliers and false detections:

[0424] The main insight relies on the fact that the true 3D distance between POIs is constant and does not change with time or location.

[0425] As mentioned above, in each frame where we are trying to find the position of the camera, we will try to find POIs in the image whose 3D coordinates we know. The fact that the pairwise distance between each pair of points is known a priori can be used to eliminate false matches.

[0426] Furthermore, we can only use the most reliable POIs, meaning they have the closest distances to other points that we know in advance.

[0427] Furthermore, we can leverage the fact that POIs are static and should be present in every scenario to update the map when a POI is missing. For example, suppose we have a POI on a traffic sign that we always use to find our geolocation. The next day, the traffic sign is removed. So now, the pairwise distances between this POI and the other POIs will be different (other distances remain the same). In this case, we can conclude that this POI is no longer valid and that we need to update the "static POI map".

[0428] Fig.11 A computerized system 100 is shown that includes a database 150 (POI information 151 , dynamic database 152 , and additional information 153 ), a communication module 130 , and one or more processors / computers / servers 140 .

[0429] Fig.12 A method 1200 for operating a vehicle is shown.

[0430] Method 1200 may include steps 1210 , 1220 , 1230 , and 1240 .

[0431] Step 1210 may include sensing an environment of the vehicle, which may include dynamic objects, via at least one sensor of the vehicle.

[0432] The environment may be captured within one or more fields of view of one or more sensors of the vehicle. A sensor may capture only a portion of the environment - for example, information related to a point of interest that forms only a portion of the entire field of view may be sensed.

[0433] The sensor can sense any type of radiation (visible light, infrared, near infrared, radio frequency, natural sound, etc.), and can be a passive sensor, an active sensor, or any type of sensor, such as but not limited to radar, a lidar system, a camera, etc.

[0434] The vehicle may include any number (or multiple) of any type of sensor. The vehicle may include a single type of sensor or multiple types of sensors.

[0435] Step 1210 may include sensing signals associated with one or more sets of points of interest in the environment. This may include illuminating the points of interest (when using active sensors) - or not illuminating the points of interest (when using passive sensors).

[0436] Step 1210 may include (or may be preceded by) retrieving information about the locations of the points of interest in the group. The information may be stored in a computerized system external to the vehicle.

[0437] Step 1210 may be followed by an estimation of the quality of the points of interest of the group. The estimation may include an assessment of whether elements present in the points of interest can be detected, whether the sensor receives any reflected or scattered signals due to illuminating or sensing the points of interest, whether the information obtained from the points of interest can be used for navigation, including whether the elements of the points of interest are unique and / or distinguishable from their environment, whether the signals received from the points of interest have a sufficient signal-to-noise ratio or sufficient strength, the degree to which the signals sensed from the same point of interest in multiple sensors are consistent to support the persistence of the point of interest over multiple frames or multiple illuminations, etc.

[0438] The algorithm for calculating the quality of interest points is:

[0439] 1. Select a set of features of the point of interest. The features can be the contrast of the feature with its surroundings, or the similarity between the detection signal from the point of interest and the information retrieved about the point of interest. These features may depend on the sensor assigned to sense the point of interest.

[0440] 2. Calculate the normalized score for each feature with respect to the interest point.

[0441] 3. Calculate the quality of the POI according to the weighted sum of the normalized scores of the POI features.

[0442] There may be different types of points of interest. For example, at least one point of interest may be relevant within a recurring (or non-recurring) time window, but may be irrelevant outside one or more recurring (or non-recurring) time windows.

[0443] The determination of points of interest for one type of sensor can be based on at least one of: (a) information about points of interest stored in a computerized system, and (b) information obtained by at least one other type of vehicle sensor. For example, one or more images acquired by a camera of the vehicle can be processed to find suggested points of interest for a radar and / or lidar system.

[0444] The method may include at least one of the following:

[0445] a. Proposing suggested points of interest found by processing one or more images (selecting all or at least most of the points of interest based on the processing of the images).

[0446] b. Propose points of interest stored in the computerized system (select all points of interest or at least most of the points of interest based on the points of interest stored in the computerized system).

[0447] c. Selecting at least some points of interest based on their quality.

[0448] d. Apply any selection criteria, such as mass, spatial and, or time interval, detection of multiple sensor types.

[0449] The method 1200 may also include selecting which type of vehicle sensor to use to sense the vehicle's environment based on at least one of an actual state or an estimated state of the environment.

[0450] For example, poor visibility may require the use of radar.

[0451] Step 1220 may include estimating an estimated impact of the dynamic object on the future travel of the vehicle.

[0452] A dynamic object (particularly the estimated advance of a dynamic object) may not affect the future progress of the vehicle, may endanger the vehicle, may endanger itself, may cause an accident unless the vehicle changes its future progress, may require the vehicle to navigate around the dynamic object, or may affect the future progress of the vehicle in any other way.

[0453] The estimation of the estimated impact may include estimating the future behavior of the dynamic object. The estimation is responsive to information stored in a dynamic database. The vehicle receives the information before making the estimation. The vehicle may monitor the actual behavior of the dynamic object and may update the dynamic database accordingly.

[0454] The information provided by the dynamic database can be obtained at different locations. Information related to the expected behavior of the dynamic object related to a specific location can be generated based on the behavior of other dynamic objects at the specific location, or at similar locations, or at a prototype location, or only from the specific location, wherein the information about the behavior of other dynamic objects is stored in the dynamic database and then distributed to the vehicle.

[0455] Step 1230 , performing a driving-related operation of the vehicle based on the estimated impact.

[0456] Step 1230 may include at least one of the following:

[0457] a. Autonomously driving the vehicle. This may involve maintaining the original driving mode or changing the original driving mode.

[0458] b. Changing the operating mode of the vehicle between autonomous driving mode and non-autonomous driving mode.

[0459] c. Generate a driver-perceivable alert.

[0460] d. Reduce vehicle speed - stop the vehicle or reduce speed without stopping.

[0461] e. Change the future movement of the vehicle to obtain better sensing of dynamic objects

[0462] Regarding option (e) -

[0463] a. Changes in the vehicle's future travel may include moving from one lane to another and / or moving within the same lane. The movement may be determined by a processor installed in the vehicle or by a computerized system installed external to the vehicle.

[0464] b. Changes in the vehicle's future travel can be determined based on previously acquired information about dynamic objects.

[0465] c. For example, storing images from certain angles, -- and losing images from other angles, -- and sensing dynamic objects from these dynamic objects.

[0466] d. Changes in the future course of the vehicle may be determined based on ambiguities associated with dynamic objects, which may be resolved if the dynamic object is sensed from another direction.

[0467] e. The memory unit may store the required sensing information for each dynamic object and / or each type of dynamic object (vehicle, person, ..) - and if some of said required sensing information is missing - future travel changes may take it into account.

[0468] f. Changes to the future travel of the vehicle may be made in order to sense hidden objects (dynamic or static) that are currently fully or partially hidden from the vehicle's sensors. The presence of a hidden object may be detected by the vehicle's sensors when the object is only partially obscured. The presence of a hidden object may be provided to the vehicle by other vehicles and / or may be indicated by a dynamic database (in the case of a dynamic object) or any other database available to a computerized system mounted within the vehicle or external to the vehicle.

[0469] A computerized system located in the vehicle and / or a computerized system located outside the vehicle can request different vehicles to sense dynamic and / or static objects from different directions—for example, vehicles traveling in different lanes can be requested to sense dynamic and / or static objects. Acquiring images from different directions can help identify objects, provide better analysis of objects, or for any other purpose.

[0470] For example, refer to Fig. 22 , vehicle 1602 may acquire an image of vehicle 1603 (or otherwise sense vehicle 1603). Vehicle 1602 may have difficulty seeing (or may not see) vehicle 1604. Vehicle 1602 or a computerized system located outside the vehicle (e.g., a computerized system that manages a dynamic database) may request other vehicles (e.g., those traveling in different lanes, traveling in different directions, or even traveling in the same lane but in front of or behind vehicle 1602) to sense vehicle 1604. Such a request may be generated with respect to any dynamic or static object.

[0471] Vehicle 1602 or a computerized system located external to the vehicle (e.g., a computerized system that manages a dynamic database) can request other vehicles to maintain their progress or change their progress (e.g., change lane, change position within a lane) in order to image vehicle 1604.

[0472] Step 1240 may include reporting to a computerized system.

[0473] This reporting may be performed each time step 1220 is performed—but this is not required.

[0474] For example, the method may include determining whether to report at least one change.

[0475] This determination can be based on one or more parameters, such as the status of one or more communication links between the vehicle and the computerized system, traffic density (more vehicles may mean that the network and / or computerized system is busy), whether there is a difference between the expected behavior and the sensed behavior of the dynamic object, the amount of the difference, the load imposed on the vehicle communication module, etc.

[0476] The decision whether to report at least one change may depend on the calculated uncertainty of the detection. At time t, the uncertainty may be too high to report the change, while at time t+1, the calculated uncertainty is below a certain uncertainty threshold and the change may be reported.

[0477] The report may include reporting information related to at least one of: dynamic objects, static objects, absence of movable objects on roads included in the environment, estimated impact of dynamic objects on future travel of the vehicle, risk maps, points of interest, etc.

[0478] The method may include receiving or generating a risk map and updating the risk map with the estimated impact of the dynamic objects on the future travel of the vehicle. The risk map may provide risk-related parameters (eg, risk levels) for all or some locations on the vehicle's path.

[0479] Step 1240 may include transmitting information about the vehicle's environment (information sensed by one or more sensors of the vehicle) to a computerized system located outside the vehicle.

[0480] The method 1200 may include generating information about the environment by applying a privacy protection measure.

[0481] The application of privacy protection measures may include masking at least one of vehicle identification information and person identification information. For example, image processing may be applied to images acquired by a vehicle camera and faces may be recognized (facial recognition) or at least detected and masked in the transmitted image. The same applies to the vehicle's license plate number or any identification information.

[0482] Fig.13 Method 1300 is shown.

[0483] Method 1300 may include steps 1310 , 1320 , and 1330 .

[0484] Step 1310 may include repeating the following steps for each of the plurality of locations:

[0485] a. Step 1320 , when the vehicle is located at the position, selecting which type of vehicle sensor is used to sense the environment of the vehicle from a plurality of types of vehicle sensors.

[0486] b. Step 1330, when the vehicle is at the location, using at least one selected type of vehicle sensor to sense the vehicle's environment; wherein the using may include sensing signals associated with a plurality of points of interest in the environment.

[0487] The plurality of locations may form the entire path of the vehicle, or may form only some of the paths. The repetition may be repeated at each time period when the vehicle passes a certain distance, at each intersection, every day, every hour, whenever visibility conditions change, and the like.

[0488] Various types of vehicle sensors may include cameras, light detection and ranging (lidar) sensors, and radio frequency radar.

[0489] The method may include estimating a quality of at least some of the plurality of points of interest.

[0490] Fig.14 Method 1400 is shown.

[0491] Method 1400 may include steps 1410 , 1420 , and 1430 .

[0492] Method 1400 is used to maintain a dynamic database. Maintenance may include establishing a dynamic database, and even updating the dynamic database. The dynamic database may be the DDB mentioned above.

[0493] Fig.14 May be performed by a computerized system located external to the vehicle.

[0494] Step 1410 may include receiving information from a first plurality of vehicles regarding different locations of the plurality of vehicles.

[0495] The vehicle can acquire information about the environment (raw data), can process it in various ways (e.g., noise reduction processing, feature extraction, event detection), and can send the raw data or processed data to the computerized system. The processed data (e.g., event information for identifying an event) may be smaller than the raw data.

[0496] Thus, step 1410 may include receiving raw data sensed by a vehicle sensor of at least one vehicle of the first plurality of vehicles, and receiving event information from at least one other vehicle of the first plurality of vehicles.

[0497] Step 1410 may include receiving information from the vehicle regarding a mismatch between information sensed by the vehicle regarding a location in a different location and information about the location of the dynamic database.

[0498] Step 1410 may include receiving information from a vehicle regarding a mismatch between actual behavior of dynamic objects within a location and statistical data included in a dynamic database regarding dynamic objects within the location.

[0499] Step 1420 may include maintaining a dynamic database, wherein the dynamic database includes statistical data related to the behavior of dynamic objects within different locations.

[0500] The first plurality of vehicles and the second plurality of vehicles may include the same vehicles, or may be different vehicles from one another. The number of vehicles in the first plurality of vehicles and the second plurality of vehicles may be any number - but it is beneficial to obtain information from as many vehicles as possible.

[0501] Step 1420 may include at least one of the following:

[0502] a. Classify different positions into prototype positions and maintain statistics by prototype position.

[0503] b. Maintain statistics by locations that are similar to each other.

[0504] c. Categorize locations and maintain statistics by category,

[0505] Wherein the classification of at least one category is responsive to an amount of data obtained regarding one or more locations belonging to the category.

[0506] d. Requesting the vehicle to provide information about a new location that is not included in the plurality of locations.

[0507] e. Requesting the vehicle to provide information about the quality of one or more points of interest illuminated by the vehicle.

[0508] f. Requesting the vehicle to provide information about the behavior of one or more dynamic objects within one or more locations within one or more future time windows.

[0509] g. Requesting the vehicle to change the location of the vehicle to a specific location, and requesting the vehicle to obtain information from the specific location.

[0510] h. Request the vehicle to provide information about points of interest.

[0511] Any responses to these requests may be received during step 1410 and then processed during step 1420 .

[0512] Step 1420 may include maintaining point of interest group information about the point of interest groups in a dynamic database, wherein each group is associated with a location in the different locations.

[0513] Two or more groups may be associated with the same location and two or more types of vehicle sensors.

[0514] The point of interest information may include location information regarding the absolute location of the point of interest.

[0515] The points of interest may include static points of interest on static objects and at least one of: (a) static points of interest on dynamic objects, and (b) dynamic points of interest on static objects.

[0516] The point of interest information may include two-dimensional position information and distance information.

[0517] Each interest point may represent a segment of the object, and wherein the distance information represents distances to portions of the segment.

[0518] Step 1430 may include distributing the relevant portions of the dynamic database to a second plurality of vehicles.

[0519] Multiple iterations of steps 1410, 1420, and 1430 may be performed. For example, the dynamic database may be updated with information provided during step 1410. One or more vehicles and the computerized system maintaining the dynamic database may exchange information, requests, and queries.

[0520] Step 1430 may include determining a relevant portion of the dynamic database to be transmitted to a vehicle in the second plurality of vehicles based on the location of the vehicle and a cost associated with transmission to the vehicle.

[0521] Fig.15 Method 1500 is shown.

[0522] Method 1500 may include steps 1510 , 1520 , and 1530 .

[0523] Method 1500 is used to facilitate updating a dynamic database.

[0524] Step 1510 may include receiving a portion of a dynamic database by the vehicle. The dynamic database may be any of the dynamic databases described above. It may include statistical data related to the behavior of dynamic objects in different locations.

[0525] Step 1520 may include searching for mismatches between the contents of the dynamic database portion and the sensed information sensed by the vehicle.

[0526] Step 1530 may include reporting the mismatch to a computerized entity involved in maintaining the dynamic database.

[0527] The dynamic database may include information about groups of points of interest, wherein each group is associated with a location among the different locations.

[0528] The method 1500 may also include sending information regarding the quality of the one or more points of interest to a database.

[0529] Communication between autonomous car and driver

[0530] In the next few years, there will be autonomous cars on the road, driving side by side with real drivers. To do this safely and efficiently, both parties will need to understand the other's predicted behavior, so the driver will need to understand the predicted behavior of the autonomous car and drive accordingly, and vice versa. Both parties will also need to signal to each other just as they do with real drivers today.

[0531] Before communication, the driver's and the driver's intentions can be understood and displayed.

[0532] This may include detecting hazardous drivers (distracted - e.g. typing an SMS message, drunk and / or fatigued).

[0533] There are various in-vehicle systems that can monitor the driver of a vehicle and generate warnings, including warning that the driver is drowsy, intoxicated, distracted, etc.

[0534] Someone in another vehicle may be able to track the car's movements and at least suspect that the driver is behaving abnormally. The tracker may not know the exact reason (drunk, on drugs, tired, SMS), but it can spot these patterns and analyze them. This is very important for normal drivers and self-driving cars, which usually give more space to these cars. In car-to-car communications, this information should be conveyed. The standard signal of a dangerous driver is a direction correction for a violent deviation within the lane.

[0535] A system and method for detecting a dangerous driver of another vehicle is provided. Specifically, the system and method can do so even without the assistance of the driver of the other vehicle.

[0536] Thus, the method may include:

[0537] a. Detect car type and brand. This can be done using image processing of the image of the car. The processing can include, for example, comparing it to a database of cars.

[0538] b. Accurately measure the movement of the car. The monitoring period can last for a few seconds (for example, between 1 and 20 seconds). If the monitored person's monitoring car can move in the same direction, it may be easier to maintain sufficient information.

[0539] c. Calculate the driver's steering wheel manipulation based on the car model and car motion. Assuming we observe the car motion in detail, this method can estimate the steering wheel motion of the manipulation performed by the driver based on the observed car motion.

[0540] d. Apply any algorithm to detect driver drowsiness and other driving risks based on control of the steering wheel. This may include using new algorithms - or algorithms that have been developed to monitor the vehicle itself - instead of monitoring another vehicle. The method can use existing, tested algorithms that are designed to understand the driver's condition from inside the car and from outside the car. For example, an algorithm may include analyzing the lateral change of another vehicle and comparing it to a lateral change threshold. Such a threshold may be constant or may be retrieved from a DDB. The DDB may store information such as the expected lateral change at one location, which may be different from the expected lateral change at another location. Another algorithm may analyze the change in speed of the other vehicle.

[0541] This technology could be useful for both self-driving cars and regular cars.

[0542] Knowing that the other driver is drowsy or intoxicated can be very useful.

[0543] Judging the driver's intentions from the way the car moves

[0544] When we drive, we are always assessing the intentions of the cars around us and displaying our intentions by the way we drive. For example, when we attempt to enter a busy street, we announce our intentions by using the relevant flashing lights but also driving slowly toward the street. Others, by the way they drive, tell us whether they are willing to let us in. We can force entry, in which case they will usually back off.

[0545] When driving between different cars, there is a dialogue, with the car expressing the driver's intentions. When a human driver tries to change lanes, he may express your intentions, and the car may slow down to let you go, in which case you accept the offer. If you want to wait until there is a break, in some places you will never get it. Some drivers show aggressive intentions, and some drivers do not use signals.

[0546] As a pedestrian trying to cross the road, we announce our intention, and if possible, we try to make eye contact with the driver to see if he acknowledges our presence. If we get it, we can start walking safely, if not, we can wait or try another approach. If the car is in front of the intersection, slowing down may mean turning. A lot of information about intent is passed and received between drivers and is critical to the way human-driven cars interact.

[0547] Obviously it depends on the car signals. See http: / / www.dailvmail.co.uk / sciencetech / article-3539399 / Google-s-self-driving-cars-soon-predict-drivers-going-Patent-reveals-plans-sensors-detect-turn-signals-braking-lights.html.

[0548] The proposed method can sense additional information related to the way a person drives or observes. This is especially important when one is trying to slowly enter traffic. Wait until it is safe and there are no cars in the area to drive very slowly. Often cars will indicate (the driver will) that they are letting you in by slowing down, and if you take advantage of this, they will slow down more if necessary. A person will enter traffic, observe if the car behaves as expected (expecting it to slow down enough to let him enter easily), and if it does not, he will not enter traffic at all.

[0549] In many cases this will be part of a car-to-car conversation (one trying to enter traffic and the other deciding whether to let it in). So the distinction might be for the car trying to enter (the other car will let me in) or for the moving car (this guy really wants to get in). The main way to calculate the driver's intention is by sensing and analyzing visual cues and comparing the situations.

[0550] For example, someone who intends to continue driving will usually drive in the middle of the lane at a fairly constant speed. Someone who is about to turn right (in addition to or independently of using the turn signal) will be on the right side of the lane. Possibly at the lane boundary. Of course, he may be a bad driver, or drunk, or texting, but he may also be trying to overtake. The vehicle intending to turn will slow down. The vehicle trying to intrude into traffic, e.g. from the lane will step forward. Create a small barrier with his car, indicating that he wants to enter. He will also look intently at the car that may respond. Once a signal is received (e.g. the car slows down because of him), we see his attention shift to the road.

[0551] In general, we are looking for deviation from normal driving, and if we see it, it may indicate a problem, but it may also indicate intent. The indication of intent itself is done using the norms for driving in a particular location, and may be different between countries, and even between regions within countries. We are not talking about the usual signaling methods used in driving tests, such as turn signals, but rather communication signals between drivers. In addition to the above, they may also include flashing lights (I want to go, or in some other places, you can go), as well as nodding, smiling, all of which are used by people to detect intent.

[0552] Indicating intentions to self-driving cars

[0553] Even a self-driving car might be required to move in a human-like way to signal its intentions — so that other cars and pedestrians can understand it. For example, it might slow down, swerve, or do something else that other drivers understand.

[0554] Additionally or alternatively, the self-driving car can be equipped with a head impression so that the passenger will know that he is seen and is being watched.

[0555] An interesting analogy is silent cars (electric). There are laws that say they need to make noise because that's what people expect from cars. Otherwise it's unsafe (people enter the road looking because they're used to hearing cars). The same goes for driving style, a person can understand intent just as well as anyone else, so even if you can drive at high speeds (higher than the average driver) and it's legal, you should slow down to signal your turn.

[0556] If you see a car trying to pull into traffic, you need to decide how polite you are. If you are polite, you can let it pull into traffic, and you need to show it by the way you drive. The same is true for self-driving cars.

[0557] Instructions for requesting to receive and grant right of way:

[0558] When an autonomous car sees someone trying to squeeze into traffic and offers to let him in, it can display this, just like a person would. Slowing down, flashing lights, and other indications relevant to the driving position. For the autonomous car, another car's intention to merge into traffic is translated into a detected change in the behavior of the object, or an unexpected change in the driving free space. For example, the merging car has a higher lateral velocity vector than the longitudinally driven car. Or the driving free space may be convex where it would have been expected to be straight. These changes in dynamic and static object behavior need to be understood together and interpreted as "the car tried to merge into traffic."

[0559] When an autonomous car wants to cut into traffic (e.g. from a side road to a main busy road), it needs to indicate that it wants to cut into traffic, and ask for permission. This will be done the same as a person in that position. When the action is reversible, slowly move forward into traffic. If another car decides not to give way, don't force it.

[0560] Turning

[0561] People signal and change speed to indicate they are about to turn. Since a self-driving car will not forget to use its turn signal, it may decide not to change speed like a human driver would when turning. However, there is an advantage to doing so because it is visible and understood by people who do not look at their turn signals.

[0562] Understanding steering wheel usage from car behavior

[0563] Looking at a moving car, and knowing the road and the car, we can reverse engineer the steering wheel motion as the driver does. Now any application that considers the motion of the steering wheel (like drowsiness and fatigue detection http: / / pid.sagepub.eom / content / 229 / 2 / 163.abstract) and many other things can be done based on what is happening outside of the car. The advantage is that we can leverage legacy applications in the car to detect certain information about the driver and use the same algorithms, but we can get the steering wheel motion from the patented parts.

[0564] A method may be provided, which may include:

[0565] a. Collect various algorithms to detect driver status from steering wheel movement.

[0566] b. Use sensors to observe the car's movement.

[0567] c. Using your observations and the model of your car, calculate the movement of the steering wheel.

[0568] d. Apply one or more algorithms to detect driver condition from the calculated steering wheel motion.

[0569] e. Influence driving decisions based on our assessment of the vehicle's driver condition.

[0570] Change your driving style to suit local customs

[0571] It has been suggested that autonomous vehicles will change the way they drive to indicate their intentions (regarding driving) and could provide a way to understand other drivers' intentions from the way they drive. They argue that autonomous vehicles should receive and pass on intention information based on driving patterns, just like normal drivers do. This has multiple advantages, as discussed above.

[0572] However, the language that cars use to communicate with each other is not universal and may depend on the location (drivers in different places may have different driving habits). So, just as self-driving cars need to follow different rules in different places, they need to understand the local customs of this communication method in order to communicate effectively. For example, Seattle and Mumbai may use different modes.

[0573] Coercing self-driving cars

[0574] Autonomous vehicles are expected to be intimidated by human drivers when two cars are sharing road space. While autonomous vehicles will be programmed to be law-abiding and considerate road users, their human counterparts will be more aggressive.

[0575] Part of this expectation is that self-driving cars will be different and will not use the car-to-car communications described in this application to coordinate with other vehicles. If they did, they would likely be less coerced.

[0576] To reduce the chances of being duress, autonomous vehicles could indicate (even to human drivers) their future predictions by changing their driving (speed, position within the lane) to provide more instructions about changing lanes and / or staying in the same lane.

[0577] Fig.16 Vehicle 1605 is shown proceeding in a problematic manner (see pattern 1605') as it repeatedly changes its position within lane 1611 without any reason.

[0578] The movement of vehicle 1605 is monitored by vehicle 1601. Vehicle 1601 may notice the problematic progress of vehicle 1605, may detect that the driver is intoxicated, fatigued, and / or distracted, and may change its progress and / or may alert at least one of: the driver of vehicle 1605, vehicle 1605, the driver of vehicle 1602 (traveling in lane 1612), vehicle 1602, the driver of vehicle 1603, vehicle 1603, the driver of vehicle 1604, vehicle 1604, pedestrian 1621, and pedestrian 1622.

[0579] The alert may be sent using any type of communication (vehicle-to-vehicle, human-perceptible, etc.).

[0580] A request may be made to the driver of vehicle 1605 and / or vehicle 1605 to change the driving mode from non-autonomous to autonomous.

[0581] The request may be sent by at least one of: the driver of vehicle 1601 , vehicle 1601 , the driver of vehicle 1602 , vehicle 1602 , the driver of vehicle 1603 , vehicle 1603 , the driver of vehicle 1604 , vehicle 1604 , pedestrian 1621 , and pedestrian 1622 .

[0582] Fig.17 A vehicle 1605 is shown approaching a turn 1613, and an estimate needs to be made as to whether the vehicle 1605 intends to turn. The vehicle 1605 may be operating in an autonomous mode (when approaching the turn 1613), and may change its behavior to indicate to a human driver that it intends to turn. The change may include signaling and performing at least one of the following operations: slowing down, transmitting a turn warning (without using lights), approaching the right edge of the lane 1611, etc.

[0583] Fig.18 A method 1800 for monitoring a vehicle and operating another vehicle is shown.

[0584] Method 1800 may include steps 1810 , 1820 , 1830 , 1840 , 1850 , and 1860 .

[0585] Step 1810 may include monitoring the movement of the vehicle by at least one sensor of another vehicle.

[0586] Step 1820 may include estimating, by a computer of the other vehicle, an estimated interaction between the driver and the vehicle based on the motion of the first vehicle and the model of the vehicle.

[0587] Step 1820 may include estimating the interaction between the driver and the vehicle's steering wheel. This estimation may be based on analyzing the behavior of dynamic (another vehicle) and static objects (driving free space) together and checking whether they match known driving patterns (drunk driving, slowly pulling into traffic). One algorithm for determining the behavior of dynamic and static objects and the relationship between behavior patterns is to use a deep neural network to learn this relationship.

[0588] Step 1830 may include determining a state of the driver based on the estimated interaction.

[0589] Step 1830 may include at least one of the following:

[0590] a. Apply the drowsiness detection process.

[0591] b. Apply the fatigue detection process.

[0592] c. Apply a drunk driving detection program.

[0593] Step 1840 may include estimating the estimated impact of the vehicle on the future travel of another vehicle.

[0594] Step 1850 may include performing a driving-related operation on another vehicle based on the estimated impact.

[0595] Step 1860 may include at least one of the following:

[0596] a. Autonomously generate alerts that are perceptible to the vehicle driver.

[0597] b. Autonomously generate alerts that can be perceived by one or more other vehicles.

[0598] c. A request by another vehicle for the vehicle to change the operation mode of the other vehicle from a non-autonomous driving mode to an autonomous driving mode.

[0599] Fig.19 A method 1900 for monitoring a vehicle and operating another vehicle is shown.

[0600] Method 1900 may include steps 1910 , 1920 , 1930 , and 1940 .

[0601] Step 1910 may include monitoring the dynamic behavior of the first vehicle by at least one sensor of another vehicle.

[0602] Step 1920 may include sensing, by the computer, a dynamic behavior of the first vehicle.

[0603] Step 1930 may include estimating the state of the first vehicle. This estimation may be based on analyzing the behavior of dynamic objects (another vehicle) and static objects (driving free space) together and checking whether they match known driving patterns (drunk driving, slowly pulling into traffic). One algorithm for determining the behavior of dynamic and static objects, and the relationship between the behavior patterns, is to use a deep neural network to learn this relationship.

[0604] Step 1940 may include estimating an interaction between the first vehicle and the other vehicle.

[0605] Step 1950 may include performing a driving-related operation on another vehicle based on the estimated state and the estimated interaction.

[0606] Estimating the state of the first vehicle may include estimating the state of the driver.

[0607] Fig. 20 A method 2000 for estimating future behavior of a vehicle and operating another vehicle is shown.

[0608] Method 2000 may include steps 2010 , 2020 , 2030 , and 2040 .

[0609] Step 2010 may include monitoring movement of the vehicle during a monitoring period and by at least one sensor of another vehicle.

[0610] Step 2020 may include attempting to predict the future behavior of the vehicle based on a combination of at least two of the following: (a) light signals generated by another vehicle during the monitoring period, (b) vehicle speed and acceleration during the monitoring period, and (c) the spatial relationship between the vehicle and the lane during the monitoring period, (d) the vehicle's environment during the monitoring period.

[0611] Step 2020 may include selecting a selected vehicle behavior pattern from a plurality of vehicle behavior patterns; wherein the selection is based on the monitored vehicle motion. Such selection may be based on a deep neural network for learning and classifying a plurality of vehicle behaviors.

[0612] The selecting may include finding the best matching vehicle behavior pattern.

[0613] The selecting may include a comparison between values ​​of the at least two elements and between values ​​of the at least two elements associated with a plurality of vehicle behavior patterns.

[0614] The plurality of vehicle behavior modes may include (a) remaining in the current lane, (b) leaving the current lane and entering the lane of another vehicle, (c) leaving the current lane without entering the lane of another vehicle, and (d) stopping the vehicle.

[0615] Step 2030 may include estimating an estimated impact of the vehicle on the future travel of another vehicle.

[0616] Step 2040, performing at least one of the following operations: (a) performing a driving-related operation on another vehicle based on the estimated impact, and (b) alerting at least a person, a computerized system external to the vehicle, and at least one entity in the other vehicle about the vehicle's behavior.

[0617] Fig.21 A method 2100 for operating a vehicle is shown.

[0618] Method 2100 may include steps 2110 , 2120 , and 2130 .

[0619] Step 2110 may include receiving, by the vehicle from at least one entity external to the vehicle, a request to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode.

[0620] The at least one entity external to the vehicle may be a person or a vehicle or any other computerized system, such as a control system.

[0621] Step 2120 may include determining, via the vehicle computer, whether to change the operating mode.

[0622] Step 2130 may include changing the operating mode of the vehicle from the non-autonomous driving mode to the autonomous driving mode when it is determined to change the operating mode.

[0623] The determination may be based on a number of requests to change the operating mode received during the time window.

[0624] The request may include an indication of a risk associated with continuation of the non-autonomous driving mode, and the determination may be responsive to the indication of risk.

[0625] project:

[0626] 1. A method for operating a vehicle, the method comprising:

[0627] sensing an environment of the vehicle by at least one sensor of the vehicle, the environment including a dynamic object;

[0628] estimating an estimated effect of the dynamic object on future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic object; and

[0629] A driving-related operation of the vehicle is performed based on the estimated impact.

[0630] 2. A method according to item 1, wherein the sensing occurs at a specific location, and wherein the estimated behavior of the dynamic object is based on the behavior of other dynamic objects at the specific location, wherein information about the behavior of the other dynamic objects at the specific location is stored in the dynamic database.

[0631] 3. A method according to claim 1, wherein the estimated behavior of the dynamic object is based on the behavior of other dynamic objects in an environment similar to the environment of the vehicle.

[0632] 4. The method according to claim 1, wherein the dynamic object is another vehicle.

[0633] 5. A method according to claim 4, wherein the information is about an expected driving pattern of the other vehicle.

[0634] 6. The method according to item 1, wherein the dynamic object is a person.

[0635] 7. The method of claim 1, comprising reporting sensing of the dynamic object.

[0636] 8. A method according to item 1, wherein performing driving-related operations on the vehicle includes autonomously driving the vehicle.

[0637] 9. The method according to item 1, wherein performing driving-related operations on the vehicle includes changing an operating mode of the vehicle between an autonomous driving mode and a non-autonomous driving mode.

[0638] 10. The method of claim 1, wherein performing a driving-related operation on the vehicle comprises generating a driver-perceivable alert.

[0639] 11. The method of claim 1, wherein performing the driving-related operation on the vehicle comprises reducing the speed of the vehicle.

[0640] 12. A method according to item 1, wherein performing driving-related operations on the vehicle includes changing the future travel of the vehicle in order to obtain better sensing of the dynamic object.

[0641] 13. The method of claim 1, comprising detecting a change between a sensed content of the environment and an expected content of the environment.

[0642] 14. The method of claim 13, comprising determining whether at least one of the changes is to be reported.

[0643] 15. The method of claim 13, comprising reporting the absence of movable objects on roads included in the environment.

[0644] 16. The method of claim 1, comprising reporting an estimated impact of the dynamic object on the future travel of the vehicle.

[0645] 17. A method according to claim 1, comprising receiving or generating a risk map and updating the risk map with the estimated impact of the dynamic object on the future travel of the vehicle.

[0646] 18. The method of claim 1, wherein the sensing comprises sensing, via a vehicle sensor, signals associated with a set of points of interest in the environment.

[0647] 19. A method according to item 18, wherein information about the location of the points of interest in the set of points of interest is retrieved before the sensing.

[0648] 20. The method of clause 18, wherein the sensing is followed by estimating the quality of the points of interest in the set of points of interest.

[0649] 21. A method according to item 18, wherein the quality of the points of interest in the set of points of interest is reported after the sensing.

[0650] 22. The method of clause 18, wherein the sensing comprises sensing a signal associated with at least one point of interest that is relevant within a recurring time window and not relevant outside of the recurring time window.

[0651] 23. The method of clause 18, wherein the sensing comprises sensing a signal associated with at least one point of interest that is relevant within one or more time windows and not relevant outside of the one or more time windows.

[0652] 24. A method according to claim 1, wherein the sensing includes collecting signals from multiple sets of points of interest in the environment through multiple types of vehicle sensors.

[0653] 25. A method according to item 24, wherein the multiple types of vehicle sensors include cameras, light detection and ranging sensors, and radio frequency radars.

[0654] 26. The method of claim 25, comprising determining at least one point of interest with respect to at least one type of vehicle sensor based on information obtained by at least one other type of vehicle sensor.

[0655] 27. A method according to item 1, wherein the sensing includes selecting which type of vehicle sensor to use for sensing the environment of the vehicle based on at least one of an actual state or an estimated state of the environment.

[0656] 28. The method of clause 1, comprising transmitting information about the environment to a computerized system located external to the vehicle.

[0657] 29. The method according to item 28 includes generating information about the environment by applying privacy protection measures.

[0658] 30. A method according to item 29, wherein applying privacy protection measures includes masking at least one of vehicle identification information and person identification information.

[0659] 31. The method of claim 1, comprising reporting the sensed movement of the dynamic object.

[0660] 32. A method for a vehicle, the method comprising: for each position in a plurality of positions, repeating the following steps:

[0661] selecting which type of vehicle sensor from a plurality of types of vehicle sensors to use for sensing the environment of the vehicle when the vehicle is at the location; and

[0662] When the vehicle is at the location, an environment of the vehicle is sensed using at least one selected type of vehicle sensor; wherein the using includes sensing signals associated with a plurality of points of interest in the environment.

[0663] 33. A method according to item 32, wherein the multiple types of vehicle sensors include cameras, light detection and ranging (lidar) sensors and radio frequency radars.

[0664] 34. The method of item 32, comprising estimating the quality of at least some of the plurality of points of interest.

[0665] 35. A method for maintaining a dynamic database, the method comprising:

[0666] receiving information from a first plurality of vehicles regarding different locations of the plurality of vehicles;

[0667] maintaining the dynamic database, wherein the dynamic database includes statistical data related to the behavior of dynamic objects within the different locations; and

[0668] Relevant portions of the dynamic database are distributed to a second plurality of vehicles.

[0669] 36. A method according to item 35, wherein at least some of the statistical data are time-sensitive.

[0670] 37. A method according to item 35, wherein the maintenance includes generating and updating the dynamic database.

[0671] 38. A method according to item 35, wherein the dynamic object includes a vehicle.

[0672] 39. The method according to item 35, wherein the dynamic object includes a person.

[0673] 40. According to the method of item 35, the dynamic objects include people and vehicles.

[0674] 41. A method according to item 35, wherein the statistical data indicates the temporal distribution of vehicle types in the lane.

[0675] 42. A method according to item 35, wherein the statistical data is related to the behavior of dynamic objects within one or more intersections.

[0676] 43. A method according to item 35, wherein the statistical data is related to the relationship between the status of one or more traffic lights located at one or more intersections and the behavior of dynamic objects within the one or more intersections.

[0677] 44. A method according to item 35, wherein the statistical data relates to speeds in different directions within one or more intersections.

[0678] 45. A method according to claim 35, wherein the statistical data relates to the behavior of different types of vehicles within one or more intersections.

[0679] 46. ​​A method according to item 35, wherein the statistical data is related to the behavior of different types of vehicles and one or more individuals within one or more intersections.

[0680] 47. A method according to item 35, wherein the statistical data is related to the behavior of dynamic objects selected from vehicles and people near one or more buildings.

[0681] 48. A method according to item 35, wherein the receiving of the information includes receiving raw data sensed by a vehicle sensor of at least one of the first plurality of vehicles, and receiving event information from at least one other vehicle of the first plurality of vehicles; wherein the size of the event information is smaller than the size of the raw data.

[0682] 49. The method according to item 35 includes classifying the different positions into prototype positions and maintaining statistical data by prototype position.

[0683] 50. The method of claim 35, comprising maintaining statistical data by locations that are similar to one another.

[0684] 51. The method of claim 35, comprising classifying locations into categories and maintaining statistics by category, wherein the classification of at least one category is responsive to an amount of data obtained relative to one or more locations belonging to said category.

[0685] 52. The method of claim 35, comprising maintaining statistics by location including lanes, intersections, and buildings.

[0686] 53. A method according to item 35, wherein the maintaining includes applying deep learning to determine the statistical data.

[0687] 54. The method of item 35 comprises receiving information from a vehicle regarding a mismatch between information regarding a location in said different locations sensed by said vehicle and information regarding said location in said dynamic database.

[0688] 55. The method according to item 35 includes receiving information from the vehicle about a mismatch between actual behavior of a dynamic object within a location and statistical data about the dynamic object within the location included in the dynamic database.

[0689] 56. The method of claim 35, comprising requesting the vehicle to provide information about a new location that is not included in the plurality of locations.

[0690] 57. The method of item 35, comprising requesting a vehicle to provide information about the quality of one or more points of interest illuminated by the vehicle.

[0691] 58. The method of item 35, comprising requesting the vehicle to provide information about the behavior of one or more dynamic objects within one or more locations within one or more future time windows.

[0692] 59. The method according to item 35 includes requesting a vehicle to change the position of the vehicle to a specific location, and requesting the vehicle to obtain information from the specific location.

[0693] 60. The method of item 35, comprising maintaining point of interest information about groups of points of interest in the dynamic database, wherein each group is associated with a location in the different locations.

[0694] 61. A method according to item 60, wherein two or more groups are associated with the same location and with two or more types of vehicle sensors.

[0695] 62. A method according to item 60, wherein the point of interest information includes location information about the absolute location of the point of interest.

[0696] 63. A method according to item 60, wherein the points of interest include static points of interest on static objects and at least one of the following: (a) static points of interest on dynamic objects, and (b) dynamic points of interest on static objects.

[0697] 64. A method according to item 60, wherein the point of interest information includes two-dimensional position information and distance information.

[0698] 65. A method according to item 64, wherein each point of interest represents a segment of the object, and wherein the distance information represents the distance to multiple parts of the segment.

[0699] 66. The method of item 60, comprising requesting the vehicle to provide information about a point of interest.

[0700] 67. A method according to item 35, comprising determining a relevant portion of the dynamic database to be sent to the vehicle based on the position of the vehicle in the second plurality of vehicles and the cost associated with transmission to the vehicle in the second plurality of vehicles.

[0701] 68. A method for facilitating updating of a dynamic database, the method comprising:

[0702] receiving, by a vehicle, a portion of the dynamic database; wherein the dynamic database includes statistical data related to behavior of dynamic objects within different locations;

[0703] searching for a mismatch between contents of the portion of the dynamic database and sensory information sensed by the vehicle; and

[0704] The mismatch is reported to a computerized entity involved in maintaining the dynamic database.

[0705] 69. A method according to item 68, wherein the dynamic database includes information about groups of points of interest, wherein each group is associated with a location in the different locations.

[0706] 70. The method according to item 69 comprises sending information about the quality of one or more points of interest to the database.

[0707] 71. A computer program product storing instructions which, when executed by a computerized system installed in a vehicle, cause the computerized system to:

[0708] sensing an environment of the vehicle by at least one sensor of the computerized system, the environment including dynamic objects;

[0709] estimating an estimated effect of the dynamic object on future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic object; and

[0710] A driving-related operation of the vehicle is performed based on the estimated impact.

[0711] 72. A computer program product storing instructions which, when executed by a computerized system installed in a vehicle, cause the computerized system to: for each of a plurality of positions, repeat the following steps:

[0712] selecting which type of vehicle sensor from a plurality of types of vehicle sensors to use for sensing the environment of the vehicle when the vehicle is at the location; and

[0713] sensing an environment of the vehicle using at least one selected type of vehicle sensor when the vehicle is at the location;

[0714] Wherein the using comprises sensing signals associated with a plurality of points of interest in the environment.

[0715] 73. A computer program product storing instructions which, when executed by a computerized system located external to a vehicle, cause the computerized system to:

[0716] receiving information from a first plurality of vehicles regarding different locations of the plurality of vehicles;

[0717] maintaining a dynamic database, wherein the dynamic database includes statistical data related to the behavior of dynamic objects within the different locations; and

[0718] Relevant portions of the dynamic database are distributed to a second plurality of vehicles.

[0719] 74. A computer program product storing instructions which, when executed by a computerized system located external to the vehicle, cause the computerized system to:

[0720] receiving a portion of a dynamic database; wherein the dynamic database includes statistical data related to behavior of dynamic objects within different locations;

[0721] searching for a mismatch between contents of the portion of the dynamic database and sensory information sensed by the vehicle; and

[0722] The mismatch is reported to a computerized entity involved in maintaining the dynamic database.

[0723] 75. A computerized system installed in a vehicle, wherein the computerized system comprises:

[0724] at least one sensor configured to sense an environment of the vehicle, the environment including dynamic objects; and

[0725] A processor configured to (a) estimate an estimated impact of the dynamic object on the future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about the estimated behavior of the dynamic object; and (b) perform a driving-related operation on the vehicle based on the estimated impact.

[0726] 76. A computerized system installed in a vehicle, wherein the computerized system includes a processor and multiple types of sensors;

[0727] Wherein the computerized system is configured to repeat the following steps for each of the plurality of locations:

[0728] selecting, by the processor, which type of sensor from the plurality of types of sensors to use for sensing the environment of the vehicle when the vehicle is at the location; and

[0729] sensing the environment of the vehicle using at least one selected type of sensor when the vehicle is at the location;

[0730] Wherein the using comprises sensing signals associated with a plurality of points of interest in the environment.

[0731] 77. A computerized system positioned external to a vehicle, the computerized system comprising a communication unit and a processor;

[0732] wherein the communication unit is configured to receive information about different locations of the plurality of vehicles from a first plurality of vehicles;

[0733] wherein the processor is configured to maintain a dynamic database, wherein the dynamic database includes statistical data related to the behavior of dynamic objects within the different locations; and

[0734] Wherein the communication unit is configured to distribute the relevant portion of the dynamic database to a second plurality of vehicles.

[0735] 78. A computerized system positioned external to a vehicle, the computerized system comprising a communication unit and a processor;

[0736] wherein the communication unit is configured to receive a portion of a dynamic database;

[0737] wherein the dynamic database includes statistical data related to the behavior of the dynamic objects in the different locations;

[0738] wherein the processor is configured to search for a mismatch between the contents of the portion of the dynamic database and the sensed information sensed by the vehicle; and

[0739] Wherein, the communication unit is configured to report the mismatch to a computerized entity involved in maintaining the dynamic database.

[0740] 79. A method for monitoring a vehicle and operating another vehicle, the method comprising:

[0741] monitoring movement of the vehicle via at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle;

[0742] estimating, by a computer of the other vehicle, an estimated interaction between a driver of the vehicle and the vehicle based on the motion of the first vehicle and the model of the vehicle;

[0743] determining a state of the driver based on the estimated interaction;

[0744] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0745] A driving related operation on the other vehicle is performed based on the estimated impact.

[0746] 80. A method according to item 79, wherein the estimating includes estimating the interaction between the driver and the steering wheel of the vehicle.

[0747] 81. A method according to item 79, wherein the determination includes applying at least one of the following: a drowsiness detection process, a fatigue detection process, and a drunk driving detection process.

[0748] 82. A method according to item 79, wherein the determination includes applying a drowsiness detection process, a fatigue detection process and a drunk driving detection process.

[0749] 83. The method according to item 79 also includes autonomously generating an alert that is perceptible to the driver of the vehicle.

[0750] 84. The method of claim 79, further comprising autonomously generating an alert perceptible by one or more other vehicles.

[0751] 85. The method of claim 79 further comprising notifying a computerized system external to the vehicle of the driver's status.

[0752] 86. The method according to item 79 also includes requesting the vehicle, through the other vehicle, to change the operation mode of the other vehicle from a non-autonomous driving mode to an autonomous driving mode.

[0753] 87. A method for monitoring a vehicle and operating another vehicle, the method comprising:

[0754] monitoring a dynamic behavior of a vehicle via at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle;

[0755] sensing the dynamic behavior of the vehicle by a computer;

[0756] estimating a state of the vehicle;

[0757] An interaction between the vehicle and the other vehicle is estimated, and a driving-related operation on the other vehicle is performed based on the estimated state and the estimated interaction.

[0758] 88. A method according to item 87, wherein estimating the state of the vehicle includes estimating the state of the driver.

[0759] 89. A method for estimating the future behavior of a vehicle and operating another vehicle, the method comprising:

[0760] monitoring movement of the vehicle during a monitoring period and by at least one sensor of the other vehicle; wherein the vehicle is different from the other vehicle;

[0761] attempting to predict the future behavior of the vehicle based on a combination of at least two of the following: (a) light signals generated by the other vehicle during the monitoring period; (b) vehicle speed and acceleration during the monitoring period; (c) a spatial relationship between the vehicle and a lane during the monitoring period; (d) the vehicle's environment during the monitoring period;

[0762] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0763] A driving related operation on the other vehicle is performed based on the estimated impact.

[0764] 90. A method according to item 89, wherein the attempt includes selecting a selected vehicle behavior mode from a plurality of vehicle behavior modes; wherein the selection is based on the monitored movement of the vehicle.

[0765] 91. A method according to item 90, wherein the selection includes finding the best matching vehicle behavior pattern.

[0766] 92. A method according to item 90, wherein the selection includes a comparison between the values ​​of the at least two elements and between the values ​​of at least two elements associated with the plurality of vehicle behavior patterns.

[0767] 93. A method according to item 90, wherein the multiple vehicle behavior modes include (a) remaining in the current lane, (b) leaving the current lane and entering the lane of the other vehicle, (c) leaving the current lane without entering the lane of the other vehicle, and (d) stopping the vehicle.

[0768] 94. A method according to item 89, wherein the estimation is responsive to information stored in a dynamic database.

[0769] 95. A method for operating a vehicle, the method comprising:

[0770] receiving, by the vehicle, from at least one entity external to the vehicle, a request to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode;

[0771] determining, by the vehicle computer, whether to change the operating mode; and

[0772] When it is determined that the operating mode is to be changed, the operating mode of the vehicle is changed from the non-autonomous driving mode to the autonomous driving mode.

[0773] 96. A method according to item 95, wherein the determination is based on a plurality of requests to change the operating mode received during a time window.

[0774] 97. A method according to item 95, wherein the request includes a risk indication associated with continuation of the non-autonomous driving mode; wherein the determination is responsive to the risk indication.

[0775] 98. A method according to item 95, wherein the at least one entity outside the vehicle includes at least one person.

[0776] 99. A method according to item 95, wherein the at least one entity outside the vehicle includes at least one other vehicle.

[0777] 100. A computer program product storing instructions which, when executed by a computerized system located within another vehicle, cause the computerized system to:

[0778] monitoring movement of a vehicle; wherein the vehicle is different from the another vehicle;

[0779] estimating an estimated interaction between a driver and the vehicle based on the motion of the vehicle and the model of the vehicle;

[0780] determining a state of the driver based on the estimated interaction;

[0781] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0782] A driving related operation on the other vehicle is performed based on the estimated impact.

[0783] 101. A computer program product storing instructions which, when executed by a computerized system located within another vehicle, cause the computerized system to:

[0784] monitoring dynamic behavior of a vehicle; wherein the vehicle is different from the another vehicle;

[0785] sensing dynamic behavior of the vehicle;

[0786] estimating a state of the vehicle;

[0787] estimating an interaction between the vehicle and the other vehicle, and

[0788] Based on the estimated state and the estimated interaction, a driving-related operation is performed on the other vehicle.

[0789] 102. A computer program product storing instructions which, when executed by a computerized system located within another vehicle, cause the computerized system to:

[0790] monitoring movement of a vehicle during a monitoring period; wherein the vehicle is different from the another vehicle;

[0791] attempting to predict the future behavior of the vehicle based on a combination of at least two of the following: (a) light signals generated by the other vehicle during the monitoring period; (b) vehicle speed and acceleration during the monitoring period; (c) a spatial relationship between the vehicle and a lane during the monitoring period; (d) the vehicle's environment during the monitoring period;

[0792] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0793] A driving related operation on the other vehicle is performed based on the estimated impact.

[0794] 103. A computer program product storing instructions which, when executed by a computerized system located within a vehicle, cause the computerized system to:

[0795] receiving a request from at least one entity external to the vehicle to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode;

[0796] determining whether to change the operating mode; and

[0797] When it is determined that the operating mode is to be changed, the operating mode of the vehicle is changed from the non-autonomous driving mode to the autonomous driving mode.

[0798] 104. A computerized system installed in another vehicle, wherein the computerized system comprises a processor and one or more sensors;

[0799] wherein the one or more sensors are configured to monitor movement of a vehicle; wherein the vehicle is different from the another vehicle;

[0800] Wherein, the processor is configured to:

[0801] estimating an estimated interaction between a driver and the vehicle based on the motion of the vehicle and the model of the vehicle;

[0802] determining a state of the driver based on the estimated interaction;

[0803] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0804] Based on the estimated impact, assisting in performing a driving-related operation of the other vehicle.

[0805] 105. A computerized system installed in another vehicle, wherein the computerized system comprises a processor and one or more sensors;

[0806] wherein the one or more sensors are configured to monitor dynamic behavior of a vehicle; wherein the vehicle is different from the another vehicle;

[0807] Wherein, the processor is configured to:

[0808] sensing dynamic behavior of the vehicle;

[0809] estimating a state of the vehicle;

[0810] An interaction between the vehicle and the other vehicle is estimated, and based on the estimated state and the estimated interaction, a driving-related operation of the other vehicle is facilitated.

[0811] 106. A computerized system installed in another vehicle, wherein the computerized system comprises a processor and one or more sensors;

[0812] wherein the one or more sensors are configured to monitor movement of a vehicle during a monitoring period; wherein the vehicle is different from the other vehicle; wherein the processor is configured to:

[0813] attempting to predict the future behavior of the vehicle based on a combination of at least two of: (a) light signals generated by the other vehicle during the monitoring period; (b) vehicle speed and acceleration during the monitoring period; and (c) a spatial relationship between the vehicle and a lane during the monitoring period; (d) the vehicle's environment during the monitoring period;

[0814] estimating an estimated impact of the vehicle on the future travel of the other vehicle; and

[0815] Based on the estimated impact, assisting in performing a driving-related operation of the other vehicle.

[0816] 107. A computerized system installed in a vehicle, wherein the computerized system comprises a processor and a communication unit;

[0817] wherein the communication unit is configured to receive a request from at least one entity external to the vehicle to change an operating mode of the vehicle from a non-autonomous driving mode to an autonomous driving mode;

[0818] The processor is configured to determine whether to change the operation mode; and when it is determined that the operation mode is to be changed, the operation mode of the vehicle is changed from the non-autonomous driving mode to the autonomous driving mode.

[0819] In the foregoing specification, the invention has been described with reference to specific examples of embodiments of the invention. It will, however, be evident that various modifications and changes may be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims.

[0820] Furthermore, the terms "front", "rear", "top", "bottom", etc. in the specification and claims, if any, are used for descriptive purposes and not necessarily for describing permanent relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in other orientations than those illustrated or otherwise described herein.

[0821] Any arrangement of components that achieve the same function is effectively "associated" so that the desired function is achieved. Thus, any two components combined herein to achieve a particular function may be considered to be "associated" with each other so that the desired function is achieved, regardless of the architecture or intermediate components. Likewise, any two components so associated may also be considered to be "operably connected" or "operably coupled" to each other so as to achieve the desired function.

[0822] In addition, those skilled in the art will recognize that the boundaries between the above operations are illustrative only. Multiple operations can be combined into a single operation, a single operation can be distributed in additional operations, and operations can be performed at least partially overlapping in time. In addition, optional embodiments can include multiple instances of specific operations, and the order of operations can be changed in various other embodiments.

[0823] However, other modifications, variations, and alternatives are also possible.

[0824] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

[0825] The phrase "may be X" indicates that condition X may be satisfied. This phrase also implies that condition X may not be satisfied.

[0826] The terms "comprising," "including," "having," "consisting of," and "consisting essentially of" are used interchangeably. For example, any method may include at least the steps included in the drawings and / or in the specification, and only the steps included in the drawings and / or in the specification. The same applies to the pool cleaning robot and the mobile computer.

[0827] It should be understood that for simplicity and clarity of illustration, the elements shown in the drawings are not necessarily drawn to scale. For example, for clarity, the size of some elements may be enlarged relative to other elements. In addition, where deemed appropriate, reference numerals may be repeated in the drawings to indicate corresponding or similar elements.

[0828] In the foregoing specification, the invention has been described with reference to specific examples of embodiments of the invention. It will, however, be evident that various modifications and changes may be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims.

[0829] Those skilled in the art will recognize that the boundaries between logic blocks are merely illustrative, and that alternative embodiments may merge logic blocks or circuit elements, or impose alternative decompositions of functionality on various logic blocks or circuit elements. It should be understood, therefore, that the architectures described herein are merely exemplary, and that in fact many other architectures that achieve the same functionality may be implemented.

[0830] Any arrangement of components that achieve the same function is effectively "associated" so that the desired function is achieved. Thus, any two components combined herein to achieve a particular function may be considered "associated" with each other so that the desired function is achieved, regardless of the architecture or intermediate components. Likewise, any two components so associated may also be considered "operably connected" or "operably coupled" to each other to achieve the desired function.

[0831] In addition, those skilled in the art will recognize that the boundaries between the above operations are illustrative only. Multiple operations can be combined into a single operation, a single operation can be distributed in additional operations, and operations can be performed at least partially overlapping in time. In addition, optional embodiments can include multiple instances of specific operations, and the order of operations can be changed in various other embodiments.

[0832] In addition, for example, in one embodiment, the illustrated examples may be implemented as circuits located on a single integrated circuit or within the same device. Alternatively, the examples may be implemented as any number of separate integrated circuits or separate devices interconnected with each other in a suitable manner.

[0833] Furthermore, for example, the examples or portions thereof may be implemented as soft representations of physical circuitry or code representations or logical representations convertible into physical circuitry, such as with any appropriate type of hardware description.

[0834] Furthermore, the present invention is not limited to physical devices or units implemented in non-programmable hardware, but may also be applied in programmable devices or units capable of performing the desired device functions by operating according to appropriate program code, such as mainframes, minicomputers, servers, workstations, personal computers, notepads, personal digital assistants, electronic game consoles, automotive and other embedded systems, cellular telephones and various other wireless devices, generally referred to in this application as "computer systems".

[0835] The present invention may also be implemented in a non-transitory computer program product storing instructions that may form a computer program for running on a computer system, the computer program comprising at least code portions for executing the steps of the method according to the present invention when running on a programmable device (e.g. a computer system) or enabling the programmable device to perform the functions of the apparatus or system according to the present invention. The computer program may cause the storage system to assign the hard disk drives to the hard disk drive group.

[0836] A computer program is a list of instructions, such as for a particular application and / or operating system. A computer program may, for example, include one or more of the following: a subroutine, a function, a procedure, an object method, an object implementation, an executable application, an applet, a servlet, source code, object code, a shared library / dynamically loaded library, and / or other instruction sequences designed to be executed on a computer system.

[0837] The computer program may be stored internally on a non-transitory computer program product. All or some of the computer programs may be provided on a computer-readable medium that is permanently, removably or remotely coupled to an information processing system. The computer-readable medium may include, for example but not limited to, any number of the following: magnetic storage media, including hard disks and tape storage media; optical storage media, such as optical disk media (e.g., CD-ROM, CD-R, etc.) and digital video disk storage media; non-volatile memory storage media, including semiconductor-based memory cells, such as flash memory, EEPROM, EPROM, ROM; ferromagnetic digital memory; MRAM; volatile storage media, including registers, buffers or caches, main memory, RAM, etc.

[0838] A computer process typically consists of an executing (running) program or part of a program, current program values ​​and state information, and resources used by an operating system to manage the execution of the process. An operating system (OS) is software that manages the sharing of computer resources and provides programmers with an interface for accessing those resources. An operating system processes system data and user input, and responds by allocating and managing tasks and internal system resources as services to the system's users and programs.

[0839] A computer system may, for example, include at least one processing unit, associated memory, and a plurality of input / output (I / O) devices. When executing a computer program, the computer system processes information according to the computer program and produces resulting output information via the I / O devices.

[0840] However, other modifications, variations, and alternatives are also possible.

[0841] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

[0842] In the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of other elements or steps than those listed in the claim. In addition, the term "a" or "an" as used herein is defined as one or more than one. In addition, the use of introductory phrases such as "at least one" and "one or more" in the claims shall not be construed as implying that the introduction of another claim element by the indefinite article "a" or "an" will limit any particular claim containing such introduced claim element to an invention containing only one such element, even if the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an". The same applies to the use of definite articles. Unless otherwise specified, terms such as "first" and "second" are used to arbitrarily distinguish the elements described by such terms. Therefore, these terms are not necessarily intended to indicate the temporal or other priority of such elements. The mere fact that certain measures are stated in mutually different claims does not indicate that a combination of these measures cannot be used advantageously.

[0843] Any system, device or apparatus referenced in this patent application includes at least one hardware component.

[0844] Although certain features of the present invention have been illustrated and described herein, many modifications, substitutions, changes and equivalents will occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes that fall within the true spirit of the invention.

[0845] Any combination of any components and / or any components of any units of the systems shown in any figures and / or description and / or claims may be provided.

[0846] Any combination of any systems listed in any figures and / or description and / or any claims may be provided.

[0847] Any combination of the steps, operations and / or methods listed in any figures and / or description and / or claims may be provided.

[0848] Any combination of the operations listed in any of the figures and / or description and / or claims may be provided.

[0849] Any combination of the methods listed in any of the figures and / or the description and / or the claims may be provided.

Claims

1. A method for operating a vehicle, the method comprising: sensing an environment of the vehicle by at least one sensor of the vehicle, the environment including a dynamic object; estimating an estimated effect of the dynamic object on future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic object; and A driving-related operation of the vehicle is performed based on the estimated impact.

2. The method according to claim 1, wherein: The sensing occurs at a specific location, and wherein the estimated behavior of the dynamic object is based on the behavior of other dynamic objects at the specific location, wherein information about the behavior of the other dynamic objects at the specific location is stored in the dynamic database.

3. The method according to claim 1, wherein: The estimated behavior of the dynamic object is based on the behavior of other dynamic objects in environments similar to the environment of the vehicle.

4. The method according to claim 1, wherein: The dynamic object is another vehicle.

5. The method according to claim 4, wherein: The information is about an expected driving pattern of the other vehicle.

6. The method according to claim 1, wherein: The dynamic object is a person. The method of claim 1 , comprising reporting sensing of the dynamic object.

8. The method according to claim 1, wherein: The performing driving-related operations on the vehicle includes autonomously driving the vehicle.

9. The method according to claim 1, wherein: Performing driving-related operations on the vehicle includes changing an operating mode of the vehicle between an autonomous driving mode and a non-autonomous driving mode.

10. The method according to claim 1, wherein: Performing a driving-related operation on the vehicle includes generating an alert perceptible to the driver.

11. The method according to claim 1, wherein: Performing a driving-related operation on the vehicle includes reducing a speed of the vehicle.

12. The method according to claim 1, wherein: Performing a driving-related operation on the vehicle includes changing a future travel of the vehicle to obtain better sensing of the dynamic object.

13. The method of claim 1, comprising detecting a change between a sensed content of the environment and an expected content of the environment. The method of claim 13 , comprising determining whether at least one of the changes is to be reported.

15. The method of claim 13, comprising reporting an absence of movable objects on roads included in the environment.

16. The method of claim 1, comprising reporting an estimated impact of the dynamic object on future travel of the vehicle.

17. The method of claim 1, comprising receiving or generating a risk map and updating the risk map with the estimated impact of the dynamic object on future travel of the vehicle.

18. The method according to claim 1, wherein: The sensing includes sensing, by vehicle sensors, signals associated with a set of points of interest in the environment.

19. The method according to claim 18, wherein: Information about locations of points of interest in the set of points of interest is retrieved prior to the sensing.

20. The method according to claim 18, wherein: The sensing is followed by estimating the quality of points of interest in the set of points of interest.

21. The method according to claim 18, wherein: A quality of a point of interest in the set of points of interest is reported after the sensing.

22. The method according to claim 18, wherein: The sensing includes sensing a signal associated with at least one point of interest, the at least one point of interest being relevant within a recurring time window and not relevant outside the recurring time window.

23. The method according to claim 18, wherein: The sensing includes sensing a signal associated with at least one point of interest, the at least one point of interest being relevant within one or more time windows and not relevant outside the one or more time windows.

24. The method according to claim 1, wherein: The sensing includes collecting signals from multiple sets of points of interest in the environment through various types of vehicle sensors.

25. The method according to claim 24, wherein: The various types of vehicle sensors include cameras, light detection and ranging sensors, and radio frequency radars.

26. The method of claim 25, comprising determining at least one point of interest for at least one type of vehicle sensor based on information obtained by at least one other type of vehicle sensor.

27. The method of claim 1, wherein: The sensing includes selecting which type of vehicle sensor to use for sensing the vehicle's environment based on at least one of an actual state or an estimated state of the environment.

28. The method of claim 1, comprising transmitting information about the environment to a computerized system located external to the vehicle.

29. The method of claim 28, comprising generating information about the environment by applying privacy protection measures.

30. The method of claim 29, wherein: Applying the privacy protection measure includes masking at least one of the vehicle identification information and the person identification information.

31. The method of claim 1, comprising reporting sensed motion of the dynamic object.

32. A computer program product storing instructions which, when executed by a computerized system installed in a vehicle, cause the computerized system to: sensing an environment of the vehicle by at least one sensor of the computerized system, the environment including dynamic objects; estimating an estimated impact of the dynamic object on the future travel of the vehicle; wherein, The estimating is responsive to information stored in a dynamic database, wherein the information is about an estimated behavior of the dynamic object; and A driving-related operation of the vehicle is performed based on the estimated impact.

33. A computerized system installed in a vehicle, wherein: The computerized system comprises: at least one sensor configured to sense an environment of the vehicle, the environment including dynamic objects; and A processor configured to (a) estimate an estimated impact of the dynamic object on the future travel of the vehicle; wherein the estimation is responsive to information stored in a dynamic database, wherein the information is about the estimated behavior of the dynamic object; and (b) perform a driving-related operation on the vehicle based on the estimated impact.

Images