Procedure and server setup for collecting environmental data

Abstract

Method for collecting environmental data, in which - Environment data (UD1, UD2, UD n ), which are driven by vehicles (F1, F2, F n ) have been sensorily detected in a limited geographical area, as well as localization data (LD1, LD2, LD n ) for the respective positions where the environmental data (UD1, UD2, UD n ) have been recorded, are received (1); - the received environment data (UD1, UD2, UD n ) and localization data (LD1, LD2, LD n ) are aggregated into swarm data (SD) (2); - the data quality of the swarm data is determined (3) and checked (4) to see if the data quality meets a predefined quality criterion; - in the event of insufficient data quality of the swarm data (SD), a request (A x ) to other vehicles (F x) transmitted (5) to adjust the respective driving style by reducing the driving speed so that it transmits additional environmental data (UD) in the limited geographical area x ) as well as location data (LD x ) be able to capture them without errors; and - additional environmental data recorded by the other vehicles after adjustment of their respective driving style (UD x ) and location data (LD x ) are received and added to the existing swarm data (SD) until the data quality of the swarm data (SD) meets the quality criterion.

Description

[0001] The present invention relates to a method for acquiring environmental data, which can be used in particular to generate improved swarm data. The present invention further relates to a corresponding server device. The improved swarm data enables, for example, a more precise recording of animal and / or plant populations in a limited geographical area.

[0002] In manned vehicles such as motor vehicles, as well as unmanned, autonomously operating vehicles such as robots, a variety of different sensors are now used to monitor the vehicle's surroundings. These sensors can include, for example, one or more video cameras or ultrasonic, radar, or laser-based sensors.

[0003] There are a wide variety of applications for the data obtained with environmental sensors. For example, they are used in motor vehicles for a wide variety of assistance systems. Furthermore, environmental sensors also form the basis for highly automated and fully autonomous driving of robots and motor vehicles. A relatively new area of ​​application is animal counting.

[0004] US 2015 / 0 339 826 A1 discloses a method for controlling a mobile robot, such as a wheel-driven robot or an aerial robot, based on video data. For this purpose, data streams from multiple image sensors can be combined to determine distance information to obstacles in the robot's environment from depth information, which can be used for control. A variety of different potential applications are mentioned, including the visual monitoring or counting of people or animals. CN 1 09 948 395 A describes a method that allows livestock farmers to identify and count their animals in real time using a drone. EP 3 121 764 A1 describes an animal species determination of animals captured by a camera based on pattern recognition in a vehicle for collision avoidance.

[0005] In order to increase the data quality and thus the meaningfulness and reliability of the data and information obtained, it is also known to collect data from many vehicles and statistically evaluate them instead of the data from a single vehicle.

[0006] The swarm data collected and evaluated by the multitude of vehicles, hereinafter also referred to as the vehicle swarm, can then, assuming a sufficient vehicle swarm size and thus a sufficient database, provide various information about the current vehicle environment, such as road conditions, lane data, or traffic jam information. This information can then be used, for example, to select or adjust routes.

[0007] EP 3 244 165 A2 discloses a method for collecting observation data for updating a geographic database. An initial observation is captured by a first mobile device, a first vehicle, or a first sensor. Data indicative of the initial observation, along with the geographic position, is sent to a server. The central server analyzes the data from this initial observation to determine whether further observations should be performed. Based on the analysis of the data and the geographic position of the first mobile device, the central server defines a bounding box and sends a request for additional observations to at least a second mobile device, a second vehicle, or a second sensor.

[0008] US 2014 / 0 081 479 A1 discloses a forest management system in which information about a forest is received and analyzed by a group of autonomous vehicles to obtain information about the forest's condition and coordinate the operation of the group of autonomous vehicles. A swarm of vehicles can be directed to a specific location to generate information at that location.

[0009] US 2020 / 0 014 759 A1 presents a method for collectively optimizing cooperative actions between autonomous, connected vehicles. A shared map of the surroundings of the autonomous vehicles can be generated, and based on this map, a navigation plan can be generated for navigating the environment. By transmitting sensor information from a vehicle that has detected an object such as a pedestrian, an animal, or a non-connected vehicle, the connected vehicles can be alerted to the presence of the foreign object.

[0010] US 2016 / 0 355 181 A1 discloses an assistance device for collision avoidance. A camera is used to capture the vehicle's surroundings and detect an image of an animal contained therein. The species of the animal detected in the image is determined, and a behavioral characteristic index value is retrieved to determine the animal's future presence area and, based on this, the probability of a collision between the animal and the vehicle, and to implement a collision avoidance measure.

[0011] DE 10 2014 226 358 A1 relates to a method for determining surface soiling of a vehicle, in which the soiling is detected visually or acoustically using a soiling sensor. Furthermore, a soiling warning device is provided to assess the degree of soiling based on the soiling data detected by the soiling sensor and, if necessary, issue a washing recommendation.

[0012] It is an object of the invention to provide an improved method for acquiring environmental data and a corresponding server device.

[0013] This object is achieved by a method having the features of claim 1 and by a server device according to claim 10. Preferred embodiments of the invention are the subject of the dependent claims.

[0014] The invention is based on the following insight. Just as swarm data, with a sufficient database, can serve as a basis for selecting vehicle routes and / or driving styles, adapting the routes and / or driving styles of vehicles collecting swarm data can also provide the opportunity to contribute to improving the data quality of this swarm data.

[0015] In a method according to the invention, environmental data recorded by sensors from vehicles within a limited geographical area, as well as localization data for the respective positions at which the environmental data was recorded, are received. The received environmental data and localization data are combined to form swarm data. The data quality of the swarm data is determined, and a check is performed to determine whether it meets a predefined quality criterion. If the data quality of the swarm data is insufficient, a request is sent to other vehicles to adapt their respective driving style so that they can accurately record additional environmental data and localization data within the limited geographical area.After adjusting the respective driving style, additional environmental data and localization data are received from the other vehicles and added to the existing swarm data until the data quality of the swarm data meets the quality criterion.

[0016] In this way, vehicles can be guided specifically to geographical regions or routes for which there is initially too little data available in order to obtain additional data there using their environmental sensors and thus meet a certain quality requirement for the swarm data or to ensure that the swarm data is reliably meaningful.

[0017] In particular, the request can be transmitted to autonomous vehicles, which then adapt their driving style based on it.

[0018] According to one embodiment of the invention, the environmental data relate to animal and / or plant populations in the vehicle environment, which are recorded by an environmental sensor system of the vehicle.

[0019] This will meet the need to optimise the recording of animal and plant populations as much as possible. This is of increasing importance because, for example, various insect and bird populations that are worthy of protection have declined drastically in recent years, with fears of a long-term decline in the entire bird population and the local or complete disappearance of species, often referred to as bird extinction. As things stand, many of the approximately 250 bird species native to Germany are threatened with extinction. The bird die-off, in turn, is caused, among other things, by the decline in insects, as insects serve as a food source for many bird species. In this context, it is important to continuously monitor the extent to which animal and plant populations are changing in order to be able to implement measures to preserve endangered animal and plant populations at an early stage if necessary.Therefore, conservation organizations regularly conduct complex counts to determine current populations and thus detect a decline in animal and plant populations, or possibly even an increase in individual populations, which in turn can reduce the numbers of other endangered animal and plant populations. This can be drastically improved by the continuous, automated recording using the environmental sensors of the swarm vehicles.

[0020] In this way, vehicle-based determination of plant and animal populations can be carried out in an optimized manner by adapting routes and / or driving styles of autonomous vehicles.

[0021] According to a particularly preferred embodiment of the invention, a request is transmitted to further vehicles to adapt the driving styles of the further vehicles in such a way that more precise swarm data on the animal and / or plant populations are made possible by additional recording of the animal and / or plant populations in the limited geographical area.

[0022] In particular, a classification and / or counting of animal and / or plant populations is carried out based on the environmental data.

[0023] According to a further particularly preferred embodiment of the invention, a request is transmitted to further vehicles to adapt the driving styles of the further vehicles in such a way that the probability of collisions with animals of the detected animal populations or of coming close to animals or plants of the detected animal and / or plant populations is minimized.

[0024] Advantageously, optical recording is carried out with subsequent image processing and object recognition of the animal and / or plant populations.

[0025] Likewise, acoustic and / or thermal features can advantageously be recorded as additional environmental data.

[0026] According to a further embodiment, insects are detected and recognized immediately before, during or after a collision with the windshield of the respective vehicle.

[0027] The server device according to the invention for recording environmental data comprises a receiving unit for receiving environmental data and localization data from a plurality of vehicles, wherein the environmental data have each been recorded by the vehicles in a limited geographical area and the localization data indicate the respective positions at which the environmental data have been recorded.A processing unit is provided which combines the received environmental data and localization data into swarm data, determines the data quality of the swarm data and checks whether the data quality meets a predefined quality criterion. If the data quality of the swarm data is insufficient, it generates a request to other vehicles to adapt their driving style by reducing their speed so that they can accurately record additional environmental data and localization data within the limited geographical area. It also adds additional environmental data and localization data received from the other vehicles after adapting their driving style to the existing swarm data until the data quality of the swarm data meets the quality criterion. Furthermore, a transmitting unit is provided for transmitting the request to the other vehicles.

[0028] Further features of the present invention will become apparent from the following description and claims in conjunction with the figures. Fig. 1 shows a flow diagram of a method according to the invention; and Fig. Figure 2 schematically shows an example with swarm vehicles that report environmental data to a server and another vehicle that receives a request from the server to adapt the driving route and / or driving style and generates additional swarm data after the adaptation.

[0029] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features may also be combined or modified without departing from the scope of the invention as defined in the claims.

[0030] Fig. Figure 1 shows a flowchart of a method according to the invention. The method according to the invention can be executed, in particular, by a central server. The method is described below as an example for adapting routes and / or driving styles of autonomous vehicles to generate improved tracking data for determining plant and animal populations, but is not limited thereto.

[0031] In a method step 1, the central server receives environmental data from a plurality of vehicles which have been recorded by them while driving through a geographical area, as well as localization data for the respective positions at which the environmental data have been recorded.

[0032] The environmental data can be recorded using various environmental sensors of the respective vehicles. The vehicle's surroundings can be recorded, in particular, using one or more of the vehicle's cameras, which image the vehicle's surroundings in the visible spectral range or can also be equipped with infrared cameras. However, this can also be done in addition to or instead of other sensors, such as ultrasound, radar, or laser-based sensors or microphones. In order to generate the environmental data from the sensor data generated by the environmental sensors, the sensor data may first be processed, depending on the type of environmental sensor.

[0033] For example, image data captured with a camera is subjected to image processing and object recognition. For example, when determining plant and animal populations, plants and animals in the vehicle's surroundings can be detected, recognized or classified based on suitable pattern recognition. For classification, an object classifier can be used, for example, which is trained using machine learning to classify visual objects found from still images or motion videos according to their type. In this way, individual animal and / or plant species can be differentiated in the field of view with their respective optical characteristics. In the case of plants, for example, this can be based on their flowers, leaves, plant shape, and correspondingly for animals based on their physical dimensions, color characteristics, plumage or fur patterns.

[0034] Furthermore, by evaluating additional sensor data, additional features can be identified and taken into account in the classification. For example, acoustic features generated by evaluating audio signals captured by a microphone integrated in the vehicle and subsequent audio processing can be taken into account. For example, birdsong, the chirping of insects, or the location calls of bats in the ultrasonic range can be acoustically detected, allowing conclusions to be drawn about the corresponding animal species and populations. Likewise, thermal features can be evaluated, for example, based on the sensor data from an infrared camera implemented in the vehicle and subsequent image processing. This means that animal species and corresponding populations can be drawn even in the dark, allowing, for example, bats or other nocturnal animals to be identified.

[0035] The sensor data can be processed directly in the respective vehicle, so that only in the event of a detection, for example of a plant or animal, does the environmental data need to be transmitted to the central server, thus keeping data traffic as low as possible. However, it can also be provided that the sensor data is transmitted from the respective vehicle as raw data to the central server, which then carries out the corresponding processing. This can be particularly advantageous when complex algorithms must be implemented for unambiguous recognition, for example, to accurately distinguish whether a detected animal belongs to an endangered species or, instead, to a similar-looking but not endangered species.

[0036] The location data of the vehicles during the collection of environmental data can be determined using satellite-based navigation systems such as GPS installed in the vehicles, possibly in combination with digital map data. However, mobile radio-based positioning is also possible.

[0037] The central server then compiles the received environmental data and localization data into swarm data in process step 2. This allows, for example, an estimate of bird or insect populations in the geographical area under consideration. For this purpose, the central server can access a database in which a comprehensive collection of swarm data for the road network in a larger geographical area is stored.

[0038] In process step 3, the swarm data is statistically evaluated by the central server using suitable algorithms to determine the data quality of the swarm data. This makes it possible to examine the confidence that can be assigned to the environmental data or to the animals or plants detected in the vehicle's surroundings, i.e. how trustworthy the obtained environmental data is. For example, it can be determined how often an animal or plant must be detected in order to be able to make a reliable statement when recording the animal and plant populations. This can also be made dependent on whether, for example, different vehicles in the swarm provide contradictory information. Furthermore, the received environmental data can be checked for plausibility in order to eliminate an outlier that is due, for example, to a malfunction of a sensor in the vehicle in question.

[0039] If necessary, time-related parameters can also be taken into account during the statistical analysis of the swarm data, thus providing time-dependent swarm data. For example, swarm data can be collected only at certain times of the day or year, for example, to specifically count or observe certain animal and / or plant species during these periods.

[0040] If the data quality of the swarm data determined in process step 3 is not sufficient to meet a specific quality requirement for the swarm data, a request is sent from the central server in process step 4 to other vehicles to adjust their respective route and / or driving style. In this way, vehicles can be prompted to enter the affected geographical area, even if the original route did not include this, in order to then collect additional environmental data with their environmental sensors. Likewise, by adjusting the driving style of vehicles that are already in the affected geographical area, for example by significantly reducing their driving speed, it can be ensured that objects in the vehicle's surroundings can be detected without error.For example, an optimal driving speed can be selected to capture specific animal species in order to optimize camera capture.

[0041] The request is transmitted primarily to autonomous vehicles, as they have comprehensive environmental sensors for autonomous operation. However, such a request can also be transmitted to manually driven vehicles. In this case, a message can be issued to the driver in accordance with the request, asking them to adjust their route and / or driving style.

[0042] It may also be provided that the transmitted request also includes information about the required environmental sensors, so that an adjustment of the route and / or driving style is only carried out by vehicles that have suitable environmental sensors.

[0043] The additional environmental data and localization data are then transmitted by the other vehicles to the central server after adjusting their respective routes and / or driving styles. These are received by the central server and added to the existing swarm data. This process continues until the data quality of the swarm data is deemed sufficient and the swarm data is then used in process step 6.

[0044] For example, detailed information can be provided on which route sections certain animal and / or plant species are prevalent, and changes in populations can be detected early on. For example, the optimized swarm data for the traveled routes can be used to determine how many and which animals or plants were sighted at which locations, at which times of day or in which season. This allows for the calculation, using appropriate statistical methods, of how many animals or plants, or animal or plant species, are present in a particular region, and also to track population expansions, decreases, or movements.

[0045] In this case, a set of swarm data along a traveled route can form a transect. The analysis can then be carried out analogously to the procedure used in traditional animal censuses using line transect mapping, in which an area is scanned along predetermined, usually linear lines and the identified animal or plant species are recorded. For regions with a sufficiently dense road network, the swarm data can then, with an appropriate approach, provide very precise information. This applies not only to rural areas but also to urban regions.

[0046] However, due to their small size, it is difficult to resolve small creatures and insects in detail at a distance using cameras. In this case, however, the invention can provide for flying insects to be recorded while driving shortly before a collision with the vehicle's windshield using a vehicle camera aimed at the windshield. Likewise, traces left on the windshield after the collision can be detected and these traces can be evaluated using image processing, which can then be used to determine the type of insect. The impact of individual insects can also be recorded acoustically from a distance in the vehicle cabin using an interior microphone with a characteristic impact noise or with a vibration sensor that is in contact with the windshield.In this way, the number of dead insects (species) on the windshield per unit of time can be determined depending on the driving speed and thus conclusions can be drawn about the insect population.

[0047] If a threshold for collisions per unit of time with insects or with a detected insect species of concern on the windshield is exceeded, the driving speed of the autonomous vehicle can be automatically reduced until the corresponding threshold is exceeded. This threshold can also depend on the detected insect species. The detection of a frequent occurrence of insect species of concern in the air can be transmitted to the central server, allowing other vehicles to change their routes if necessary to use routes that potentially harm less vulnerable insects.

[0048] In addition to the direct visual detection of animal and plant species, inferences can also be made about animal and plant populations indirectly. For example, in addition to the aforementioned visual detection of traces on the windshield, animal tracks, such as animal droppings, fur remains, bird feathers, and animal dander, can also be detected on the road or at the edge of the road. Plant traces, such as accumulations of pollen, plant seeds, leaves, and flower petals, can also be detected on the road or at the edge of the road. Furthermore, sightings of animal nests and structures, such as bird nests in trees, shelters, deer tracks in tree bark, or anthills, can also be evaluated.

[0049] Information about detected animal and plant populations can also be used to issue warnings or make route adjustments, ensuring that routes are chosen that, for example, have as little contact as possible with allergenic animals, such as the oak processionary moth, or plants, such as ragweed, and thus pose as little health risks to vehicle users as possible. Compared to air quality measurements using air quality sensors on the vehicle, the above-mentioned approaches, with detection based primarily on optical characteristics, have the advantage that the measurements are not dependent on wind direction.

[0050] It can also be planned that, for warnings to the vehicle user or optimized route adjustments, the data from the vehicle's environmental sensors can be shared with measurement data from air quality sensors on the vehicle and / or stationary air quality measuring stations. Furthermore, route adjustments or warnings can be carried out according to individual vehicle user profiles. For example, user profiles can be stored on a server for individual people, including the known allergies of the respective people.

[0051] In Fig. 2 schematically shows how environmental data is collected by a swarm of vehicles and transmitted to a server device S, wherein the server device S transmits a request to adapt the route guidance or driving style to another vehicle due to insufficient data quality of the swarm data.

[0052] The central server can be deployed online as a backend server for the vehicle swarm and be part of an IT infrastructure not further described here. Communication between the respective vehicle and the server takes place via a wireless data connection, for example, using mobile radio units provided in the vehicles.

[0053] A variety of swarm vehicles F1 to F n drive within the same limited geographical area. Vehicles F1 to F n record environmental data UD1, UD2, UD n to the current vehicle environment as well as their current position when the environment data is recorded, from which they generate localization data LD1 to LD ngenerate. In the example shown, the swarm vehicles each have a camera K for optically capturing the environmental data, with which, for example, a wild animal T is detected. Furthermore, a communication unit C is provided in each swarm vehicle, by means of which the currently recorded environmental data and localization data, in the example shown the animal species and position of the detected wild animal, are transmitted to the server device S.

[0054] The server device S has a receiving unit 11 with which the environmental data UD1, UD2, UD n and localization data LD1 to LD nreceived by the sponge vehicles. The receiving unit 11 feeds the received data to a processing unit 12, where it is combined into swarm data SD and statistically evaluated using suitable algorithms to check whether the data quality of the swarm data meets a predefined quality criterion. The swarm data can be temporarily stored in a storage device 13. Likewise, information about the geographical area currently being traveled by the vehicles when the environmental data is being recorded can also be accessed. This information is present in the storage device 13 as part of a digital map and can be retrieved from it.

[0055] In the event of insufficient data quality of the swarm data SD in the limited geographical area, the processing unit 12 determines an optimized route for vehicles currently or in the near future in the area, or an optimized driving style for vehicles in this area. For this purpose, the processing unit 12 can also access digital map data stored in the storage device 13, which describe the road network and can be used for route calculation.

[0056] A request A x with information for an optimized route guidance or for an optimized driving style is then sent by a transmitting unit 14 of the server device S to other vehicles, of which, for example, a vehicle F x shown, in order to prompt them to adjust the route or driving style. This additional vehicle F xis also equipped with a vehicle camera K and a communication unit C.

[0057] After the other vehicle F x a corresponding adjustment of the route and / or driving style has been made, this can then provide additional environmental data UD x and localization data LD x generate and transfer to the server facility S to supplement the swarm data already available. List of reference symbols 1 - 6 process steps F1, F2, F n Swarm vehicles F x another vehicle K Camera C Communication unit T Animal around vehicle environment UD1, UD2, UD n , UD x Environmental data LD1, LD2, LD n , LD x Localization data A x request SD swarm data S Server 11 Receiving unit 12 processing unit 13 Storage device 14 Transmitter unit

Claims

[1] Method for collecting environmental data in which - Environmental data (UD1, UD2, UD n ) used by vehicles (F1, F2, F n ) have been recorded by sensors in a limited geographical area, as well as localization data (LD1, LD2, LD n ) for the respective positions where the environment data (UD1, UD2, UD n ) have been recorded, are received (1); - the received environmental data (UD1, UD2, UD n ) and localization data (LD1, LD2, LD n ) are summarized to form swarm data (SD) (2); - the data quality of the swarm data is determined (3) and it is checked (4) whether the data quality meets a predefined quality criterion; - in case of insufficient data quality of the swarm data (SD), a request (A x ) to other vehicles (F x) is transmitted (5), to adapt the respective driving style by reducing the driving speed so that this additional environmental data (UD x ) and localization data (LD x ) without errors; and - additional environmental data recorded by the other vehicles after adjusting the respective driving style (UD x ) and localization data (LD x ) and added to the existing swarm data (SD) until the data quality of the swarm data (SD) meets the quality criterion. [2] Method according to claim 1, wherein the request (A x ) is transmitted to autonomous vehicles (5), which then adapt their driving style based on this. [3] Method according to claim 1 or 2, wherein the environmental data (UD1, UD2, UD n) Animal and / or plant populations in the vehicle's surroundings, which are recorded by the vehicle's environmental sensors. [4] Method according to claim 3, wherein a request (A x ) to other vehicles (F x ) is transmitted (5), the driving styles of the other vehicles (F x ) in such a way that an additional recording of the animal and / or plant populations in the limited geographical area enables more precise swarm data (SD) on the animal and / or plant populations. [5] Method according to claim 4, wherein a classification and / or counting of the animal and / or plant populations is carried out based on the environmental data. [6] Method according to claim 3, wherein a request to further vehicles (F x ) is transmitted (5), the driving styles of the other vehicles (F x) in such a way as to minimise the likelihood of collisions with animals belonging to the recorded animal populations or of coming into close contact with animals or plants belonging to the recorded animal and / or plant populations. [7] Method according to one of claims 3 to 6, wherein an optical detection with subsequent image processing and object recognition of the animal and / or plant populations takes place. [8] Method according to claim 7, wherein the additional environmental data (UD1, UD2, UD n ) acoustic and / or thermal characteristics are recorded. [9] Method according to one of claims 3 to 8, wherein insects are detected and recognized immediately before, during or after a collision with the windshield of the respective vehicle. [10] Server device (S) for recording environmental data, with - a receiving unit (11) for receiving environmental data (UD1, UD2, UD n ) and localization data (LD1, LD2, LD n) from a variety of vehicles (F1, F2, F n ), where the environmental data (UD1, UD2, UD n ) from the vehicles (F1, F2, F n ) have been recorded in a limited geographical area and the localization data (LD1, LD2, LD n ) the respective positions at which the environment data (UD1, UD2, UD n ) have been recorded; - a processing unit (12) which - the received environmental data (UD1, UD2, UD n ) and localization data (LD1, LD2, LD n ) into swarm data (SD), - determines the data quality of the swarm data and checks whether the data quality meets a predefined quality criterion, - if the data quality of the swarm data is insufficient, a request (A x ) to other vehicles (F x) to adapt the respective driving style by reducing the driving speed so that in the limited geographical area additional environmental data (UD x ) and localization data (LD x ) without errors; and - environmental data received from the other vehicles and additionally recorded after adjusting the respective driving style (UD x ) and localization data (LD x ) to the previous swarm data (SD) until the data quality of the swarm data (SD) meets the quality criterion; and - a transmitting unit (14) for transmitting the request to the other vehicles.

Images