Method and vehicle for capturing and providing information and information system for this purpose
Mobile image capture devices in vehicles address the challenge of incomplete spatial data from stationary units by recording and analyzing location-specific, time-stamped data, enhancing data timeliness and availability through a fleet-based system for dynamic data collection.
Patent Information
- Application Number
- DE102010022707
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2010-06-04
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2030-06-04
AI Technical Summary
Existing stationary image capture devices provide incomplete spatial data despite high equipment costs, and there is a need to improve data timeliness and availability, particularly for changeable information across multiple locations.
A method involving mobile image capture devices in vehicles that record, analyze, and transmit location-specific and time-stamped image data to a database, enabling extraction and display of timely information, such as fuel prices, using a fleet of vehicles to dynamically collect data at relevant locations and times.
Enhances data timeliness and availability by providing current and location-dependent information, allowing for temporal analysis and correction of data sets, optimizing resource use, and supporting dynamic data collection.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to a method for capturing and providing information using a plurality of image capture devices, each arranged in a vehicle. Furthermore, the invention relates to a vehicle for capturing and providing information using image capture devices and to an information system comprising a plurality of such vehicles.
[0002] The collection of data, its processing for a specific purpose, and the provision of the processed data by a commercial data service have a wide range of possible applications. Among the various raw data that can be used for such data services, image data in particular poses a challenge. On the one hand, it contains large amounts of information, making data processing for a specific purpose a complex task. The collection of image data at a variety of locations can typically be carried out using many stationary recording units, for example permanently installed so-called web cameras. The recorded image data can be saved and either analyzed at a later time or transmitted in real time to a central evaluation unit, where the various image data from several such recording units are evaluated together.The disadvantage of fixed-location recording devices, however, is that many of them, despite the relatively high equipment costs, can only capture very fragmented data. For example, a fixed-location webcam can block or adjust the sector being monitored.
[0003] In recent years, various applications for mobile data capture and the utilization of such data for data services have been proposed. Image data can be captured, for example, by a camera installed in a vehicle and assigned to the respective position of the vehicle at the time of capture. For example, the Google Maps road map service provides panoramic street-level images captured in a vehicle. This allows users to view not only the logical street map but also a true-to-life view of the respective street, which was previously captured by a vehicle while passing by.
[0004] DE 20 2007 018 605 U1 describes a navigation program in which image data is captured at an approximate location using a digital camera. Pattern recognition is used to identify predefined features in the captured images and compare them with a reference. This allows correction data for digital maps to be obtained.
[0005] US Pat. No. 7,333,676 B2 discusses a method for automatic text recognition in a landscape image and the extraction of text for subsequent processing. In particular, text is recognized from an oblique viewing angle and transformed into a normalized coordinate system.
[0006] DE 10 2005 016 006 A1 discloses a method for providing gas station information for a motor vehicle. The method involves determining the current position of a vehicle and transmitting information on fuel prices for a region surrounding that position. If the fuel level falls below a minimum level, all accessible gas stations, especially the cheapest one, can be displayed along with the current fuel price information. The fuel price information is provided promptly, for example, by the gas station chains themselves.
[0007] JP 2008 - 165 652 A discloses a fuel price recording system that captures images of gas station price information using a device installed in a motor vehicle. The images are then evaluated, and the position and price information are forwarded to a central information storage device.
[0008] DE 101 05 175 A1 discloses a vehicle computer system and a method for determining a petrol station to be approached, in which a transmitting unit for transmitting location data to a provider, a receiving unit for receiving data from a petrol station, and a selection device for selecting one of the received petrol stations according to a predeterminable criterion are provided.
[0009] It is the object of the present invention to provide a method, a vehicle and an information system of the type mentioned above, in which the data timeliness and data availability of data services is improved, in particular when information on changing data is provided at a plurality of locations.
[0010] This object is achieved according to the invention by a method having the features of claim 1, a vehicle having the features of claim 7, and an information system having the features of claim 8. Advantageous embodiments and further developments emerge from the dependent claims.
[0011] In the method according to the invention for capturing and providing information, image data is captured at a plurality of locations by a plurality of image capture devices, each arranged in a vehicle, and the corresponding location and time of capture are recorded. The image data is analyzed, and information of a specific category is extracted from the image data. A data set containing the extracted information and the corresponding location and time is transmitted to a database, and the data set is made available by the database for output.
[0012] According to the invention, the extracted information includes fuel prices from the price displays at petrol stations, while additional information about the petrol station, namely the length of the queue at the pumps, is also included.
[0013] The extracted information includes fuel prices from the price displays at petrol stations, whereby alternatively or additionally the fuel types and / or information about the petrol station, for example the opening hours of the petrol station, are also included.
[0014] By assigning a point in time to each piece of information extracted from the image data, the timeliness of the information is calculated from this timestamp of the respective image data. The timeliness of the information is then output by the database in a suitable form or is made available for output. The timeliness is typically calculated from the time difference between the time of capture and the time the information is queried. However, a different measure of timeliness can also be defined, which is based on daily or weekly deadlines, for example, so that there is no linear and / or continuous dependence on the time period elapsed since capture. The representation can be done, for example, using a color code, so that information with a high level of timeliness is displayed in green, for example, and information with a low level of timeliness is displayed in red, for example.In particular, the time- and location-dependent information can be displayed at the corresponding positions on a geographical map.
[0015] The temporal marking of data collection also allows for a temporal analysis of the data records. For example, information in the database that is assigned to at least two different points in time but the same location is compared with each other, and the result of the comparison is then output by the database or made available for output. If the information is price information, a price trend could, for example, be determined. In particular, the data records can be interpolated or extrapolated spatially and temporally if this seems appropriate, e.g., if data records from certain locations have been strongly correlated with each other in time in the past, but no sufficiently accurate data record is available for such a location at the time the information is retrieved.Data sets captured through a computational analysis could also be compared and validated, for example, to identify incorrectly transmitted price information. For example, a spatial and / or temporal comparison can be used to reject an extracted figure that lies outside a specified value range as unrealistic or correct it. The data can also be further evaluated and made available to users in a processed form, e.g., according to previously defined user criteria.
[0016] The arrangement of image capture devices in a large number of vehicles makes it possible to repeatedly collect image data at many locations and at many times, in the vicinity of which at least one of the vehicles is currently located. This is particularly advantageous when capturing image data that contains regularly changing information that is generally not assumed to be the same at two consecutive points in time.
[0017] The multitude of vehicles forms a vehicle fleet, which can be operated physically by a company or virtually managed by a service provider, with the goal of dynamically capturing image data at the locations and times when these can be captured. The vehicles themselves can serve a dual function. On the one hand, they support the dynamic data capture process through their own mobility, and on the other hand, the vehicle driver can benefit from the dynamically captured data sets.
[0018] The location associated with a captured image can be determined, for example, using a navigation program in the vehicle in question or using common positioning methods such as satellite positioning, e.g., GPS (Global Positioning System), or terrestrial mobile radio positioning methods, e.g., OTDOA (Observed Time Difference Of Arrival). The time can easily be determined using a synchronizable clock in the vehicle that is connected to the image capture device. The time determined for a captured image is also referred to as the image's timestamp.
[0019] The transmission of a data set to the database typically occurs wirelessly, e.g., via a cellular connection between the vehicle and a service center where the database is operated and the data sets can be stored centrally. A cellular connection can be provided, for example, by a cellular module integrated into the vehicle or a detachably connected mobile phone using a common cellular standard such as GPRS or UMTS, provided that the connection can be linked to the image capture device for data transmission purposes.
[0020] The image data can, for example, be recorded continuously or periodically, buffered, and processed at a later time. However, it is advantageous to determine the current position of the vehicle, to determine whether the geographical position of an object assigned to a certain category is in the vicinity of the determined current position of the vehicle, and to only automatically record the image data when the geographical position of an object in the category is in the vicinity of the determined current position of the vehicle. This makes it possible to optimize energy consumption and other system resources. In this case, image data acquisition is activated when the vehicle is near locations relevant for the respective data service. These can be determined, for example, using a navigation program.
[0021] Advantageously, the information is extracted from the image data by an analysis device located in the vehicle. This allows the amount of information provided for the data service to be reduced to the relevant portion at the point of acquisition, thus conserving the storage and transmission resources required in the vehicle.
[0022] The information extracted from the captured image data of the specific category includes, in particular, text, numbers, and / or symbols. These include, for example, publicly displayed price information, company logos, or logos of retail chains, such as gas stations. Text recognition and extraction can be performed, for example, as in the aforementioned US Pat. No. 7,333,676 B2, without limiting the inventive method to this. Alternatively or additionally, the extracted information includes pattern recognition features.
[0023] The data records in the database are advantageously made available for output via a wireless interface. A user of the corresponding data service can thus access the information from any location, for example, while traveling in a vehicle that may or may not be part of the fleet used for image data collection. Alternatively, or additionally, the information can also be provided wirelessly, allowing, for example, a driver of a vehicle to download advance information from the relevant data service via an internet portal on their home desktop computer before starting a journey.
[0024] In a development of the method according to the invention, the database is assigned to a price information service for fuels which is received in a vehicle via a radio interface. The current position of the vehicle is determined and price information for petrol stations in an area around the current position of the vehicle is displayed in the vehicle if the fuel reserve in the vehicle has fallen below a threshold value. The driver of the vehicle in question can also be given additional information, e.g. whether the petrol station is currently open and, if so, whether it is busy. In particular, the driver can be offered an alternative route to a route stored in their navigation system if this makes it possible to reach a cheaper petrol station. If the navigation destination is the same as the existing .If the fuel reserve is still safely reached, the driver can also be given an indication as to whether, based on the temporal analysis of the price information, refuelling will probably be cheaper at a later time, e.g. the following day.
[0025] The invention further relates to a vehicle comprising an image capture device for recording image data and for capturing the image data together with the associated location and time of recording. According to the invention, the vehicle is characterized by an analysis device by means of which information of a specific category can be extracted from the image data, and by an interface coupled to the analysis device for transmitting a data set containing the extracted information and the associated location and time to a database.
[0026] The invention further relates to an information system comprising a plurality of such vehicles and a device for providing an electronic service, in whose database at least a subset of the data records transmitted by the vehicles is stored and can be made available for output via a user interface. The information system according to the invention is particularly suitable for implementing the method according to the invention. It thus also has the advantages of the method according to the invention.
[0027] The invention will now be explained in more detail using exemplary embodiments with reference to the figures. Fig. 1 shows schematically the structure of an embodiment of a vehicle device of a vehicle according to the invention, Fig. 2 schematically shows a plurality of vehicles and a device for providing an electronic service according to an embodiment of the information system according to the invention, Fig. 3 shows the output contents of a navigation program in a vehicle according to the invention in a situation in which the conditions for the automatic recording of image data according to an embodiment of the method according to the invention are met, and Fig. 4 shows the output contents of a navigation program in a vehicle with the display of price information according to an embodiment of the method according to the invention.
[0028] In Fig. 1 schematically shows the structure of an exemplary embodiment of a vehicle device arranged in a vehicle 10 according to the invention. The vehicle device comprises a camera system 11 for recording image data. This can be, for example, but is not limited to, a cost-effective, easy-to-install camera, e.g., a CMOS camera that generates digital image data in an integrated circuit based on CMOS (Complementary Metal Oxide Semiconductor) technology. The camera system 11 is connected to an analysis device 12 in which the digital image data can be analyzed. The analysis device 12 is connected for data purposes to the data bus 18 in the vehicle. Filter criteria are stored in the analysis device 12 or can be received via the data bus 18, according to which filter criteria can be used to extract information on specific categories from the image data.In particular, lettering and numbers can be extracted from the image data for evaluating openly displayed price information. The captured image data can be conveniently buffered in a ring buffer. The buffered data can then be analyzed continuously or event-driven, and the data in the ring buffer can be overwritten after the analysis is complete and the desired information has been extracted.
[0029] Furthermore, a navigation system 13 is connected to the data bus 18. The navigation system 13 receives the position data of the vehicle 10 via an integrated satellite receiver 14, for example a GPS receiver (GPS = Global Positioning System). The navigation system 13 receives time information either via the satellite receiver 14 or via a built-in system clock 15. The navigation system 13 stores the geographical positions of objects 42 of specific categories, which can be displayed on a geographical map 40 of the navigation system 13, which can be displayed on a display surface 19.
[0030] The extracted information from the image data, along with the associated location and time information, can be transmitted as data sets via a mobile radio device 16 immediately after the analysis is completed. The mobile radio device 16 is typically a mobile radio terminal integrated into the vehicle 10 or detachably connected, e.g., a microcomputer, cell phone, or PDA (Personal Digital Assistant), which operates according to a common mobile radio standard, such as GPRS or UMTS, and is connected to the data bus 18.
[0031] Furthermore, a fuel reserve measuring device 17, via which the fill level of the fuel reserve can be called up in a manner known per se, is connected to the data bus 18.
[0032] An embodiment of the information system according to the invention is shown schematically in Fig. 2. A plurality of vehicles 10 according to the invention travel along a road network, wherein a plurality of locations can be reached in any desired pattern at regular or irregular time intervals. The vehicles 10 are connected via radio links 33 to base stations 32 of a public mobile radio network 30, so that the data records acquired in the vehicles 10 can be transmitted to a database 21 of a data service portal 20, which is connected to the mobile radio network 30 via a data switching center 31. The mobile radio network 30 typically comprises a plurality of transmission and control devices such as radio network control units (e.g., RNC = Radio Network Controller) and mobile radio switching centers (MSC = Mobile Switching Center), which are not explicitly shown here for the sake of simplicity.The data exchange point 31 is, for example, a so-called gateway that regulates data traffic between the transmission and switching network of the mobile network 30 and so-called backend servers and data service portals 20. The data sets delivered by the vehicles 10 can be further processed in the data service portal 20, for example, by comparing data sets with each other and examining them for possible data acquisition or data transmission errors through a consistency check.
[0033] The processed data sets can be made available to a data service user for retrieval. For example, a user receives a user name and password with which they must authenticate themselves and can use them to log into the data service in a conventional manner. In particular, the driver of a vehicle 10 can retrieve this information via the radio connection 33. Use of the data service is not limited to those vehicles 10 that have sent or can send data sets from extracted image data to the data service portal 20. Other users who have appropriate communication devices can also use the data service. For example, vehicles without appropriate image capture devices can still retrieve the data sets captured and transmitted by other vehicles 10 from the data service portal 20 via a radio connection 33.In addition, the data records can be retrieved, for example, from a desktop computer 35 via a landline connection 34, as may be practical for a driver before starting a journey.
[0034] The method according to the invention will now be described using an embodiment with reference to the Fig. 3 and Fig. 4. With reference to the Fig. 3 discusses the conditions for the acquisition of image data and their preparation for the data service, while with reference to the Fig. 4 the output contents when retrieving the data service are explained.
[0035] One of the vehicles 10 according to the invention, as shown in Fig. 1 and Fig. 2, travels along a route on the road network, wherein image data can be continuously recorded by the camera system 11 arranged in the vehicle 10. In order to conserve system resources, the camera system 11 can be provided to normally operate in a so-called "sleep mode," in which a large part of the camera system 11 and the connected analysis device 12 are switched off. The current position 41 of the vehicle 10 is also continuously determined by the navigation system 13 and displayed on a geographical map 40. The navigation system 13 also stores the geographical positions of a large number of objects 42, which can be assigned to specific categories. One of these categories consists of gas stations. Other categories can relate, for example, to busy intersections.
[0036] The following section examines the category “gas stations” as an example. Fig. 3 If the vehicle 10 arrives at a gas station 42, for example, if the navigation system 13 detects that it is about to drive past the gas station 42 based on the entered route (see direction arrow 44), the camera system 11 and the analysis device 12 are started up from sleep mode and the image data recording begins automatically. During the drive-by, the camera system 11 can now collect image data from the gas station 42. The analysis device 12 analyzes the image data for the characteristics of fuel types and prices according to the notice board, queue length at the individual pumps, gas station chain, and opening hours. An analysis of the image data shows, for example, that diesel costs 1.19 euros / liter, gasoline 1.34 euros / liter, and premium 1.39 euros / liter.To save transmission resources, this price information is extracted, transmitting only the cent amounts and the initial letter of the fuel type, so that the information is reduced to D=19; B=34; S=39 and further compressed using conventional compression methods. The current position and the time of the recording, which can be determined from the navigation system 13, are added to this information, and the entire data set is transmitted via the radio connection 33, for example, in XML format (XML = Extension Mark-Up Language), to the database 21 of the data service portal 20. Finally, the camera system 11 and the analysis device 12 are switched back to sleep mode.
[0037] Another vehicle is equipped with at least one fuel reserve measuring device 17, a mobile radio device 16 and a navigation device 13 according to Fig. 1. Data from the data service portal 20 can be received via the radio connection 33. Now, at a later time, pass this vehicle along the Fig. 3 described area of the road network, and is located according to Fig.4 is currently at position 41 when its fuel reserve is falling below a predetermined threshold, the current position 41 is determined by means of the navigation system 13 and the price information for petrol stations in an area around the current position 41 is displayed. Petrol stations 42a which were last recorded less than a certain period of time ago, e.g. one hour, are highlighted, for example shown in bold. Last records of other petrol stations 42b, c which were recorded longer ago can be marked accordingly with other visual features. In particular, the price information for the required type of fuel can be displayed using a color code, e.g. particularly inexpensive petrol stations in green or particularly expensive ones in yellow or red. Other visual effects, e.g. flashing of the corresponding display element, can also be used.
[0038] Alternatively or additionally, a notice field 43 can be displayed, in which the price information of the nearby petrol stations 42 is displayed in tabular form. The data service can also be configured by the user so that, for example, only petrol stations of a specific petrol station chain are displayed, or only those stations are displayed preferentially. In addition, the notice field 43 can also indicate whether a price trend can be derived from the temporal development of the data, for example, an increase or decrease in prices on weekends or at the start of the holidays. With the notice "Better to fill up tomorrow," the driver could be shown a corresponding message that enables them to fill up as cost-consciously as possible. List of reference symbols 10 vehicles 11 Camera system 12 Analysis device 13 Navigation system 14 satellite receivers 15 System clock 16 Mobile radio device 17 Fuel reserve measuring device 18 Data bus in the vehicle 19 display area 20 Data Service Portal 21 Database 30 public mobile network 31 Data exchange center 32 radio base stations 33 Radio connection 34 landline connection 35 desktop computers 40 geographical map of a navigation system 41 current position of the vehicle 42a-c object assigned to a specific category 43 Note field 44 Direction arrow
Claims
[1] A method for collecting and providing information in which - image data are recorded at a plurality of locations by a plurality of image capture devices (11), each arranged in a vehicle (10), and the associated location and time of the recording are recorded, - the image data is analyzed and information of a specific category is extracted from the image data, - a data set containing the extracted information and the corresponding location and time is transmitted to a database (21), - the data set is provided for output by the database (21) and - the extracted information includes fuel prices from the price displays at petrol stations, characterized by , that - the timeliness of information associated with a recording is calculated from the time of recording, - the timeliness of the information is output or made available for output by the database (21) and - the extracted information includes information on the length of the queue at the pumps. [2] Method according to claim 1, characterized by , that - the extracted information includes fuel types or petrol station opening hours. [3] Method according to claim 1 or 2, characterized by , that - the current position (41) of the vehicle (10) is determined, - it is determined whether the geographical position of an object (42) assigned to the category is in the vicinity of the determined current position (41) of the vehicle (10), and - the image data are only recorded automatically if the geographical position of an object (42) of the category is in the vicinity of the determined current position (41) of the vehicle (10). [4] Method according to one of the preceding claims, characterized by that the information is extracted from the image data by an analysis device (12) arranged in the vehicle (10). [5] Method according to one of the preceding claims, characterized by that the data records of the database (21) are made available for output via a radio interface (33). [6] Method according to one of the preceding claims, characterized by , that - the database (21) is associated with a price information service for fuels which is received via a radio interface (33) in a vehicle, - the current position (41) of the vehicle is determined and - Price information for petrol stations (42) in an area around the current position (41) of the vehicle is displayed in the vehicle when the fuel reserve in the vehicle has fallen below a threshold value. [7] Vehicle (10) which is designed to carry out the method according to the invention according to one of claims 1 to 6, comprising - an image capture device (11) for recording image data and for capturing the image data together with the associated location and time of recording, characterized by - an analysis device (12) by means of which information of a specific category can be extracted from the image data, and - an interface (33) coupled to the analysis device (12) for transmitting a data set with the extracted information and the associated location and time to a database (21). [8] Information system which is arranged to carry out the method according to the invention according to one of claims 1 to 6, comprising - a plurality of vehicles (10) according to claim 7 and - a device (20) for providing an electronic service, in whose database (21) at least a subset of the data records transmitted by the vehicles (10) is stored and configured for output via a user interface (33; 34).
Citation Information
Patent Citations
Information system for providing gas station information for motor vehicles, determines gas station with reasonable fuel price suitable for refueling based on remaining amount of fuel in vehicle fuel tank
DE102005016006A1
improved navigation device
DE202007018605U1
Method and apparatus for recognizing text in an image sequence of scene imagery
US7333676B2
Vehicle computer system using the Internet for determining fuel station to approach, has transmitter unit sending position data to provider, receiver unit receiving refueling station data, selection device
DE10105175A1
Parking guidance system for navigating a vehicle looking for a parking space to a free parking space
DE102008028550A1
