Provision and transmission of position data of the vehicle's surroundings

Abstract

According to a method for providing position data of the surroundings of a motor vehicle (8), an image sequence (13) of the surroundings is generated by means of an environment sensor system (10). A gesture (14) of a person (15) in the surroundings is detected by means of a computing unit (7) based on the image sequence (13). A point or area (16) in the surroundings is identified by means of the computing unit (7) based on the gesture (14) and position data of the point or area (16) is generated. The position data is provided for transmission to a receiving device (18) by means of a communication interface (9) of the motor vehicle (8).

Description

Technical Field

[0001] The invention relates to a method for providing position data of motor vehicle surroundings, a method for transmitting position data of motor vehicle surroundings, and a corresponding communication system and a motor vehicle. Background Art

[0002] In some cases where a person wants to describe a specific place, it is not feasible or sufficient to describe its location using conventional information such as the street and house number of the place. This may be the case, for example, when describing the exact location of a parking lot or a storage place for an object.

[0003] In this case, the person can determine the exact coordinates of the location, for example, with the aid of a global navigation satellite system GNSS. However, for this purpose, correspondingly equipped equipment is required, and determining and providing the coordinates for transmission to a receiver can be cumbersome.

[0004] Document DE 10 2009 045 040 A1 describes a navigation system which includes a camera to record images and simultaneously determine the position of the navigation system and to associate the position with the image. The image and the associated position can be stored and the image can be used to easily select a navigation destination at a later time.

[0005] Document US 2006 / 0164412 A1 also describes a navigation system for a motor vehicle. The geographical location of the motor vehicle can be determined with the help of a GPS system, and a three-dimensional image of the motor vehicle's surroundings can be detected with the help of an environmental scanner. The detected data can be shared with other motor vehicles, which can use the data for navigation.

[0006] However, a disadvantage of the known systems is that the position of the navigation system or of the motor vehicle may also be too imprecise to define the position of the location. Summary of the invention

[0007] Against this background, the object of the present invention is to specify an improved concept for providing position data of the surroundings of a motor vehicle, with which the position of a location can be provided with greater accuracy.

[0008] This improved concept is based on the idea of ​​detecting a person's gesture with the aid of an ambient sensor system of the motor vehicle and providing position data based on the detected gesture.

[0009] According to a first independent aspect of the development, a method for providing position data of the surroundings of a motor vehicle is described, in particular a method that is at least partially implemented by a computer. An image sequence of the surroundings is generated by means of an environmental sensor system of the motor vehicle. A gesture of a person in the surroundings is recognized based on the image sequence by means of a computing unit of the motor vehicle. A point or an area in the surroundings is identified based on the gesture by means of the computing unit, and position data of the point or area is generated. The position data is provided via a communication interface of the motor vehicle for transmission to a receiving device.

[0010] An image sequence of an environment is a sequence of images that reflect the environment, in particular a plurality of, i.e. at least two, images of the environment that follow one another in time. The images of the environment can correspond, for example, to camera images, so that the image sequence can also be referred to as a video sequence. The images can also correspond to a two-dimensional or three-dimensional point cloud of sample points in the environment.

[0011] Accordingly, the surroundings sensor system comprises a sensor system suitable for time-resolved imaging of the surroundings, for example a camera system, for example a top-view camera, or a sensor system for two-dimensional or three-dimensional scanning of the surroundings, for example a radar system or a lidar system.

[0012] A person's gesture may be understood as a characteristic, predefined movement of a person or a sequence of characteristic, predefined movements of a person. For example, a predefined movement of a person's body part, such as a person's arm, hand, leg or head movement, may be included in a gesture. The gesture may also include a person walking.

[0013] In order to recognize a gesture based on an image sequence, the computing unit in particular analyzes the images of the image sequence that are sequential in time and identifies positional deviations of a person or a body part of the person in different images as a movement of the person. The movement of the person or the body part is compared with previously stored reference data. If the computing unit recognizes that the reference data corresponds to the recognized movement with a predetermined accuracy, the gesture can be considered recognized. In particular, reference data associated with one or more predetermined reference gestures can be stored in a storage unit or computing unit of the motor vehicle. The computing unit then recognizes in particular that the gesture of the person at least approximately corresponds to one of the reference gestures in order to recognize the gesture.

[0014] The identification of a point or region in the environment by the computing unit may, for example, include the identification of a corresponding image point or image region in one or more images of the image sequence. Depending on the design of the environment sensor system, the image points or image regions may be designed differently. In the case of a camera system, a point or region of the surrounding environment may, for example, correspond to an image point or image region of a corresponding camera image, i.e., a continuous set of a plurality of image points. In the case of a lidar system or a radar system, a point or region of the surrounding environment may correspond to a subset of a point cloud, i.e., a corresponding number of sampling points.

[0015] Identifying a point or region may also include, for example, determining the corresponding coordinates of the point or region in a sensor coordinate system of the surroundings sensor system. The sensor coordinate system corresponds in particular to a coordinate system that is rigidly connected to the surroundings sensor system, in particular to the motor vehicle.

[0016] The position data may contain in particular the respective position, in particular the relative position, of the point or region within the image of the image sequence and / or the position of the point or region relative to a reference coordinate system, for example a world coordinate system such as WGS84 or a map coordinate system of a digital motor vehicle map.

[0017] Providing the position data for transmission to the receiving device comprises, in particular, generating a transmission signal to be transmitted based on the position data, or providing the position data in order to be able to generate a transmission signal.

[0018] By determining position data based on the person's posture, it is possible in particular to identify points or areas of the surroundings, regardless of whether they are recorded in a map or in another database. Points not recorded in the map may, for example, be located on private property or be associated with deviations from the positions stored in the map, for example if the map is not current or outdated. Such points can be detected with high accuracy according to the improved concept and can be transmitted and used accordingly.

[0019] By evaluating the gesture, a high degree of intuitiveness for the user, in particular a person or a recipient, is also achieved. In particular, the recipient of the position data does not need a system for position determination in order to be able to find a point or area of ​​the surroundings.

[0020] According to at least one embodiment, the motor vehicle is designed as an autonomous motor vehicle, in particular according to level 5 of the SAE J3016 classification as of June 2018.

[0021] In such an embodiment, the motor vehicle can, for example, automatically follow a person in the surroundings in order to keep the person within the field of view of the surroundings sensor system.

[0022] According to at least one embodiment of the method for providing position data, the position of the motor vehicle is determined by means of a computing unit in a predefined reference coordinate system. The location coordinates of the point or region in the reference coordinate system are determined based on the position of the motor vehicle, wherein the position data contain the location coordinates of the point or region.

[0023] In particular, the location coordinates of the point or area in the reference coordinate system may be determined based on the position of the motor vehicle in the reference coordinate system, the position of the point or area in the sensor coordinate system and the position of the sensor coordinate system relative to the reference coordinate system.

[0024] The reference coordinate system may in particular be a reference coordinate system of a global navigation satellite system, for example a GPS system, for example a geodetic reference system, for example the WGS84 coordinate system.

[0025] In such an embodiment, the position data contain in particular objective position information, so that a recipient or potential recipient can find a point or area without having to know its approximate location.

[0026] According to at least one embodiment, satellite signals, in particular of a global navigation satellite system, are detected by means of a satellite receiving unit of the motor vehicle and the position of the motor vehicle is determined from the satellite signals by means of a computing unit.

[0027] According to at least one embodiment, the position of the motor vehicle is determined by means of a computing unit based on the image sequence and map data of a digital map of the motor vehicle.

[0028] In particular, based on the image sequence or on individual images of the image sequence, the computing unit can detect one or more landmarks in the surroundings of the motor vehicle and compare the landmarks with data stored on the digital map in order to determine the position of the motor vehicle.

[0029] Such an embodiment may be particularly advantageous if the surroundings sensor system includes a radar system or a lidar system in order to detect particularly precise three-dimensional or two-dimensional sampling points of the surroundings.

[0030] A digital map is in particular a database in which position data of lanes and objects are stored with an accuracy in the range of one centimeter or a few centimeters. Such a digital map may also be referred to as an HD map. In particular, partially or fully autonomous motor vehicles use such digital maps for self-localization. This allows particularly high accuracy to be achieved.

[0031] According to at least one embodiment, environmental data of the surroundings are determined by means of the environmental sensor system or by means of another environmental sensor system of the motor vehicle, and the position of the motor vehicle is determined by means of a computing unit based on the environmental data and map data of a digital map of the motor vehicle.

[0032] In contrast to the previously described specific embodiments, separately generated surroundings data are therefore used here instead of an image sequence in order to determine the position of the motor vehicle.

[0033] According to at least one embodiment, processed image data are generated based on the image sequence by means of the processing unit, in which the point or the area is visually highlighted, wherein the position data contain the processed image data.

[0034] The processed image data correspond, for example, to one or more camera images or another representation of the surroundings, in which points or areas are marked or highlighted.

[0035] In such an embodiment, for example, the exact position of the motor vehicle is not required, which can be advantageous, in particular if the position of the motor vehicle could not be determined precisely when the image sequence was recorded, for example because the motor vehicle was located in a building and could not receive GNSS signals. By providing the processed image data and providing them, for example, to a potential recipient, the recipient can recognize the position of a point or area based on relevant content highlighted in the camera image or presentation, in particular if the approximate position of the point or area is known to the recipient.

[0036] According to at least one embodiment, the environment and the representation of points or areas in the environment are visually displayed based on the image sequence by means of a user interface of the motor vehicle. Correction instructions from a user of the motor vehicle are detected by means of the user interface. Position data are generated by means of a computing unit as a function of the correction instructions.

[0037] The representation of the surroundings and the point or area can correspond, for example, to an image or one of the images of an image sequence or to another image of the surroundings, for example a satellite image.

[0038] This allows the user to check whether the position of a point or area has been detected as expected and to make corrections if necessary.

[0039] According to another independent aspect of the development, a method for transmitting position data of the surroundings of a motor vehicle is described. According to the method for providing position data, the position data are provided according to the development. The position data are transmitted wirelessly to a receiving device by means of a communication interface.

[0040] The receiving device may include, for example, a receiving device of another motor vehicle, for example another communication interface of another motor vehicle. The receiving device may also be part of a server computer or other fixed or mobile computer system, for example part of a mobile electronic device, such as a smartphone or a tablet computer.

[0041] Wireless transmission can be performed, for example, by means of WLAN or other radio connections, for example mobile radio connections according to a predefined mobile radio standard. In particular, the communication interface can be designed as a V2V (vehicle to vehicle) and / or V2X (vehicle to X) interface.

[0042] In order to transmit the position data, the communication interface generates, in particular, a transmission signal and sends it out.

[0043] According to another independent aspect of the improved scheme, a communication system is described. The communication system has a computing unit for a motor vehicle, a communication interface for the motor vehicle, and an environment sensor system for the motor vehicle. The environment sensor system is configured to generate an image sequence of the surroundings of the motor vehicle. The computing unit is configured to recognize a gesture of a person in the surroundings based on the image sequence, and to identify a point or area in the surroundings based on the gesture, and to generate position data of the point or area. The communication interface is configured to provide the position data for transmission to a receiving device.

[0044] Further embodiments of the communication system according to the improvement result directly from the various embodiments of the method according to the improvement, and vice versa. In particular, the communication system according to the improvement can be programmed or set to carry out the method according to the improvement for providing position data and / or the method for transmitting position data, or the communication system according to the improvement carries out such a method according to the improvement.

[0045] According to a further independent aspect of the development, a motor vehicle is specified having a communication system according to the development.

[0046] According to a further independent aspect of the improved scheme, a computer program having instructions is described, wherein, when the computer program is executed by a communication system according to the improved scheme, in particular by a computing unit of the communication system, the instructions cause the communication system to execute a method for providing position data and / or a method for transmitting position data according to the improved scheme.

[0047] According to a further independent aspect of the development, a computer-readable storage medium is specified, which stores a computer program according to the development.

[0048] The present invention also includes an embodiment of the communication system according to the present invention, which has its features already described in conjunction with the embodiment of the method according to the present invention. For this reason, the corresponding embodiment of the communication system according to the present invention is not explicitly described again here.

[0049] The invention also includes combinations of features of the described embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Embodiments of the present invention are described below.

[0051] in:

[0052] Figure 1 A motor vehicle is shown, wherein a schematic diagram

[0053] An exemplary implementation of a communication system according to an improved solution; and

[0054] Figure 2 A flow chart of an exemplary embodiment of a method for transmitting position data according to a further development is shown. DETAILED DESCRIPTION

[0055] The embodiments described below are preferred embodiments of the present invention. In these embodiments, the components described in the embodiments are respectively individual features of the present invention that can be regarded as independent of one another, which also expand the present invention independently of one another and can therefore also be regarded as part of the present invention individually or in a combination different from that shown. In addition, the described embodiments can also be supplemented by other features of the present invention that have already been described.

[0056] In the figures, functionally identical elements are respectively provided with the same reference symbols.

[0057] Figure 1 2 shows a motor vehicle 8 which has a communication system 12 according to a development.

[0058] The communication system 12 comprises a computing unit 7 and an environmental sensor system connected to the computing unit 7, which has, for example, a camera 10. A person 15 and a point or area 16 are located in the field of view 21 of the camera 10. In an alternative embodiment, the environmental sensor system can have a lidar system in addition to or as an alternative to the camera 10. The communication system 12 also has a communication interface 9 coupled to the computing unit 7.

[0059] Optionally, communication system 12 or motor vehicle 8 may have a user interface 20 which is coupled to computer unit 7 and contains, for example, a display or another visual display device.

[0060] Optionally, motor vehicle 8 or communication system 12 may include a satellite reception unit 11 , for example a GPS receiver, which is likewise coupled to computer unit 7 .

[0061] exist Figure 1 Also shown is a receiving device 18, such as a smartphone, which Figure 1 In the non-limiting example, the receiver 17 carries the transmission signal 19 which is emitted by the communication interface 9 and can be received by the receiver 18.

[0062] Reference below Figure 2 The operating principle of the communication system 12 according to the development is explained with reference to an exemplary embodiment of the method for providing and transmitting position data according to the development.

[0063] In step 1 of the method, camera 10 records a sequence of images of the surroundings of motor vehicle 8 , which image sequence images field of view 21 .

[0064] In step 2a of the method, the computing unit 7 recognizes from the image sequence that the person 15 performs an arm movement corresponding to the predefined gesture 14, in that the person 15 points with his arm, in particular, at a point or area 16. Based on the gesture, the computing unit 7 identifies the position of the point or area 16 within the image of the image sequence 13 and / or in the coordinates of the corresponding sensor coordinate system.

[0065] As an alternative to step 2a, gesture 14 may correspond to a movement of user 15 in the surroundings of motor vehicle 8. In step 2b, computing unit 7 recognizes the movement of person 15 as a predefined gesture and identifies area 16 based on the path traversed by person 15.

[0066] In step 3 , computer unit 7 can determine the position of motor vehicle 8 in a reference coordinate system, for example based on satellite data, using satellite reception unit 11 or based on map data from a digital map of motor vehicle 8 .

[0067] Alternatively or additionally, in step 3 computing unit 7 can process image sequence 13 , in particular one or more images of image sequence 13 , by generating an image showing the surroundings of motor vehicle 8 , in which points or areas 16 are graphically or visually highlighted.

[0068] In optional step 4 of the method, the processed image data are visually displayed, in particular on a display or screen, by means of a user interface 20. The user of the motor vehicle 8 can make corrections by means of the user interface 20 in order to correct the position or definition of a point or area in the surroundings.

[0069] In step 5 , the communication interface 9 generates a transmission signal 19 based on the position data and, if appropriate, corrections, and sends it to the receiving device 18 .

[0070] In step 6 of the method, the receiving device 18 receives the transmission signal 19 and can determine the processed image data and / or the position of the point or area 16 in the form of coordinates in a reference coordinate system based on the transmission signal 19 .

[0071] According to various embodiments of the development, a person can describe a location or an area with the aid of sensors located in the motor vehicle, for example with the aid of a top-view camera. For this purpose, the motor vehicle can be designed in various embodiments as an autonomous motor vehicle and can follow the user of the system, for example, within a limited area, in particular by switching between sensors or with the aid of a swiveling camera, other orientable sensors, or also by driving behind. The person "points" the motor vehicle to a certain area or point in the surroundings, the position of which is to be transmitted to another person or other recipient. The position can be transmitted, for example, in the form of coordinates, which are determined by the motor vehicle and can be entered, for example, into a digital map, or in the form of image data, which are provided with additional information, in particular position information about the point or area.

[0072] The recipient can use this information in a variety of ways, for example in order to park another motor vehicle correctly at the location specified for this purpose, or, for example, to store an object.

[0073] Alternatively or in addition to the position, for example in the absence of the possibility of accurately determining the position of the motor vehicle, the image data or sensor data can be transmitted together with the visually highlighted position of the point or area.

[0074] Three exemplary application scenarios of the improved solution are outlined below. In the first application scenario, user A wants to indicate available parking spaces on his property to user B. To this end, user A indicates to his motor vehicle the area in which user B should park. Accordingly, user A can also book a rental vehicle to be parked at the transmitted location.

[0075] In a second application example, a user wants to deliver a large object and indicates to his vehicle where to leave it. This location is communicated to the supplier of the object, for example. This approach can also be used to unload building materials, for example.

[0076] In a third embodiment, the user wants to indicate in which aisles other vehicles should move without other markings over a larger area. Through corresponding gestures and movements, he can mark areas that can, for example, be entered into a digital map and transmitted to other users.

[0077] Reference numerals list:

[0078] 1 Method steps

[0079] 2a Method steps

[0080] 2b Method Steps

[0081] 3 Method steps

[0082] 4 Method steps

[0083] 5 Method steps

[0084] 6 Method steps

[0085] 7 Computing Units

[0086] 8 Motor Vehicles

[0087] 9 Communication interface

[0088] 10 cameras

[0089] 11 Satellite receiving unit

[0090] 12Communication System

[0091] 13 Image Sequence

[0092] 14 Postures

[0093] 15 people

[0094] 16 points or areas

[0095] 17 Receiving Personnel

[0096] 18 Receiving equipment

[0097] 19 Transmission Signal

[0098] 20 User Interface

[0099] 21 Field of view.

Claims

1. A method for transmitting position data of the surroundings of a motor vehicle (8), in, generating an image sequence (13) of the surroundings by means of an environmental sensor system (10) of the motor vehicle (8); Recognizing a gesture (14) of a person (15) in the surroundings based on the image sequence (13) by means of a computing unit (7) of the motor vehicle (8); Using the computing unit (7) to identify a point or region (16) in the surroundings based on the gesture (14), and to generate position data of the point or region (16); providing the position data for transmission to a receiving device (18) by means of a communication interface (9) of the motor vehicle (8), wherein providing the position data for transmission to the receiving device (18) comprises generating a transmission signal to be transmitted based on the position data; The position data are wirelessly transmitted to the receiving device (18) by means of the communication interface (9) by emitting the transmission signal, wherein the receiving device (18) comprises a receiving device of another motor vehicle or is part of a server computer or part of a mobile electronic device, and generating, by means of the computing unit (7), processed image data based on the image sequence (13), in which the point or area (16) is visually highlighted, wherein the position data contains the processed image data; and / or Visually displaying the surroundings and a representation of a point or area (16) in the surroundings based on the image sequence (13) by means of a user interface (20) of the motor vehicle (8); The correction instruction is detected by the user via the user interface (20); and The position data are generated by means of the computing unit (7) as a function of the correction command.

2. The method according to claim 1, It is characterized in that By means of the computing unit (7), - determining the position of the motor vehicle (8) in a predetermined reference coordinate system; - determining the location coordinates of the point or area (16) in the reference coordinate system based on the position of the motor vehicle (8), wherein the position data contain the location coordinates of the point or area (16).

3. The method according to claim 2, Features: detecting satellite signals by means of a satellite receiving unit (11) of the motor vehicle (8); and The position of the motor vehicle (8) is determined using the computing unit (7) on the basis of the satellite signals.

4. The method according to claim 2, It is characterized in that The position of the motor vehicle (8) is determined by means of the computing unit (7) based on the image sequence (13) and based on map data of a digital map of the motor vehicle (8).

5. The method according to claim 2, Features: determining environmental data of the surroundings by means of the environmental sensor system (10) or by means of another environmental sensor system of the motor vehicle (8), wherein the environmental data are generated separately from the image sequence (13); and The position of the motor vehicle (8) is determined by means of the computing unit (7) based on the surroundings data and based on map data of a digital map of the motor vehicle (8).

6. The method according to any one of claims 1 to 5, It is characterized in that The gesture (14) corresponds to a movement of the user (15) in the surroundings of the motor vehicle (8); and The computing unit (7) recognizes the movement of the person (15) as the gesture (14) and identifies an area (16) based on the path traversed by the person (15).

7. A communication system comprising a computing unit (7) for a motor vehicle (8), a communication interface (9) for the motor vehicle (8) and an environmental sensor system (10) for the motor vehicle (8), in, The environmental sensor system (10) is configured to generate an image sequence (13) of the surroundings of the motor vehicle (8); The calculation unit (7) is configured to - recognizing a gesture (14) of a person (15) in the surroundings based on the image sequence (13); and - identifying a point or area (16) in the surrounding environment based on the gesture (14) and generating position data of the point or area (16); The communication interface (9) is configured to provide the position data for transmission to a receiving device (18), wherein providing the position data for transmission to the receiving device (18) comprises generating a transmission signal to be transmitted based on the position data, and The communication interface (9) is configured to wirelessly transmit the position data to the receiving device (18) by emitting the transmission signal, wherein the receiving device (18) comprises a receiving device of another motor vehicle or is part of a server computer or is part of a mobile electronic device, and The computing unit (7) is configured to generate processed image data based on the image sequence (13), in which the point or area (16) is visually highlighted, wherein the position data comprises the processed image data; and / or The communication system (12) has a user interface (20) for the motor vehicle (8), which is configured to visually display the surroundings and a representation of a point or area (16) in the surroundings based on the image sequence (13) and to detect correction instructions by a user; and The calculation unit (7) is configured to generate the position data according to the correction instruction.

8. A motor vehicle having a communication system (12) according to claim 7.

Images