Method and system for determining the position and orientation of a virtual camera of a surrounding observation system - Patents.com

Abstract

The present invention relates to a method for determining a virtual camera position and a camera orientation in order to be able to display the surroundings of a vehicle (1) on a display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation.

Description

[Technical field]

[0001] The present invention relates to the field of vehicle surround observation systems, in particular to a method and a system for determining the position and orientation of a virtual camera of a surround observation system. [Background technology]

[0002] Surrounding observation systems for vehicles are known in principle. In particular, so-called 3D surround view systems are known, which allow a 360° surround view around the vehicle in three dimensions and perspective. In such systems, the image information of several cameras is combined with a surround view representation and mapped onto a three-dimensional surface in order to obtain a spatial and perspective representation. The spatial and perspective representation of the surroundings of the vehicle is performed in known systems from different viewpoints. In this way, the position and orientation of the virtual camera, i.e. the position and the viewing direction from which the virtual observer is observing the scene, can be changed so that different areas of the surroundings of the vehicle can be observed. Particularly preferably, the position and the orientation of the virtual camera can be selected such that, for example, not only the surrounding area but also the vehicle in the surrounding area can be recognized.

[0003] In the conventional surrounding observation system, the position and orientation of the virtual camera must be selected by the user according to the respective situation so that the important area can be displayed on the display unit of the vehicle, which makes the vehicle user unable to concentrate on driving the vehicle, which is a disadvantage, and may even cause an accident. Summary of the Invention [Problem to be solved by the invention]

[0004] Starting from this, the object of the invention is to provide a method which allows automatic determination of the position and orientation of a virtual camera of a surrounding observation system. [Means for solving the problem]

[0005] This object is achieved by a method having the features of the independent claim 1. Preferred embodiments are the subject of the dependent claims. A system for determining the position and orientation of a virtual camera of a surroundings observation system is the subject of a parallel independent claim 9.

[0006] According to a first aspect, a method is disclosed for determining a position and an orientation of a virtual camera of a surroundings observation system. Based on this virtual camera position and camera orientation, surroundings of the vehicle are depicted on a display unit of the vehicle. In other words, determining the position and orientation of the virtual camera defines a viewpoint from which the surroundings are observed. Here, the method generally comprises the following steps:

[0007] First, the surroundings of the vehicle are captured by multiple cameras on the vehicle. photograph Based on the information from these cameras, the current The vehicle surroundings information is created. Particularly preferably, the image information provided by these cameras is the current They can be linked together to enable a 360° surround view around the vehicle.

[0008] Additionally, a memory unit is provided in which a number of data sets can be stored, each of which contains vehicle surroundings information captured by a number of cameras of the vehicle and stored. Additionally, the data sets also contain information on the virtual camera positions and camera orientations, respectively, such that for each data set an attribution of the stored vehicle surroundings information to the virtual camera positions and camera orientations can be performed. The data sets are preferably created by storing a virtual camera position and a camera orientation selected by the driver during the driving situation together with the vehicle surroundings information.

[0009] Then, at that point photographThe vehicle surroundings information thus obtained is compared with at least one data set of vehicle surroundings information stored in a memory unit.

[0010] At least one similarity parameter for the at least one data set is then determined. The similarity parameter may be: photograph was done the current The vehicle surroundings information is given a degree of similarity to the vehicle surroundings information stored in each of the data sets.

[0011] Finally, the similarity parameter is compared with a threshold value. If the similarity parameter is greater than a predefined threshold value, the virtual camera position and camera orientation information of the stored data set containing the vehicle surroundings are retrieved. In addition, the virtual camera position and camera orientation are determined based on the retrieved virtual camera position and camera orientation information, i.e. the surroundings are depicted from a viewpoint determined by the retrieved virtual camera position and camera orientation information.

[0012] The method comprises: The surrounding observation system In the past, virtual camera positions and virtual· Camera orientation Already It was decided Automatically selects the virtual camera position and orientation for the surroundings of the vehicle A depiction of the vehicle's surroundings is displayed on the vehicle's display unit. Generate This has the technical advantage that if the driver wants to reuse a previously determined virtual camera position and camera orientation, no intervention by the driver in the surround observation system is required. This means that the driver is no longer distracted by the operation of the surround observation system and can concentrate more on driving maneuvers, improving the user-friendliness of the driver assistance system.

[0013] According to one embodiment, the currentThe task of comparing the vehicle surroundings information with the stored vehicle surroundings information is the current It also involves comparing image similarities and image features between the current vehicle surroundings and the stored vehicle surroundings. In this way, image processing can be used to determine whether a vehicle is present or not in a position where information about the virtual camera position and virtual camera orientation already exists.

[0014] According to one embodiment, a number of similarity parameters are determined between a number of different data sets, where: the current The largest similarity parameter, which corresponds to the highest match between the vehicle surroundings information of the vehicle surroundings and the stored vehicle surroundings information, is then determined. This maximum similarity parameter is then used for comparison with the threshold. To determine the virtual camera position and virtual camera orientation, information on the virtual camera position and virtual camera orientation of the dataset containing the stored vehicle surroundings information for which the maximum similarity parameter was obtained is retrieved. In this way, vehicle surroundings information of multiple datasets can be examined, and the vehicle surroundings information stored therein is compared to determine the virtual camera position and virtual camera orientation. the current The vehicle surroundings information is most consistent with or It becomes possible to use similar data sets.

[0015] According to an embodiment, the data sets each have neighborhood information associated with the location. the current By comparing the vehicle surroundings information of the vehicle with the stored vehicle surroundings information and comparing the similarity parameter with a threshold, it is determined whether the vehicle is already in a vehicle position for which a virtual camera position and a virtual camera orientation have been determined. photograph, and by comparing with known surrounding information, it is determined whether there is a match with the known vehicle surroundings for which the observation viewpoint was previously determined.

[0016] According to one embodiment, the vehicle includes a sensor-based location system, e.g., a GPS-based location system, and the data set includes position data provided by the location system in the past, the position data being stored in accordance with each stored vehicle surroundings information. photograph The position data may be, for example, geographic coordinates in a global coordinate system. This provides a geographic location of the location detection system. the current Obtain location information of the the current By comparing the location information of the dataset with the position data of the dataset, the relevance of each dataset is calculated. the current The estimation can be performed without the need to compare the vehicle surroundings information of the vehicle with the stored vehicle surroundings information.

[0017] According to one embodiment, the location information is determined by a sensor-based position detection system. These location information are compared to the position data stored in the data set. The comparison of the location information with the position data stored in the data set limits the data set, i.e., which data set, or , the vehicle surroundings information contained and stored in each data set for which the similarity parameters should be calculated is limited. The sensor-based position detection system has inaccuracies in its position detection method that make it impossible to perform accurate position detection, or , has unsharpness. Typical inaccuracies or The unsharpness is on the order of a few meters. Thus, although a sensor-based location system cannot provide an accurate location for a vehicle, it can support visual location based on a stored data set. the current A comparison of the vehicle surroundings information with the stored vehicle surroundings information in the area of ​​the current vehicle position is carried out photographIt is possible to limit the comparison to the most relevant datasets, which significantly reduces the computational load required for the comparison.

[0018] According to one embodiment, the vehicle's computing unit creates, from the information provided by the camera, a three-dimensional surroundings information that is displayed as spatial and perspective surroundings information on a display unit of the vehicle based on a virtual camera position and a camera orientation, whereby the virtual camera position and the camera orientation provide the viewpoint from which the surroundings information is observed as well as the sector of the surroundings information that is displayed on the display unit, thereby enabling the surroundings information to be depicted by the surroundings awareness system in a natural and user-friendly way.

[0019] According to one embodiment, the spatial and perspective surroundings information includes a perspective view of the vehicle itself. For example, the surrounding area above the vehicle itself may be depicted in perspective. That is, the perspective surroundings information includes not only the surroundings of the vehicle, but also the vehicle's attitude, or It can also sense when the vehicle is embedded in its surroundings, which allows for improved driver support from the surrounding awareness system, as the driver can be aware of the vehicle's position relative to its surroundings.

[0020] According to a further aspect, the invention relates to a system for determining the position and orientation of a virtual camera of a surroundings observation system, comprising a number of cameras distributed around a vehicle, a computing unit for processing information provided by said cameras, a memory unit and a display unit arranged to be able to display information about the surroundings of the vehicle, said computing unit comprising the following steps: 、 - Multiple onboard cameras capture the surroundings of the vehicle photograph Based on the information from these cameras, the current A step of creating vehicle surrounding information; - storing the plurality of data sets in a memory unit; AndThe datasets are each captured by multiple on-board cameras. photograph The vehicle surroundings information includes the stored vehicle surroundings information and the virtual camera position and camera orientation information assigned to the stored vehicle surroundings information. , Step ; - the current comparing the vehicle surroundings information of the vehicle surroundings information with stored vehicle surroundings information of at least one data set; - determining at least one similarity parameter for at least one data set; And , the similarity parameters are respectively the current The degree of similarity between each of the stored data sets of vehicle surrounding information and the stored vehicle surrounding information is shown. , Step ; - if the similarity parameter is greater than a predetermined threshold, retrieving information on the virtual camera position and the camera orientation of the stored data set comprising the vehicle surroundings information and determining the virtual camera position and the camera orientation based on the retrieved information on the virtual camera position and the camera orientation; It is configured to be able to carry out the following.

[0021] According to one embodiment of the system, the computing unit comprises: the current determining image similarity to compare the vehicle surroundings information of the vehicle surroundings to the stored vehicle surroundings information; the current The system is configured to compare image features between the current vehicle surroundings information and the stored vehicle surroundings information. In this way, it is possible to determine, by image processing, whether a vehicle is present or not in a position for which information about the virtual camera position and virtual camera orientation already exists.

[0022] According to one embodiment of the system, the computing unit comprises a plurality of different Multiple Similarity Parameters for Data Sets but Divide Be , the current Vehicle surroundings information and stored vehicle surroundings information With between of Maximum indicates the best match of Similarity Parameter but decision Be , The maximum similarity parameter is used for the threshold comparison, and maximum of Similarity Parameter Leading The virtual camera position and the vehicle surroundings are stored in a dataset that contains the vehicle surroundings. camera Orientation Information is read out This allows the vehicle surroundings of multiple data sets to be examined and the vehicle surroundings stored in it to determine the virtual camera position and virtual camera orientation. the current The vehicle surroundings information is most consistent with or It becomes possible to use similar data sets.

[0023] According to an embodiment of the system, the vehicle includes a sensor-based location system, the computing unit is adapted to receive respective stored vehicle surroundings information provided by the location system. photograph The vehicle is configured to store position data in the data set that indicates the geographic location of the vehicle. The position data is stored according to the vehicle surroundings information stored in the data set. photograph The position data may be, for example, geographic coordinates in a global coordinate system. This provides a geographic location of the location detection system. the current Based on the location information, the relevance of each data set is calculated. the current The estimation can be performed without the need to compare the vehicle surroundings information of the vehicle with the stored vehicle surroundings information.

[0024] According to an embodiment of the system, the sensor-based location detection system is configured to determine geographic location information, and the computing unit is configured to compare the geographic location information with position data stored in a data set, and to limit the data set to only those for which a similarity parameter is determined by comparing the geographic location information with the position data stored in the data set, respectively. the current This significantly reduces the computational load required to compare the vehicle surroundings information with the stored vehicle surroundings information.

[0025] According to an embodiment of the system, the computing unit is configured to generate three-dimensional surroundings information from information provided by the cameras based on a virtual camera position and a camera orientation for display as spatial and perspective surroundings information on a display unit of the vehicle, such that the computing unit of the vehicle is capable of generating three-dimensional surroundings information from information provided by the cameras based on a virtual camera position and a camera orientation for display as spatial and perspective surroundings information on a display unit of the vehicle, thereby enabling surroundings information to be depicted by the surroundings awareness system in a natural and user-friendly manner.

[0026] According to an embodiment of the system, the computing unit is configured to present a perspective view of the vehicle itself in the spatial and perspective surroundings information, which allows the driver to be aware of the relative position of the vehicle with respect to the surroundings, thus enabling improved driver support by the surrounding awareness system.

[0027] In the present invention, "approximately" and "substantially" mean also Expressions such as "approximately" and "approximately" indicate an error of + / - 10%, preferably + / - 5%, from the exact value, and also is interpreted as an error with no significance to the function.

[0028] Further developments, advantages and scope of application of the invention are shown in the following description with reference to examples and drawings, in which all described and / or / without departing from the spirit and scope of the invention. also The illustrated features may be used individually and in any combination as claimed in the claims. also is the subject of the present invention independently of any combination with the reference thereto. The contents of the claims are also incorporated into the specification.

[0029] The present invention will now be described in more detail with reference to the drawings and examples. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 illustrates a schematic top view of a vehicle equipped with a system for depicting surrounding information on a display unit; [Diagram 2] FIG. 2 illustrates a representation of a hemisphere as a 3D ball onto which peripheral information of the surrounding area of ​​the vehicle is mapped to represent the peripheral information in perspective; [Diagram 3] FIG. 3 illustrates a schematic block diagram of a system for determining a position and orientation of a virtual camera to perspectively depict surrounding information within a surrounding area of ​​a vehicle; and [Figure 4] FIG. 4 is a flow chart illustrating method steps for determining a position and orientation of a virtual camera to perspectively depict surrounding information within a surrounding area of ​​a vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] 1 illustrates diagrammatically a vehicle 1 having a system, hereinafter also called a 3D surround view system, for a three-dimensional representation of the surrounding area of ​​the vehicle 1 on a display unit 2. The display unit 2 is particularly preferably provided inside the vehicle and serves to support the driver during driving maneuvers, such as for example reversing or parking in a parking space.

[0032] Vehicle 1 has a plurality of cameras 3 that are provided outside and dispersed around Vehicle 1. For example, the vehicle 1 has a front camera, a rear camera, and at least one side camera each. The cameras 3 are preferably arranged and configured such that by synthesizing the images provided by each camera 3, a 360° description of the area around the vehicle 1 can be obtained as a single image.

[0033] Vehicle 1 is preferably configured such that three-dimensional peripheral information that is displayed as spatial and perspective peripheral information on the display unit 2 can be generated from the image information provided by at least two cameras 3.

[0034] Figure 2 illustrates a perspective description of a three-dimensional space within the peripheral area of Vehicle 1. Such a description is called, for example, a 3D ball. From the image information provided by the cameras 3, spatial and perspective peripheral information is calculated by the arithmetic unit 5 provided in the vehicle. In the case of this spatial and perspective peripheral information, preferably, Vehicle 1 itself or is recognizable by its outline. Such a type of perspective surround view description, unlike an overhead view (bird's-eye view), enables the peripheral area of Vehicle 1 to be observed perspectively from a specific reference point that certifies the following camera positions, or i.e., the observation point. At this time, the camera position is variable, that is, the virtual position from which Vehicle 1 and its surroundings are observed can be changed (virtual movement within the 3D ball). This enables a realistic description of the peripheral area of Vehicle 1.

[0035] In addition, from the set camera position, the camera orientation, that is, the angle at which Vehicle 1 or observes the peripheral part of Vehicle 1 from that camera position can also be changed.

[0036] The vehicle 1 is configured to automatically determine the camera position and the camera orientation, i.e. the vehicle 1 has a virtual surroundings recognition system configured to determine whether the vehicle 1 is currently in a position where the camera position and the camera orientation have already been defined by the vehicle user. The virtual surroundings recognition system uses the image information provided by the camera 3 and derives from them: the current and compares these with stored vehicle surroundings information in which the vehicle user has already defined the camera position and camera orientation.

[0037] As indicated diagrammatically in FIG. 1, the vehicle includes a computing unit 5 by means of which an automatic determination of the camera position and the camera orientation is performed. FIG. 3 illustrates in a schematic block diagram the architecture of a system for automatically determining the camera position and the camera orientation. The computing unit 5 of the vehicle implements, for example, a virtual surroundings recognition system 7 and a 3D surround view system 8. The surroundings recognition system 7 is connected to the cameras 3 of the vehicle 1 and receives their image information. The image information of the individual cameras 3 is then combined and the current For example, the image information from each camera 3 can be stitched together to complete a 360° surround view.

[0038] The surroundings recognition system 7 is connected to the memory unit 4. The memory unit 4 is configured to store one or preferably several data sets, where one data set contains vehicle surroundings information determined at a certain time point in the past on the basis of image information from the camera 3. Here, the vehicle surroundings information can be particularly preferably also obtained by the surroundings recognition system 7 by combining image information from several cameras 3. In addition, the data set also contains information on virtual camera positions and virtual camera orientations, respectively, that may already be used by the vehicle user in the vehicle surroundings at the current time point. In other words, the memory unit 4 is responsible for storing virtual camera position information and virtual camera orientation information by the user at a certain location, for example a place frequently visited by the vehicle user.

[0039] The Peripheral Recognition System 7 is the current of photograph By comparing the acquired vehicle surroundings information with the vehicle surroundings information stored in a data set stored in the memory unit 4 of the vehicle 1, the vehicle 1 is configured to recognize whether it is already in a position where the virtual camera position and camera orientation have been determined by the vehicle user.

[0040] For more information, the current of photograph The vehicle surrounding information is based on image similarity and also is compared to stored vehicle surroundings information by determining feature match. photograph Based on a comparison between the vehicle surroundings information and the stored vehicle surroundings information, the current of photograph A similarity parameter is determined that gives the probability that the captured vehicle surroundings information was captured at the same vehicle position as the stored vehicle surroundings information. If the similarity parameter exceeds a predetermined threshold, the currentThe vehicle position at the current time is assumed to be the same as that at which the stored vehicle surroundings information was captured. In this case, the information on the virtual camera position and camera orientation attributed to the stored vehicle surroundings information is read from the memory unit 4 and used on that basis to determine the virtual camera position and camera orientation at the current vehicle position.

[0041] The retrieved virtual camera position and camera orientation are particularly preferably based on these the current The information about the surroundings of the vehicle is transmitted to a 3D surround view system 8 which prompts the information to be depicted on the display unit 2 of the vehicle 1.

[0042] According to an embodiment, the vehicle 1 comprises a sensor-based location system 6. This sensor-based location system 6 may be, for example, a satellite. star By means of this position detection system 6 it is possible to determine position information of the vehicle 1, which can show where the vehicle is currently located, particularly preferably coordinates in a global coordinate system.

[0043] This location information can be used to support visual localization based on the surroundings of the image. In particular, the current This can be used to limit the data set that is examined for a match between the vehicle surroundings information and the stored vehicle surroundings information. In this way, the data set can include location information obtained from the location detection system 6 in addition to the stored vehicle surroundings information. the current Before comparing the vehicle surroundings information of the vehicle 1 with the stored vehicle surroundings information, the data set obtained at a position close to the current location of the vehicle 1 is identified, and only the stored vehicle surroundings information of the data set obtained at a position close to the current location of the vehicle 1 is compared. the currentThis makes it possible to compare the vehicle's surroundings with other information.

[0044] Preferably, the data set stored in the memory unit 4 is attributed to a vehicle user profile, so that the position and orientation of the virtual camera can be stored dependent on the vehicle user and can be used when determining the position and orientation of the virtual camera dependent on the current user profile.

[0045] FIG. 4 depicts a block diagram showing method steps for determining the position and orientation of a virtual camera.

[0046] First, the surroundings of the vehicle are captured by multiple cameras on the vehicle. photograph Based on the information from these cameras, the current The vehicle surroundings information is created (S10).

[0047] Further, a memory unit is provided in which a number of data sets can be stored, each of which is captured by a number of cameras of the vehicle. photograph In addition, these datasets contain information about the virtual camera positions and camera orientations to which the stored vehicle surroundings are attributed (S11).

[0048] Next, the current The vehicle surroundings information is compared with at least one stored data set of vehicle surroundings information (S12).

[0049] At least one similarity parameter for the at least one data set is then determined. The similarity parameter may be: the current The degree of similarity of each of the vehicle surroundings information stored in each data set is given to the vehicle surroundings information (S13).

[0050] If the similarity parameter is greater than a predefined threshold, information about the virtual camera position and the camera orientation of the stored data set containing the vehicle surroundings information is retrieved, and a virtual camera position and a camera orientation are determined based on the retrieved information about the virtual camera position and the camera orientation (S14).

[0051] The invention has been described by way of the above examples, however it will be appreciated that numerous modifications and variations are possible without departing from the scope of the invention as defined in the appended claims. This application relates to the invention described in the claims, but also includes the following as other aspects. 1. 1. A method for determining a virtual camera position and a camera orientation so that the surroundings of a vehicle (1) can be displayed on a display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation, the method comprising the steps of: 、 - The vehicle (1) is equipped with multiple cameras (3) to monitor the surroundings of the vehicle. photograph Based on the information from these cameras (3) the current A step of creating vehicle surrounding information (S10); - providing (S11) a memory unit (4) in which a plurality of data sets are stored; And , the data set is respectively obtained by a number of cameras (3) of a vehicle (1). photograph The vehicle surroundings information includes the stored vehicle surroundings information and the virtual camera position and camera orientation information assigned to the stored vehicle surroundings information. , Step ; - the current comparing the vehicle surroundings information with stored vehicle surroundings information of at least one data set (S12); - determining (S13) at least one similarity parameter for at least one data set; And, the similarity parameters are respectively the current The degree of similarity between each of the stored data sets of vehicle surrounding information and the stored vehicle surrounding information is shown. , Step ; - if the similarity parameter is greater than a predetermined threshold, reading information on the virtual camera position and the camera orientation of the stored data set containing the vehicle surroundings information and determining the virtual camera position and the camera orientation based on the read information on the virtual camera position and the camera orientation (S14). 、 A method comprising: . 2. the current The task of comparing the vehicle surroundings information with the stored vehicle surroundings information is the current 2. The method according to claim 1, further comprising comparing image similarity and image features between the vehicle surroundings information and the stored vehicle surroundings information. 3. Multiple Number different Multiple Similarity Parameters for Data Sets but Divide Be , the current Vehicle surroundings information and stored vehicle surroundings information With between of Maximum indicates the best match of Similarity Parameter but decision Be , The maximum similarity parameter is used for the threshold comparison, and maximum of Similarity Parameter Leading The virtual camera position and the vehicle surroundings are stored in a dataset that contains the vehicle surroundings. camera Orientation Information is read out 3. The method according to claim 1 or 2, characterized in that 4. The data sets each have a location and associated neighborhood information, and are run to determine at least one similarity parameter. the current4. A method according to any one of 1 to 3 above, characterized in that by comparing the vehicle surroundings information with stored vehicle surroundings information and comparing the similarity parameter with a threshold, it is determined whether the vehicle (1) is in a vehicle position for which a virtual camera position and a camera orientation have already been determined. 5. The vehicle (1) has a sensor-based location detection system (6), and the data sets each include stored vehicle surroundings information provided by the location detection system (6). photograph 5. The method according to any one of 1 to 4 above, further comprising the step of: providing position data indicating the determined geographical position. 6. 6. The method according to claim 5, characterized in that the location information is determined by a sensor-based position detection system (6), the location information is compared with position data stored in a dataset, and the comparison of the location information with the position data stored in the dataset limits the number of datasets used to determine the similarity parameter. 7. A method according to any one of claims 1 to 6, characterized in that the vehicle's computing unit (5) creates three-dimensional surrounding information from the information provided by the camera (3) based on the virtual camera position and camera orientation, which is displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1). 8. 8. The method according to claim 7, characterized in that the spatial and perspective surrounding information comprises a perspective view of the vehicle (1) itself. 9. A system including a plurality of cameras (3) distributed around a vehicle (1), a computing unit (5) for processing information provided by the cameras (3), a memory unit (4), and a display unit (2) configured to display information about the surroundings of the vehicle (1), the computing unit (5) performing the following steps: 、 - The vehicle (1) is equipped with multiple cameras (3) to monitor the surroundings of the vehicle. photographBased on the information from these cameras (3) the current A step of creating vehicle surrounding information; - storing the plurality of data sets in a memory unit (4); And The datasets are captured by multiple cameras (3) on a vehicle (1), photograph The vehicle surroundings information includes the stored vehicle surroundings information and the virtual camera position and camera orientation information assigned to the stored vehicle surroundings information. , Step ; - the current comparing the vehicle surroundings information of the vehicle surroundings information with stored vehicle surroundings information of at least one data set; - determining at least one similarity parameter for at least one data set; And , the similarity parameters are respectively the current The degree of similarity between each of the stored data sets of vehicle surrounding information and the stored vehicle surrounding information is shown. , Step ; - if the similarity parameter is greater than a predetermined threshold, retrieving information on the virtual camera position and the camera orientation of the stored data set containing the vehicle surroundings information and determining the virtual camera position and the camera orientation based on the retrieved information on the virtual camera position and the camera orientation. 、 A system that is configured to run . 10. The arithmetic unit (5) the current determining image similarity to compare the vehicle surroundings information of the vehicle surroundings to the stored vehicle surroundings information; the current 10. The system according to claim 9, configured to compare image features between the vehicle surroundings information and stored vehicle surroundings information. 11. The arithmetic unit (5) different determining a plurality of similarity parameters for the data set; the currentVehicle surroundings information and stored vehicle surroundings information With between of Maximum indicates the best match of determining a similarity parameter; The maximum similarity parameter is used for the threshold comparison, and maximum of Similarity Parameter Leading The virtual camera position of the dataset that contains the stored vehicle surroundings information and Virtual Camera 11. The system according to claim 9 or 10, characterized in that it is configured to be able to read out information relating to orientation. 12. The vehicle (1) has a sensor-based location detection system (6), and the computing unit (3) stores in a data set each stored piece of vehicle surroundings information provided by the location detection system (6). photograph 12. The system according to any one of claims 9 to 11, configured to store position data indicative of the geographical location of the detected vehicle. 13. The system described in claim 12, characterized in that the sensor-based location detection system (6) is configured to determine geographical location information, and the computing unit (5) is configured to compare the geographical location information with position data stored in a dataset and, by comparing this location information with the position data stored in the dataset, limit the dataset to only those for which a similarity parameter is determined, respectively. 14. The system according to any one of claims 9 to 13, characterized in that the computing unit (5) is configured to generate, from the information provided by the camera (3), three-dimensional surrounding information, which is displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1) based on a virtual camera position and a camera orientation. 15. 15. The system according to claim 14, characterized in that the computing unit (5) is configured to represent in the spatial and perspective surroundings information also a perspective view of the vehicle (1) itself. [Explanation of symbols]

[0052] 1 vehicle 2 Display unit 3. Camera 4 Memory Unit 5 Calculation Unit 6 Position Detection System 7 Surrounding Recognition System 8 3D Surround View System

Claims

1. A method for determining a virtual camera position and a virtual camera orientation so that the surroundings of a vehicle (1) can be displayed on a display unit (2) of the vehicle (1) based on the virtual camera position and the virtual camera orientation, the method comprising the steps of: - a step (S10) of photographing the surroundings of the vehicle (1) using a number of cameras (3) of the vehicle (1) and creating current vehicle surroundings information based on the information of the images provided by these cameras (3); - storing (S11) a number of data sets in a memory unit (4), each of which contains information about the vehicle surroundings captured by the plurality of cameras (3) of the vehicle (1) and stored, as well as about virtual camera positions and virtual camera orientations attributed to said stored vehicle surroundings information; - a step (S12) of comparing the current vehicle surroundings information with stored vehicle surroundings information of at least one data set; - determining (S13) at least one similarity parameter for at least one data set, said similarity parameter being indicative of a degree of similarity of the current vehicle surroundings information with a respective stored data set of stored vehicle surroundings information; if the similarity parameter is greater than a predefined threshold, reading information on the virtual camera position and the virtual camera orientation of the stored data set comprising the vehicle surroundings information and determining the virtual camera position and the virtual camera orientation on the basis of the read information on the virtual camera position and the virtual camera orientation (S14); The method according to claim 1, further comprising:

2. The method of claim 1, characterized in that the task of comparing the current vehicle surroundings information with the stored vehicle surroundings information also includes comparing image similarity and image features between the current vehicle surroundings information and the stored vehicle surroundings information.

3. A method according to claim 1 or 2, characterized in that multiple similarity parameters are calculated for multiple different data sets, a maximum similarity parameter indicating the greatest degree of match between the current vehicle surroundings information and the stored vehicle surroundings information is determined, the maximum similarity parameter is used for comparison with a threshold, and information regarding the virtual camera position and virtual camera orientation of the data set containing the stored vehicle surroundings information from which the maximum similarity parameter was derived is read out.

4. 3. The method according to claim 1, wherein the data sets each comprise surrounding information associated with a position, and wherein a comparison of the current vehicle surroundings information with the stored vehicle surroundings information for determining at least one similarity parameter and a comparison of the similarity parameter with a threshold value determines whether the vehicle (1) is already in a vehicle position for which a virtual camera position and a virtual camera orientation have been determined.

5. 3. The method according to claim 1 or 2, characterized in that the vehicle (1) comprises a sensor-based position detection system (6), and the data sets each comprise position data indicating a geographical position at which each stored piece of vehicle surroundings information provided by the position detection system (6) was captured.

6. 6. The method according to claim 5, characterized in that the location information is determined by a sensor-based position detection system (6), the location information is compared with position data stored in a dataset, and the comparison of the location information with the position data stored in the dataset limits the number of datasets used to determine the similarity parameter.

7. 3. The method according to claim 1 or 2, characterized in that a computing unit (5) of the vehicle creates three-dimensional surroundings information from the information provided by the camera (3) on the basis of a virtual camera position and a virtual camera orientation, which is displayed as spatial and perspective surroundings information on a display unit (2) of the vehicle (1).

8. 8. The method according to claim 7, characterized in that the spatial and perspective surroundings information comprises a perspective view of the vehicle (1) itself.

9. A system including a plurality of cameras (3) distributed around a vehicle (1), a computing unit (5) for processing information provided by the cameras (3), a memory unit (4), and a display unit (2) configured to display information about the surroundings of the vehicle (1), the computing unit (5) performing the following steps: - taking pictures of the vehicle's surroundings by a number of cameras (3) mounted on the vehicle (1) and creating current vehicle surroundings information based on the image information provided by these cameras (3); - storing a number of data sets in a memory unit (4), each data set containing stored vehicle surroundings information captured by a number of cameras (3) of the vehicle (1) and information on virtual camera positions and virtual camera orientations attributed to said stored vehicle surroundings information; - comparing the current vehicle surroundings information with stored vehicle surroundings information of at least one data set; - determining at least one similarity parameter for at least one data set, the similarity parameter being indicative of a degree of similarity of the current vehicle surroundings information to a respective stored data set of stored vehicle surroundings information; if the similarity parameter is greater than a predetermined threshold, retrieving information on the virtual camera position and the virtual camera orientation of the stored data set comprising the vehicle surroundings information and determining the virtual camera position and the virtual camera orientation on the basis of the retrieved information on the virtual camera position and the virtual camera orientation; A system that is configured to run

10. 10. The system of claim 9, wherein the computing unit (5) is configured to determine image similarity and compare image features between the current vehicle surroundings information and the stored vehicle surroundings information in order to compare the current vehicle surroundings information and the stored vehicle surroundings information.

11. 11. The system according to claim 9 or 10, characterized in that the computing unit (5) is configured to determine a plurality of similarity parameters for a plurality of different data sets, determine a maximum similarity parameter that indicates a maximum degree of match between the current vehicle surroundings information and the stored vehicle surroundings information, use the maximum similarity parameter for a threshold comparison, and retrieve information regarding the virtual camera position and virtual camera orientation of the data set containing the stored vehicle surroundings information that led to the maximum similarity parameter.

12. 11. The system according to claim 9 or 10, characterized in that the vehicle (1) comprises a sensor-based position detection system (6), and the computing unit (3) is configured to store in the data set position data indicative of the geographical position at which each stored piece of vehicle surroundings information provided by the position detection system (6) was captured.

13. 13. The system according to claim 12, characterized in that the sensor-based location detection system (6) is configured to determine geographical location information, and the computing unit (5) is configured to compare the geographical location information with position data stored in a dataset and, by comparing the location information with the position data stored in the dataset, to limit the dataset to only those for which a similarity parameter determination is performed, respectively.

14. 11. The system according to claim 9 or 10, characterized in that the computing unit (5) is configured to generate, from the information provided by the camera (3), three-dimensional surrounding information based on a virtual camera position and a virtual camera orientation, which is displayed as spatial and perspective surrounding information on a display unit (2) of the vehicle (1).

15. 15. The system according to claim 14, characterized in that the computing unit (5) is configured to represent in the spatial and perspective surroundings information also a perspective view of the vehicle (1) itself.

Images