Method and system for determining the position and orientation of a virtual camera of a peripheral observation system

Abstract

The present invention relates to a method for determining a virtual camera position and a camera orientation in order to be able to display the surroundings of a vehicle (1) on a display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation.

Description

Technical Field

[0001] The present invention relates to the field of vehicle surrounding observation systems. In particular, the present invention relates to a method and a system for determining the position and orientation of a virtual camera of a surrounding observation system.

Background Art

[0002] Vehicle surrounding observation systems are in principle known. In particular, there are known surrounding observation systems, namely so-called 3D surround view systems, which enable a three-dimensional and perspective 360° surround view around the vehicle. In the case of such a system, the image information of a plurality of cameras is connected for the surround view depiction and mapped onto a three-dimensional surface in order to obtain a spatial and perspective depiction. The spatial and perspective depiction of the surrounding part of the vehicle is carried out from various viewpoints in known systems. By doing so, the position and orientation of the virtual camera, that is, the position and the line-of-sight direction from which the virtual observer is observing the scene, can be changed so that various regions of the surrounding part of the vehicle can be observed. Particularly preferably, the position and orientation of the virtual camera can be selected such that not only the surrounding area but also the entire vehicle can be recognized within that surrounding area.

[0003] In conventional surrounding observation systems, the position and orientation of the virtual camera have to be selected by the user according to the situation so that important areas can be displayed on the display unit of the vehicle. Therefore, the vehicle user cannot concentrate on driving the vehicle, feels this as a disadvantage, and furthermore, it can be a cause of accidents.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Based on this, an object of the present invention is to provide a method capable of automatically determining the position and orientation of a virtual camera of a surrounding observation system.

Means for Solving the Problems

[0005] This problem is achieved by a method having the features described in independent claim 1. Preferred embodiments are the subject of the dependent claims. A system for determining the position and orientation of a virtual camera of a surround observation system is the subject of parallel independent claim 9.

[0006] According to a first aspect, a method for determining the position and orientation of a virtual camera of a surround observation system is disclosed. Based on this virtual camera position and camera orientation, the surrounding information of the vehicle is depicted on the display unit of the vehicle. In other words, by determining the position and orientation of the virtual camera, the viewpoint from which the surrounding information is observed in that direction is defined. Here, the general method includes the following steps:

[0007] First, the surrounding part of the vehicle is Shooting captured by a plurality of cameras of the vehicle, and based on the information of these cameras Currently the surrounding information of the vehicle is created. Particularly preferably, the image information provided by these cameras is Currently connected to each other to enable a 360° surround view of the surrounding of the vehicle.

[0008] In addition, a memory unit capable of storing a plurality of data sets is provided. Each of these data sets includes the surrounding information of the vehicle captured and stored by a plurality of cameras of the vehicle. In addition, these data sets each include information regarding the virtual camera position and camera orientation so that the attribution of the stored surrounding information of the vehicle to the virtual camera position and camera orientation can be implemented for each data set. These data sets are preferably created by the driver during its driving situation by storing the selected virtual camera position and camera orientation together with the surrounding information of the vehicle.

[0009] Subsequently, at that point ShootingThe acquired vehicle surrounding information is compared with the vehicle surrounding information of at least one data set stored in the memory unit.

[0010] Next, at least one similarity parameter for at least one data set is determined. The similarity parameter Shooting acquired Currently gives the degree of similarity of each of the acquired vehicle surrounding information to the vehicle surrounding information stored in each data set.

[0011] Finally, the similarity parameter is compared with a threshold value. If the similarity parameter is greater than a predetermined threshold value, information regarding the virtual camera position and camera orientation of the data set including the stored vehicle surrounding information is read out. In addition, the virtual camera position and camera orientation are determined based on the read information regarding the virtual camera position and camera orientation. That is, the description of the surrounding information is carried out from the viewpoint determined by the read information regarding the virtual camera position and camera orientation.

[0012] This method The surrounding observation system automatically selects the virtual camera position and virtual camera orientation for the current vehicle surrounding area has the technical advantage that Virtual· in the past, the virtual camera position and Already the camera orientation were determined and generates and the description of the vehicle surrounding part is carried out on the display unit of the vehicle. Generate When the driver wants to reuse the virtual camera position and camera orientation determined in the past, driver intervention in the surrounding observation system is not required. That is, the driver is not distracted by the operation of the surrounding observation system and can be more concentrated on the driving maneuver, so the user-friendliness of the driver assistance system is further improved.

[0013] According to one embodiment, CurrentlyThe task of comparing the vehicle surrounding information with the stored vehicle surrounding information includes Currently comparing the image similarity and image features between the vehicle surrounding information of Currently and the stored vehicle surrounding information. By adopting this method, through image processing, it is possible to determine whether the vehicle is at a position where information regarding the virtual camera position and virtual camera orientation already exists.

[0014] According to an embodiment, a plurality of similarity parameters are determined for a plurality of different data sets. At this time, Currently the largest similarity parameter with the highest degree of coincidence between the vehicle surrounding information of Currently and the stored vehicle surrounding information is determined. And this maximum similarity parameter is used for comparison with the threshold value. In order to determine the virtual camera position and virtual camera orientation, information regarding the virtual camera position and virtual camera orientation of the data set including the stored vehicle surrounding information from which the maximum similarity parameter is obtained is read out. By doing so, the vehicle surrounding information of a plurality of data sets can be examined, and in order to determine the virtual camera position and virtual camera orientation, the vehicle surrounding information stored therein Currently is most consistent with the vehicle surrounding information of Currently Or and it becomes possible to use a similar data set.

[0015] According to an embodiment, each data set has surrounding information associated with a position. The comparison between the vehicle surrounding information of Currently and the stored vehicle surrounding information, which is performed to determine at least one similarity parameter, and the comparison between the similarity parameter and the threshold value determine whether the vehicle is already at a vehicle position where the virtual camera position and virtual camera orientation have been determined. Thereby, camera-based surrounding Currently Shooting ​and whether there is a match with a known vehicle periphery where the observation point was determined in the past is determined by comparison with known surrounding information.

[0016] According to one embodiment, the vehicle has a sensor-based position detection system, for example, a GPS-based position detection system. The data set has position data that the position detection system provided in the past. Those position data give the geographical positions where each stored vehicle surrounding information Shooting was located. The position data are, for example, geographical coordinates in a global coordinate system. Thereby, the Currently position information of the position detection system is obtained, and from the comparison between the Currently position information and the position data of the data set, the relevance of each data set can be estimated without having to compare the Currently vehicle surrounding information with the stored vehicle surrounding information.

[0017] According to one embodiment, the position information is determined by a sensor-based position detection system. These position information are compared with the position data stored in the data set. By comparing the position information with the position data stored in the data set, the data set is limited. That is, for which data set, Or and for which vehicle surrounding information contained and stored in each data set, the determination of the similarity parameter should be performed respectively is limited. The sensor-based position detection system has inaccuracies, Or blurriness that make accurate position detection impossible in its position detection method. Typical inaccuracies Or and blurriness are on the order of several meters. Therefore, in a sensor-based position detection system, accurate position detection of the vehicle is impossible, but it is possible to support visual position detection based on the stored data set by the sensor-based position detection system. That is, Currently the comparison between the vehicle surrounding information and the stored vehicle surrounding information is performed in the area of the current vehicle position ShootingIt is possible to limit to highly relevant datasets that have been obtained. As a result, the computational load required for comparison can be significantly reduced.

[0018] According to one embodiment, an arithmetic unit of a vehicle creates three-dimensional surrounding information that is displayed as spatial and perspective surrounding information on a display unit of the vehicle based on a virtual camera position and a camera orientation from information provided by a camera. At this time, the virtual camera position and the camera orientation provide a viewpoint from which the surrounding information is observed and a sector of the surrounding information displayed on the display unit. As a result, the surrounding information can be depicted in a natural and user-friendly manner by the surrounding recognition system.

[0019] According to one embodiment, the spatial and perspective surrounding information also includes a perspective view of the vehicle itself. For example, the surrounding area above the vehicle itself can also be depicted in perspective, that is, the perspective surrounding information includes not only the periphery of the vehicle but also the posture of the vehicle, Or and it may be recognized that it is embedded in the periphery. As a result, the driver can recognize the relative position of the vehicle with respect to the surroundings, enabling more improved driver support by the surrounding recognition system.

[0020] In a further aspect, the present invention relates to a system for determining the position and orientation of a virtual camera of a surrounding observation system. The system includes a plurality of cameras distributed around the vehicle, an arithmetic unit for processing information provided by these cameras, a memory unit, and a display unit configured to be able to display the surrounding information of the vehicle. The arithmetic unit performs the following steps 、 - capturing the vehicle surroundings with a plurality of in-vehicle cameras Shooting and creating vehicle surrounding information based on the information of these cameras Currently ; - storing a plurality of datasets in the memory unit And, each dataset is obtained by a plurality of in-vehicle cameras Shooting and includes the vehicle surrounding information saved thereby, and information regarding the virtual camera position and camera orientation attributed to the saved vehicle surrounding information Step ; - Currently comparing the vehicle surrounding information of [vehicle surrounding information of -] with the saved vehicle surrounding information of at least one dataset; - determining at least one similarity parameter for at least one dataset And , each of the similarity parameters Currently indicates the degree of similarity between the saved vehicle surrounding information of each dataset of the vehicle surrounding information of [vehicle surrounding information of -] Step ; - when the similarity parameter is greater than a predetermined threshold value, reading out the information regarding the virtual camera position and camera orientation of the dataset including the saved vehicle surrounding information, and determining the virtual camera position and camera orientation based on the read information regarding the virtual camera position and camera orientation; is configured to be able to execute.

[0021] According to an embodiment of the system, the arithmetic unit Currently determines the image similarity in order to compare the vehicle surrounding information of [vehicle surrounding information of -] with the saved vehicle surrounding information, Currently and is configured to be able to compare the image features between the vehicle surrounding information of [vehicle surrounding information of -] and the saved vehicle surrounding information. By adopting this method, it is possible to determine whether there is a vehicle at the position where the information regarding the virtual camera position and virtual camera orientation already exists by performing image processing.

[0022] According to an embodiment of the system, the arithmetic unit Different determines a plurality of similarity parameters for a plurality of Is datasets Performed ,Currently The vehicle surrounding information and the stored vehicle surrounding information With between Of The maximum similarity parameter that shows the maximum degree of match Of Similarity parameter Is Determine Performed , The maximum similarity parameter is used for comparison with the threshold, and Maximum Of Similarity parameter Leads to The virtual camera position and Camera Information regarding orientation Is read configured to be able to do so. By doing so, the vehicle surrounding information of multiple datasets can be examined, and in order to determine the virtual camera position and virtual camera orientation, the vehicle surrounding information stored therein Currently is most consistent with the vehicle surrounding information of Or it becomes possible to use similar datasets.

[0023] According to an embodiment of the system, the vehicle has a sensor-based position detection system. The arithmetic unit is configured to store, in the dataset, position data indicating the geographical position where each of the stored vehicle surrounding information provided from the position detection system Shooting was obtained. Those position data give the geographical position where each of the stored vehicle surrounding information Shooting was obtained. The position data are, for example, geographical coordinates in a global coordinate system. Thereby, based on the Currently position information of the position detection system, the relevance of each dataset can be estimated without having to compare the vehicle surrounding information of Currently with the stored vehicle surrounding information.

[0024] According to an embodiment of the system, a sensor-based position detection system is configured to determine geographical position information. The arithmetic unit compares the geographical position information with the position data stored in the dataset, and configures the dataset to be limited only to those for which similarity parameters are respectively determined by comparing this position information with the position data stored in the dataset. Thereby, Currently the computational load required for comparing the vehicle surrounding information of Currently with the stored vehicle surrounding information can be significantly reduced.

[0025] According to an embodiment of the system, the arithmetic unit is configured to generate three-dimensional surrounding information from the information provided by a plurality of cameras for display as spatial and perspective surrounding information on the display unit of the vehicle based on the virtual camera position and camera orientation. The arithmetic unit of the vehicle is configured to be able to create three-dimensional surrounding information to be displayed as spatial and perspective surrounding information on the display unit of the vehicle based on the virtual camera position and camera orientation from the information provided by the camera. Thereby, it becomes possible to depict the surrounding information in a natural and user-friendly manner by the surrounding perception system.

[0026] According to an embodiment of the system, the arithmetic unit is configured to be able to show a perspective view of the vehicle itself in the spatial and perspective surrounding information. Thereby, the driver can recognize what relative position the vehicle is in with respect to the surroundings, enabling improved driver support by the surrounding perception system.

[0027] "Approximately" and "substantially" in the present invention Also expressions such as "substantially" mean an error within + / - 10%, preferably within + / - 5% respectively from the exact value, and / Also is interpreted as an error that is not significant for the function.

[0028] The developed forms, advantages, and application scopes of the present invention are shown by the following descriptions of the embodiments and the drawings. At that time, all the described and / Also are, in each case, and in any combination, each claim, Also is an object of the present invention independently from the combination with its reference source. Note that the content of the claim is also a component of the specification.

[0029] Hereinafter, the present invention will be described in detail based on the drawings and embodiments. Description of the drawings:

Brief Description of the Drawings

[0030]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0031] FIG. 1 schematically illustrates a vehicle 1 having a system, also referred to as a 3D surround view system hereinafter, for three-dimensionally depicting the surrounding area of the vehicle 1 on a display unit 2. The display unit 2 is particularly preferably provided inside the vehicle and, for example, plays a role of supporting the driver during driving maneuvers such as reverse driving and parking in a parking space.

[0032] Vehicle 1 has a plurality of cameras 3 that are provided outside and dispersed around Vehicle 1. For example, the vehicle 1 has a front camera, a rear camera, and at least one side camera each. The cameras 3 are preferably arranged and configured such that by synthesizing the images provided by each camera 3, a 360° depiction of the area around Vehicle 1 can be obtained as a single image.

[0033] Vehicle 1 is preferably configured such that three-dimensional peripheral information that is displayed as spatial and perspective peripheral information on the display unit 2 can be generated from the image information provided by at least two cameras 3.

[0034] Figure 2 illustrates a perspective depiction of a three-dimensional space within the peripheral area of Vehicle 1. Such a depiction is, for example, called a 3D ball. From the image information provided by the cameras 3, spatial and perspective peripheral information is calculated by an arithmetic unit 5 provided in the vehicle. In the case of this spatial and perspective peripheral information, preferably, Vehicle 1 itself Or and its outline can be recognized. This type of perspective surround view depiction, unlike an overhead view (bird's-eye view), enables the peripheral area of Vehicle 1 to be observed perspectively from a specific reference point that corresponds to the following camera position, Or the observation point. At that time, the camera position is variable, that is, the virtual position from which Vehicle 1 and its surroundings are observed can be changed (virtual movement within the 3D ball). This enables a realistic depiction of the peripheral area of Vehicle 1.

[0035] In addition, from the set camera position, the camera orientation, that is, the angle at which Vehicle 1 Or observes the peripheral part of Vehicle 1 from that camera position can also be changed.

[0036] Vehicle 1 is configured to automatically determine the camera position and camera orientation. In short, the vehicle 1 is configured such that it can determine whether the vehicle 1 is at a position where the camera position and camera orientation have already been determined by the vehicle user at that time. It has a virtual peripheral recognition system. At that time, the virtual peripheral recognition system uses the image information provided by camera 3 and generates vehicle peripheral information therefrom, Currently and compares these with the stored vehicle peripheral information where the vehicle user has already determined the camera position and camera orientation there.

[0037] As schematically suggested in FIG. 1, the vehicle includes an arithmetic unit 5 by which the automatic determination of the camera position and camera orientation is carried out. FIG. 3 illustrates in a schematic block diagram the configuration of a system for automatically determining the camera position and camera orientation. The arithmetic unit 5 of the vehicle implements, for example, a virtual peripheral recognition system 7 and a 3D surround view system 8. The peripheral recognition system 7 is connected to the cameras 3 of the vehicle 1 and receives their image information. Subsequently, the image information of the individual cameras 3 is synthesized and Currently processed into vehicle peripheral information. For example, the image information of the individual cameras 3 can be joined together so that a 360° surround view is completed.

[0038] The surrounding recognition system 7 is connected to the memory unit 4. The memory unit 4 is configured to store one, preferably a plurality of, data sets. At this time, one data set includes vehicle surrounding information calculated based on the image information of the camera 3 at a certain past time point. Here, the vehicle surrounding information can be obtained, particularly preferably, also by the surrounding recognition system 7 by synthesizing the image information of a plurality of cameras 3. In addition, each of the data sets also has information regarding virtual camera positions and virtual camera orientations that have already been used within the vehicle surrounding area by the vehicle user at that time. In other words, the memory unit 4 serves to store virtual camera position information and virtual camera orientation information by the user at a specific position, for example, at a location frequently visited by the vehicle user.

[0039] The surrounding recognition system 7 is Currently of Shooting configured to recognize whether the vehicle 1 is already at a position where the virtual camera position and the camera orientation have been determined by the vehicle user by comparing the calculated vehicle surrounding information with the vehicle surrounding information stored in the data set stored in the memory unit 4 of the vehicle 1.

[0040] Specifically, Currently of Shooting the calculated vehicle surrounding information is compared with the stored vehicle surrounding information by determining the image similarity and / Also or the feature matching degree. Shooting Based on the comparison between the calculated vehicle surrounding information and the stored vehicle surrounding information, Currently of Shooting a similarity parameter is determined that gives the probability that the calculated vehicle surrounding information was taken at the same vehicle position as the stored vehicle surrounding information. When the similarity parameter exceeds a predetermined threshold value, CurrentlyThe vehicle position is recognized to be the same location where the stored vehicle surrounding information was captured. In this case, information regarding the virtual camera position and camera orientation attributed to the stored vehicle surrounding information is read from the memory unit 4, and based on this, it is used to determine the virtual camera position and camera orientation at the current vehicle position.

[0041] The read virtual camera position and camera orientation are particularly preferably transmitted to a 3D surround view system 8 that encourages depicting the vehicle surrounding information based thereon Currently on the display unit 2 of vehicle 1.

[0042] According to an embodiment, vehicle 1 has a sensor-based position detection system 6. This sensor-based position detection system 6 can be, for example, a Star GPS system (GPS: Global Positioning System) supported by a satellite. By using this position detection system 6, it is possible to determine the position information of vehicle 1. These position information can thereby indicate where the vehicle is currently located, and are particularly preferably coordinates in a global coordinate system.

[0043] This position information can be used to support visual position identification based on the surrounding information of the image. In particular, Currently it can be used to limit the dataset to be examined regarding the correspondence between the vehicle surrounding information and the stored vehicle surrounding information. By doing so, the dataset can also have the position information obtained from the position detection system 6 in addition to the stored vehicle surrounding information. In short, Currently before comparing the vehicle surrounding information and the stored vehicle surrounding information, by determining the dataset obtained at a position close to the current location of vehicle 1, only the stored vehicle surrounding information of the dataset obtained at a position close to the current location of vehicle 1 CurrentlyIt becomes possible to compare with the vehicle surrounding information.

[0044] Preferably, the dataset stored in the memory unit 4 belongs to the profile of the vehicle user. By doing so, the position and orientation of the virtual camera are stored depending on the vehicle user, and when determining the position and orientation of the virtual camera, they can be used depending on the user profile at that time.

[0045] FIG. 4 is a block diagram showing steps of a method for determining the position and orientation of a virtual camera.

[0046] First, the vehicle periphery is Shooting captured by a plurality of cameras of the vehicle, and vehicle surrounding information Currently is created based on the information of these cameras (S10).

[0047] Furthermore, a memory unit capable of storing a plurality of datasets is provided. Each of the datasets is Shooting captured by a plurality of cameras of the vehicle and includes the stored vehicle surrounding information. In addition, these datasets also include information regarding the virtual camera position and camera orientation to which the stored vehicle surrounding information belongs (S11).

[0048] Subsequently, Currently the vehicle surrounding information is compared with the vehicle surrounding information of at least one of the stored datasets (S12).

[0049] Next, at least one similarity parameter for at least one dataset is determined. The similarity parameter is Currently one that gives the degree of similarity of each of the vehicle surrounding information to the vehicle surrounding information stored in each dataset (S13).

[0050] If the similarity parameter is greater than a predefined threshold, information about the virtual camera position and the camera orientation of the stored data set containing the vehicle surroundings information is retrieved, and a virtual camera position and a camera orientation are determined based on the retrieved information about the virtual camera position and the camera orientation (S14).

[0051] The invention has been described by way of the above examples, however it will be appreciated that numerous modifications and variations are possible without departing from the scope of the invention as defined in the appended claims. This application relates to the invention described in the claims, but also includes the following as other aspects. 1. 1. A method for determining a virtual camera position and a camera orientation so that the surroundings of a vehicle (1) can be displayed on a display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation, the method comprising the steps of: 、 - The vehicle (1) is equipped with multiple cameras (3) to monitor the surroundings of the vehicle. Shooting Based on the information from these cameras (3) Currently A step of creating vehicle surrounding information (S10); - providing (S11) a memory unit (4) in which a plurality of data sets are stored; And , the data set is respectively obtained by a number of cameras (3) of a vehicle (1). Shooting The vehicle surroundings information includes the stored vehicle surroundings information and the virtual camera position and camera orientation information assigned to the stored vehicle surroundings information. Step ; - Currently comparing the vehicle surroundings information with stored vehicle surroundings information of at least one data set (S12); - determining (S13) at least one similarity parameter for at least one data set; And, each of the similarity parameters indicates the degree of similarity between the stored vehicle surrounding information in each dataset of Currently the stored vehicle surrounding information; Step ; - When the similarity parameter is greater than a predetermined threshold value, read the information regarding the virtual camera position and camera orientation of the dataset including the stored vehicle surrounding information, and determine the virtual camera position and camera orientation based on the read information regarding the virtual camera position and camera orientation (step S14) 、 A method characterized by including . 2. Currently Comparing the vehicle surrounding information of Currently with the stored vehicle surrounding information also includes comparing the image similarity and image features between the vehicle surrounding information of 3. Multiple number of Different similarity parameters for multiple datasets Is calculate Performed , Currently the vehicle surrounding information of With and the stored vehicle surrounding information Of to determine the maximum Of similarity parameter Is showing the highest degree of match, Performed , The maximum similarity parameter is used for comparison with the threshold, and the maximum Of similarity parameter Leads to and read the information regarding the virtual camera position and orientation of the dataset including the stored vehicle surrounding information corresponding to the maximum Camera similarity parameter Is read The method according to 1 or 2 above, characterized in that. 4. Each of the datasets has surrounding information related to a position, and is implemented to calculate at least one similarity parameter CurrentlyBy comparing the vehicle surrounding information with the stored vehicle surrounding information, and by comparing the similarity parameter with the threshold value, it is determined whether the vehicle (1) is already at the vehicle position where the virtual camera position and the camera orientation have been determined. The method according to any one of the above 1 to 3, characterized in that. 5. The vehicle (1) has a sensor-based position detection system (6), and each dataset has respective stored vehicle surrounding information provided from the position detection system (6) Shooting The method according to any one of the above 1 to 4, characterized in that it has position data indicating the determined geographical position. 6. The position information is determined by a sensor-based position detection system (6), the position information is compared with the position data stored in the dataset, and the number of datasets used to determine the similarity parameter by comparing the position information with the position data stored in the dataset is limited. The method according to 5 above, characterized in that. 7. The arithmetic unit (5) of the vehicle creates three-dimensional surrounding information to be displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation from the information provided by the camera (3). The method according to any one of the above 1 to 6, characterized in that. 8. The method according to 7 above, characterized in that the spatial and perspective surrounding information includes the perspective view of the vehicle (1) itself. 9. A system including a plurality of cameras (3) distributed around the vehicle (1), an arithmetic unit (5) for processing the information provided from these cameras (3), a memory unit (4), and a display unit (2) configured to be able to display the surrounding information of the vehicle (1), wherein the arithmetic unit (5) performs the following steps 、 - The surrounding part of the vehicle by a plurality of cameras (3) of the vehicle (1) Shootingand, based on the information of these cameras (3), Currently a step of creating vehicle surrounding information; - a step of storing a plurality of data sets in the memory unit (4) And , each data set being Shooting obtained by a plurality of cameras (3) of the vehicle (1), and including the stored vehicle surrounding information and information regarding the virtual camera positions and camera orientations attributed to the stored vehicle surrounding information Step ; - Currently a step of comparing the vehicle surrounding information of with the stored vehicle surrounding information of at least one data set; - a step of determining at least one similarity parameter for at least one data set And , each of the similarity parameters Currently indicating the degree of similarity between the stored vehicle surrounding information of each data set storing the vehicle surrounding information of Step ; - when the similarity parameter is greater than a predetermined threshold value, reading out information regarding the virtual camera positions and camera orientations of the data set including the stored vehicle surrounding information, and determining the virtual camera positions and camera orientations based on the read information regarding the virtual camera positions and camera orientations 、 A system configured to execute . 10. The arithmetic unit (5) is Currently configured to determine image similarity in order to compare the vehicle surrounding information of with the stored vehicle surrounding information, and Currently configured to be able to compare image features between the vehicle surrounding information of and the stored vehicle surrounding information. The system according to 9 above. 11. The arithmetic unit (5) is Different configured to determine a plurality of similarity parameters for a plurality of CurrentlyThe vehicle surrounding information and the stored vehicle surrounding information With between Of the maximum that indicates the maximum degree of match Of determine the similarity parameter, The maximum similarity parameter is used for comparison with the threshold, and the maximum Of similarity parameter Leads to the virtual camera position of the dataset that includes the stored vehicle surrounding information And virtual camera The system according to any one of the above 9 or 10, characterized in that it is configured to be able to read information regarding the orientation. 12. The vehicle (1) has a sensor-based position detection system (6), and the arithmetic unit (3) stores position data indicating the geographical position provided by the position detection system (6) in each of the stored vehicle surrounding information in the dataset. The system according to any one of the above 9 to 11, characterized in that it is configured as such. Shooting The system according to any one of the above 9 to 11, characterized in that the sensor-based position detection system (6) is configured to determine geographical position information, and the arithmetic unit (5) compares the geographical position information with the position data stored in the dataset, and by comparing this position information with the position data stored in the dataset, the dataset is configured to be limited only to those for which the determination of the similarity parameter is carried out. 13. The system according to the above 12, characterized in that the sensor-based position detection system (6) is configured to determine geographical position information, and the arithmetic unit (5) compares the geographical position information with the position data stored in the dataset, and by comparing this position information with the position data stored in the dataset, the dataset is configured to be limited only to those for which the determination of the similarity parameter is carried out. 14. The system according to any one of the above 9 to 13, characterized in that the arithmetic unit (5) is configured to be able to create three-dimensional surrounding information to be displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1) based on the virtual camera position and the camera orientation from the information provided by the camera (3). 15. The system according to the above 14, characterized in that the arithmetic unit (5) is configured to be able to depict the perspective view of the vehicle (1) itself in the spatial and perspective surrounding information.

Description of Symbols

[0052] 1 Vehicle 2 Display Unit 3 Camera 4 Memory Unit 5 Arithmetic Unit 6 Position Detection System 7 Peripheral Recognition System 8 3D Surround View System

Claims

1. A method for determining a virtual camera position and a virtual camera orientation so that the surrounding area of a vehicle (1) can be displayed on a display unit (2) of the vehicle (1) based on the virtual camera position and the virtual camera orientation, the method comprising the following steps: - A step (S10) of photographing the surrounding area of the vehicle by a plurality of cameras (3) of the vehicle (1) and creating current vehicle surrounding information based on information of images provided by these cameras (3); - A step (S11) of storing a plurality of data sets in a memory unit (4), each data set including vehicle surrounding information photographed and stored by a plurality of cameras (3) of the vehicle (1) and information regarding the virtual camera position and the virtual camera orientation attributed to the stored vehicle surrounding information, step (S11); - A step (S12) of comparing the current vehicle surrounding information with the stored vehicle surrounding information of at least one data set; - A step (S13) of determining at least one similarity parameter for at least one data set, each similarity parameter indicating the degree of similarity between the current vehicle surrounding information and the stored vehicle surrounding information of each stored data set, step (S13); - If the similarity parameter is greater than a predetermined threshold value, reading information regarding the virtual camera position and the virtual camera orientation of the data set including the stored vehicle surrounding information, and determining the virtual camera position and the virtual camera orientation based on the read information regarding the virtual camera position and the virtual camera orientation, step (S14), characterized by including the above steps.

2. The method according to claim 1, characterized in that the task of comparing the current vehicle surrounding information with the stored vehicle surrounding information also includes comparing the image similarity and the image features between the current vehicle surrounding information and the stored vehicle surrounding information. **Claim 3**: A plurality of similarity parameters for a plurality of different data sets are determined, a maximum similarity parameter indicating the maximum degree of match between current vehicle surrounding information and stored vehicle surrounding information is determined, the maximum similarity parameter is used for comparison with a threshold value, and information regarding the virtual camera position and virtual camera orientation of the data set including the stored vehicle surrounding information from which the maximum similarity parameter is derived is read out. The method according to claim 1 or 2, characterized in that. **Claim 4** The data set has surrounding information associated with a position respectively, and by comparing the current vehicle surrounding information and the stored vehicle surrounding information which is carried out to determine at least one similarity parameter, and by comparing the similarity parameter with a threshold value, it is determined whether the vehicle (1) is already at a vehicle position where the virtual camera position and virtual camera orientation have been determined. The method according to claim 1 or 2, characterized in that. **Claim 5** The vehicle (1) has a sensor-based position detection system (6), and the data set has position data indicating the geographical position where each of the stored vehicle surrounding information provided from the position detection system (6) was taken respectively. The method according to claim 1 or 2, characterized in that. **Claim 6** Position information is determined by a sensor-based position detection system (6), the position information is compared with the position data stored in the data set, and the number of data sets used to determine a similarity parameter by comparing the position information with the position data stored in the data set is limited. The method according to claim 5, characterized in that. **Claim 7** The arithmetic unit (5) of the vehicle creates three-dimensional surrounding information that is displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1) based on the virtual camera position and virtual camera orientation from the information provided by the camera (3). The method according to claim 1 or 2, characterized in that. **Claim 8** The spatial and perspective surrounding information includes the perspective view of the vehicle (1) itself. The method according to claim 7, characterized in that. **Claim 9** A system including a plurality of cameras (3) distributed around a vehicle (1), an arithmetic unit (5) for processing information provided by these cameras (3), a memory unit (4), and a display unit (2) configured to display peripheral information of the vehicle (1), wherein the arithmetic unit (5) performs the following steps: - A step of photographing the periphery of the vehicle by a plurality of cameras (3) of the vehicle (1) and creating current vehicle peripheral information based on the image information provided by these cameras (3); - A step of storing a plurality of data sets in the memory unit (4), each data set including the vehicle peripheral information photographed and stored by a plurality of cameras (3) of the vehicle (1) and information regarding the virtual camera position and virtual camera orientation attributed to the stored vehicle peripheral information; - A step of comparing the current vehicle peripheral information with the stored vehicle peripheral information of at least one data set; - A step of determining at least one similarity parameter for at least one data set, each similarity parameter indicating the degree of similarity between the current vehicle peripheral information and the stored vehicle peripheral information of each stored data set; - When the similarity parameter is greater than a predetermined threshold value, reading out information regarding the virtual camera position and virtual camera orientation of the data set including the stored vehicle peripheral information, and determining the virtual camera position and virtual camera orientation based on the read information regarding the virtual camera position and virtual camera orientation. A system configured to execute the above steps.

10. The system according to claim 9, characterized in that the arithmetic unit (5) is configured to determine image similarity and compare image features between the current vehicle peripheral information and the stored vehicle peripheral information in order to compare the current vehicle peripheral information with the stored vehicle peripheral information.

11. The calculation unit (5) calculates a plurality of similarity parameters for a plurality of different data sets, determines the maximum similarity parameter indicating the maximum degree of match between the current vehicle surrounding information and the stored vehicle surrounding information, uses the maximum similarity parameter for comparison with a threshold value, and can read out information regarding the virtual camera position and virtual camera orientation of the data set that includes the stored vehicle surrounding information from which the maximum similarity parameter was derived. The system according to claim 9 or 10, characterized in that it is configured as such.

12. The vehicle (1) has a sensor-based position detection system (6), and the calculation unit (3) is configured to store in the data set position data indicating the geographical position at which each stored vehicle surrounding information provided from the position detection system (6) was taken. The system according to claim 9 or 10, characterized in that it is configured as such.

13. The sensor-based position detection system (6) is configured to determine geographical position information, and the calculation unit (5) compares the geographical position information with the position data stored in the data set. By comparing this position information with the position data stored in the data set, the data set is configured to be limited only to those for which the calculation of similarity parameters is carried out. The system according to claim 12, characterized in that it is configured as such.

14. The calculation unit (5) is configured to be able to create three-dimensional surrounding information to be displayed as spatial and perspective surrounding information on the display unit (2) of the vehicle (1) based on the virtual camera position and virtual camera orientation from the information provided by the camera (3). The system according to claim 9 or 10, characterized in that it is configured as such.

15. The calculation unit (5) is configured to be able to depict the perspective view of the vehicle (1) itself in the spatial and perspective surrounding information. The system according to claim 14, characterized in that it is configured as such.

Images