Image systems for vehicles

The geometric parameters of the virtual projection surface are adjusted through sensor capture and processor aggregation data, which solves the problem of object distortion in the virtual projection surface, improves the authenticity and security of the vehicle's surrounding environment, and reduces the demand for computing resources.

CN113412499BActive Publication Date: 2024-12-03JAGUAR LAND ROVER LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202080013174.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-02-08
Filing Date
2020-01-22
Publication Date
2024-12-03
Estimated Expiration
2040-01-22

AI Technical Summary

Technical Problem

The prior art is prone to object distortion when generating virtual projection surfaces for displaying the surrounding environment of a vehicle, especially in perspective, affecting security and user experience.

Method used

Capture object data through multiple sensors, the processor aggregates the data to determine the geometric parameters of the virtual projection surface, adjusts or modifies its geometry to reduce distortion, and maps the image texture to the virtual projection surface, using sensors such as visible light or ultrasonic, combined with The image capture device is used to correct the image data.

Benefits of technology

Reduces object distortion in virtual projection surfaces, improves the authenticity and safety of the vehicle's surroundings, enhances the user experience, and reduces the demand for computing resources, especially in the case of complex geometric shapes and irregular shapes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113412499B_ABST
    Figure CN113412499B_ABST
Patent Text Reader

Abstract

One aspect of the present invention relates to a system, method and vehicle for generating a virtual projection surface. The geometric parameters of the virtual projection surface are determined based on the properties of objects around the vehicle so as to reduce the distortion of the objects in the subsequent visual representation of the vehicle's surroundings. The system includes: one or more image capture devices associated with the main vehicle and configured to capture image data indicating the environment of the main vehicle; multiple sensors associated with the main vehicle and configured to capture object data indicating the presence of at least one object near the main vehicle; and a processor, which is communicatively coupled to the one or more image capture devices and the multiple sensors to receive the captured image data and the captured object data. The processor is configured to: aggregate the object data captured by each of the multiple sensors; determine one or more geometric parameters of the virtual projection surface based on the aggregated object data; generate the virtual projection surface based on the one or more geometric parameters; determine an image texture based on the captured image data; and map the image texture onto the generated virtual projection surface.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an imaging system and method for a vehicle, and in particular, but not exclusively, to an imaging system and method for generating a virtual projection surface for displaying an image of the vehicle's surroundings. Aspects of the invention relate to systems, vehicles, and to methods. Background Art

[0002] It is known to provide an image of the environment surrounding the vehicle to a user of the vehicle. To generate the image, an image texture representing the environment of the vehicle may be mapped onto a virtual projection surface disposed around a model of the vehicle. The virtual projection surface defines a 3D geometry onto which one or more 2D images may be mapped in order to display a 3D image of the vehicle's surroundings to the user. One such virtual projection surface is described in U.S. Patent Publication No. US2015 / 0178884A1. However, one or more objects near the vehicle may appear distorted in the resulting image.

[0003] It is an object of the present invention to address one or more disadvantages of distortion associated with the prior art. Summary of the invention

[0004] Aspects and embodiments of the present invention provide an imaging system of a host vehicle, a vehicle incorporating the imaging system, and a method of generating a virtual projection surface.

[0005] According to one aspect of the present invention, an image system of a main vehicle is provided. The image system includes: a plurality of sensors associated with the main vehicle and configured to capture object data indicating the presence of at least one object near the main vehicle; and a processor communicatively coupled to the plurality of sensors to receive the captured object data. The processor is configured to: aggregate the object data captured by each of the plurality of sensors; and determine one or more geometric parameters of a virtual projection surface based on the aggregated object data. The processor may be configured to generate a virtual projection surface based on the one or more geometric parameters, or output the one or more geometric parameters to another vehicle system. The image system may also include one or more image capture devices associated with the main vehicle and configured to capture image data indicating the environment of the main vehicle. The processor may be communicatively coupled to the one or more image capture devices to receive the captured image data. The processor may be configured to: determine an image texture based on the captured image data; and map the image texture onto the generated virtual projection surface.

[0006] According to an aspect of the invention, there is provided a vehicle comprising any aspect or embodiment of an image system.

[0007] According to one aspect of the present invention, a method for generating a virtual projection surface using an image system of a host vehicle is provided. The method includes: capturing object data indicating the presence of at least one object near the host vehicle; aggregating the captured object data; and determining one or more geometric parameters of the virtual projection surface based on the aggregated object data. The method may include: generating the virtual projection surface based on the one or more geometric parameters; or outputting the one or more geometric parameters to another vehicle system. The method may include: capturing image data indicating the environment of the host vehicle; determining an image texture based on the captured image data; and mapping the image texture onto the generated virtual projection surface.

[0008] According to an aspect of the present invention, there is provided a non-transitory computer readable medium comprising computer readable instructions which, when executed by a processor, cause the performance of any aspect or embodiment of the method.

[0009] Typically, a virtual projection surface with a fixed geometry is used. By using object data to determine one or more geometric parameters of the virtual projection surface, i.e., by adjusting or modifying the geometry of the virtual projection surface, the distortion of the object in any resulting image or representation (generated according to the mapped image texture) can be reduced. Specifically, the geometric parameters can be adjusted according to the one or more objects identified so as to provide a more realistic representation of the surroundings of the vehicle to the user of the vehicle. In this way, safety can be improved. This may be particularly relevant when the surroundings of the vehicle are presented to the user in the form of a perspective view, because the distortion in the perspective view may be greater than, for example, the distortion in the plan view. In addition, in the case where a complex or irregular geometry is required for the virtual projection surface, the distortion can be reduced, which can reduce the computing resources for calculating and generating the virtual projection surface. This can also improve the practicality when the vehicle is moving, because the complex geometry will not need to be constantly recalculated, which may prove to be distracting to the user viewing the resulting image.

[0010] By aggregating object data from multiple sensors, aspects of the present invention may be more robust and may operate more effectively under adverse conditions. For example, when one or more of the sensors are not operating properly due to inclement weather or malfunction (or any other reason), the aggregated data may help compensate for the reduction in the number of fully operational sensors and may therefore allow objects to be identified in a more robust manner.

[0011] Optionally, in any of the above aspects, the multiple sensors operate in the invisible portion of the electromagnetic spectrum, or operate using pressure waves (e.g., ultrasound). By using (one or more) image capture devices for image data in combination with other such sensors for capturing object data, the (one or more) geometric parameters can be adjusted or modified even in the case of poor visibility of objects around the vehicle. In other words, other information about the vehicle environment can be used to correct or improve the images captured by the (one or more) image capture devices. Optionally, multiple sensors capture images of the vehicle's surroundings and determine properties of the objects from the images themselves.

[0012] Optionally, in any of the above aspects, the one or more geometric parameters define the extent of a ground plane of the virtual projection surface. The ground plane represents the surface on which the host vehicle is located; for example, a road immediately adjacent to the vehicle may be represented as a plane. The extent (i.e., size) of the ground plane may depend on the properties of objects surrounding the vehicle; this arrangement may, for example, enable the reduction of distortion of objects in any image or representation of the vehicle's surroundings by preventing 3D objects near the vehicle from being mapped onto the ground plane.

[0013] Optionally, in any of the above aspects, the virtual projection surface is curved beyond the ground plane, and more optionally, the virtual projection surface is a parabola beyond the ground plane. Compared with other surface geometries, a curved or parabola-shaped virtual projection surface can help to more realistically represent objects near the vehicle. Therefore, the distortion of objects around the vehicle can be reduced. This can improve safety because the size of the object or the distance between the object and the vehicle can be more realistically represented to the user. Optionally, the curvature of the virtual projection surface is consistent around the ground plane, that is, the virtual projection surface has a regular geometric shape.

[0014] Optionally, to determine the one or more geometric parameters, the method includes or the processor is configured to: determine the height of at least one object; and determine the curvature of the virtual projection surface beyond the ground plane based on the height. By basing the curvature of the curved (or optionally, parabolic) portion of the virtual projection surface on the height of the (one or more) objects around the vehicle, distortion of the object due to curvature can be reduced. This can help to more realistically represent the size and shape of the object to the user and thus improve safety.

[0015] Optionally, in order to determine the one or more geometric parameters, the method includes or the processor is configured to: determine an average value of the distance between each of the at least one object and the host vehicle; and determine the extent of the ground plane based on the determined average distance. In this way, distortion of objects in any image or representation of the vehicle's surroundings can be reduced, for example by preventing 3D objects near the vehicle from being mapped onto the ground plane (which can indicate to the user that the objects are lying on the ground). Therefore, a more realistic representation of the vehicle's surroundings can be provided, and safety can therefore be improved.

[0016] Furthermore, in various aspects of the invention, by considering the average distance between all objects and the vehicle, objects closer to the vehicle (which are typically more important to the user) can be better represented to the user without overly distorting objects farther away in the resulting image or representation. Thus, a balanced reduction in object distortion can be achieved. Optionally, the extent of the ground plane is positively correlated with the determined average distance; for example, if objects are farther away from the vehicle on average, the ground plane can have a larger extent than if objects are closer to the vehicle. Thus, a more realistic representation of the vehicle's surroundings can be provided.

[0017] Optionally, in any of the above aspects, the one or more geometric parameters include one or more of the radius, semi-major axis or semi-minor axis of the ground plane. For example, depending on the size and geometry of the vehicle, the ground plane may be elliptical or circular. By selecting a ground plane shape that fits the geometry of the vehicle, the entire surroundings of the vehicle may be represented in a consistent manner. More specifically, when the virtual projection surface has a uniform or regular geometry, the entire surroundings of the vehicle may be represented in a consistent manner. This method of consistent object representation around the vehicle may improve practicality for the user, and therefore may improve safety. It will be understood that any other suitable shape of the ground plane may be provided, and the range may be any suitable geometric parameter representing the size of the shape.

[0018] Optionally, in order to aggregate object data, the method includes or the processor is configured to: determine the distribution of at least one object around the host vehicle. Optionally, in order to determine the one or more geometric parameters, the method includes or the processor is configured to: determine the range of the ground plane according to the determined distribution. This arrangement can allow the ground plane to extend to a position that is closer to where most objects are located, thereby providing a better representation of the aggregated objects without excessively distorting isolated or individual objects. Therefore, a balanced reduction in object distortion can be achieved. Optionally, the range is negatively correlated with the determined distribution. For example, for a given average object distance, if the objects are distributed around a larger proportion of the vehicle's surroundings, the ground plane may extend closer than when the objects are distributed around a smaller proportion of the vehicle's surroundings. This can provide a more balanced representation of the vehicle's surroundings and help reduce object distortion.

[0019] Optionally, in any of the above aspects, the captured object data indicates that at least one object exists in at least one of the multiple areas near the host vehicle (i.e., in one of the multiple areas). In order to determine the distribution of the at least one object around the host vehicle, the method includes or the processor is configured to: determine the ratio of the number of areas including at least one object to the total number of areas based on the captured object data. In another embodiment, the method includes or the processor may be configured to: determine the ratio of the number of areas including at least one object to the number of areas not including at least one object. Such calculations may be more efficient in terms of calculation than alternative methods of determining the distribution, which may improve the use of the system in real-time applications. Optionally, the range is negatively correlated with the determined ratio; for example, if the objects are distributed around 75% of the vehicle, the ground plane may be smaller than when the objects are distributed around 25% of the car. This can help to appropriately better represent larger or more tightly clustered (one or more) objects identified in the vehicle's surroundings to the user. This can improve safety.

[0020] Optionally, in any of the above aspects, the one or more geometric parameters include an offset of the center of the virtual projection surface relative to the center of the host vehicle. In order to determine the one or more geometric parameters, the method includes or the processor is configured to: determine the offset according to the determined distribution. The offset can help provide a virtual projection surface that helps better represent most objects near the vehicle without distorting other objects due to unnecessary adjustment of the range of the ground plane. The arrangement can provide the user with an improved balance of visual representations of information about properties of near objects and far objects.

[0021] Optionally, in an embodiment, the method includes or the processor is configured to: generate an image based on the mapped image texture; or provide the mapped image texture to another vehicle system for generating an image based on the mapped image texture. The image is a visual representation of the vehicle's surroundings, and may include a representation of the vehicle itself (e.g., a model of the vehicle) optionally located on a ground plane. Generating the image may optionally include applying perspective to the mapped image texture. For example, the mapped image texture may be shown in a plan view to generate the image, or in a perspective view.

[0022] Optionally, the processor includes: one or more electronic processors having an electrical input for receiving the captured object data and optionally for receiving the captured image data; and an electronic storage device electrically coupled to the electronic processor and having instructions stored therein. The processor is configured to perform any of the above operations including the one or more electronic processors being configured to access the storage device and execute the instructions stored in the storage device so as to be able to operate to perform the above operations.

[0023] Optionally, the vehicle comprises a display configured to display an image in the camera view. Optionally, the display is configured to display an image generated in accordance with the mapped image texture in the camera view. Thus, the surroundings of the vehicle can be represented to the user. Optionally, the vehicle comprises one or more further vehicle systems arranged to: appropriately generate a virtual projection surface in accordance with the one or more output geometric parameters, and / or determine an image texture in accordance with the captured image data and map the image texture onto the generated virtual projection surface.

[0024] Optionally, the method includes or the processor is configured to: receive the captured object data and determine one or more geometric parameters of the virtual projection surface before the display enters the camera view. In other words, before activating the camera view, the geometric parameters are determined (e.g., calculated or selected), just as the camera view is activated. Figure 1 This partially dynamic arrangement can facilitate real-time use of the image system without requiring the computational resources required for a fully dynamic system that modifies the virtual projection surface on the fly. In addition, user engagement can be improved because the perspective and boundaries of the image displayed to the user do not continuously change (which could be distracting and potentially unsafe). Therefore, the practicality of the above aspects can be improved.

[0025] The geometrical parameters of the virtual projection surface are determined based on properties of objects surrounding the vehicle in order to reduce distortion of the objects in a subsequent visual representation or image of the vehicle surroundings.

[0026] It is expressly intended that within the scope of the present application, the various aspects, embodiments, examples and alternatives set forth in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular individual features thereof, may be employed independently or in any combination. That is, all embodiments and / or features of any embodiment may be combined in any manner and / or combination, unless such features are incompatible. Applicants reserve the right to amend any originally filed claim or to submit any new claim accordingly, including the right to amend any originally filed claim to be subordinate to any other claim and / or to incorporate any features of any other claim, even though protection was not originally claimed in that manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] One or more embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0028] Figure 1 A block diagram of an image system according to an embodiment of the present invention is shown;

[0029] Figure 2 Shown include Figure 1 A block diagram of an image system of a vehicle and the vicinity of the vehicle;

[0030] Figure 3A and Figure 3B shows the distribution of objects according to an embodiment of the present invention— Figure 3A shows an example distribution of objects around a vehicle, and Figure 3B A schematic representation of an area around a vehicle for determining distribution of objects is shown;

[0031] Figure 4A and Figure 4B A schematic representation of a virtual projection surface according to an embodiment of the present invention is shown - Figure 4A The virtual projection surface is shown in plan view, and Figure 4B The virtual projection surface is shown in side view;

[0032] Figure 5 shows a schematic representation of the offset of the virtual projection surface according to an embodiment of the present invention;

[0033] Figure 6 shows a schematic representation of the effect of the curvature of the virtual projection surface on the resulting representation of an object according to an embodiment of the present invention;

[0034] Figure 7 A flow chart showing a method according to an embodiment of the present invention;

[0035] Figure 8A flowchart showing a method according to an embodiment of the present invention; and

[0036] Fig. 9 A vehicle according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0037] In this article, refer to Figure 1 and 2 An image system 100 according to an embodiment of the present invention is described.

[0038] Reference Figure 1 , the (image) system 100 includes a processor 102 and a plurality of sensors 104. The system also includes one or more image capture devices 106. The processor 102 is communicatively coupled to the one or more image capture devices 106 and the plurality of sensors 104.

[0039] Each image capture device 106 may be a camera or a video camera, or any other device configured to capture image data. Where more than one image capture device 106 is provided, the devices 106 need not be the same type of device.

[0040] Each of the plurality of sensors 104 is configured to capture object data indicating the presence of one or more objects near the sensor. Such object data may be captured by any suitable sensing device. For example, an object may be sensed by a proximity sensor 104 using pressure waves (e.g., sound waves), electromagnetic waves (e.g., ultraviolet light, visible light, or near infrared light), or any other suitable sensing device. In the embodiments described herein, ultrasound is used to sense the presence of an object, but, for example, LIDAR may be used as an alternative.

[0041] Reference Figure 2 , the image system 100 is disposed within a host vehicle 200. Each of one or more image capture devices 106 is associated with the vehicle 200. The image capture devices 106 (shown here as two devices 106a, 106b) may be appropriately distributed around the interior or exterior of the vehicle. In the present embodiment, the image capture devices 106a, 106b are disposed at the front and rear of the vehicle 200, respectively; image data for the sides of the vehicle may then be added from these front and rear image capture devices 106. In other arrangements, the image capture devices may be disposed at any suitable location of the vehicle 200, i.e., appropriately disposed at the front, rear and / or sides of the vehicle, as well as in the center of the vehicle, along one or more sides of the vehicle, or at one or more corners of the vehicle.

[0042] The image capture device 106 is configured to capture image data of the environment or surroundings of the vehicle 200. In this embodiment, the device 106 is configured to capture image data representing one or more of the objects 108, 110, and 112 distributed around the vehicle 200. Each of the one or more image capture devices 106 captures image data from the environment of the device; the range of the environment covered by each device and from which the device captures image data depends on the type of image capture device 106. For example, a fisheye lens has a very wide viewing angle or field of view, and therefore can capture a large portion of the environment surrounding the vehicle 200, while other types of lenses may have a smaller field of view.

[0043] In this embodiment, device 106a captures image data about object 112, and device 106 captures image data about objects 108 and 110. Processor 102 is configured to receive image data of the vehicle environment from one or more image capture devices 106. The resulting image data from the two image capture devices is then stitched together by processor 102 to provide an image texture for generating an image representing all objects near the vehicle. The image texture represents the pixels of the stitched image captured by image capture device 106 and is a 2D representation of the surroundings of vehicle 200 including objects 108, 110, and 112.

[0044] Each of the sensors 104 is associated with the vehicle 200 and can be located at any suitable position of the vehicle. In this embodiment, the sensor 104 is a parking sensor of the vehicle 104, and ultrasonic waves are used to detect the presence of an object. This arrangement may be advantageous because it does not require an additional sensor arrangement to be assembled to the vehicle 200 to implement the system 100. Optionally, when a parking sensor is used as the sensor 104, the sensor can detect an object up to 5m or 6m from the vehicle, but only the object data captured from the object within 3m (optionally 2m) from the vehicle will be used in any subsequent operation. This may be because the accuracy of the sensor will decrease at a larger object distance. However, it should be understood that the sensor 104 can alternatively be a dedicated sensor of the system 100, and any suitable sensing technology can be used. When the sensor 104 is different from the image capture device 106, that is, when a different technology is used to detect the presence of objects around the vehicle, object data about the presence of the object can be captured even when the visibility of the objects around the vehicle is poor. This can improve safety by improving the representation of the object.

[0045] The sensor 104 is configured to capture object data indicating the presence of at least one object near the vehicle 200. In this embodiment, the sensor 104 is configured to capture object data indicating objects 108, 110, and 112 around the vehicle 200. The object data may indicate the width of each object, the height of each object, the distance between each object and the vehicle 200, etc. The objects 108, 100, 112 may be one or more persons, bollards, walls, or any other objects.

[0046] The processor 102 is configured to receive object data indicating the presence of one or more objects near the vehicle from the sensor 104. The processor is configured to aggregate the captured object data to determine one or more geometric parameters of a virtual projection surface to which an image texture can be mapped based on the aggregated data (this aggregation and determination of geometric parameters will be described in more detail below). By modifying the geometric parameters of the virtual projection surface based on one or more objects whose presence is detected by the sensor 104, the distortion of the vehicle environment in any resulting image can be reduced. It should be understood that the aggregated object data can be used to calculate the geometric parameters, or select one or more geometric parameters from a pre-existing set of parameters stored in a memory. For example, the parameters can be stored in a lookup table, or accessed in any other suitable manner.

[0047] The geometric parameters may be calculated or selected to reduce parallax and other image artifacts in any image generated from the mapped image texture. Such artifacts may occur because different image capture devices have different fields of view and thus project a single real-world object differently; this may result in blurring of the generated image, or duplication (parallax) or elimination of one or more objects. Such distortions may be reduced by determining the geometric parameters of the virtual projection surface from the object data, and by stitching together the resulting image data from the various image capture devices to match the curved surface geometry of the virtual projection surface. In particular, when forming the image texture, the image data may be stitched together in different places to account for the 3D curved nature of the virtual projection surface onto which the image texture is to be mapped.

[0048] Aggregation of object data may also provide a more robust image system 100 that may operate more effectively under adverse conditions. When one or more sensors are unable to operate properly, such as due to bad weather or a malfunction, using the aggregated data may help compensate for the reduction in the number of fully operational sensors, and may therefore allow for the identification of properties and parameters of objects surrounding the vehicle 200 in a more robust manner. Additionally, in the event that object data cannot be effectively captured, the system 100 may revert to a default, predetermined, virtual projection surface to ensure that the user has some visibility of their surroundings, even if the resulting image is slightly distorted.

[0049] The mapped image texture can be used to generate an image or representation of the environment around the vehicle 200. In particular, the pixels or textures of the 2D image captured by the image capture device 106 and stitched by the processor 102 are respectively mapped to the generated 3D virtual projection surface to form the mapped texture. The mapped texture can then be displayed to the user from a certain angle (e.g., in a plan view or a side view) to generate an image of the vehicle's environment.

[0050] It should be understood that in other examples, the system 100 may include only the processor 102 and the plurality of sensors 104. The object data captured by the sensors 104 may be aggregated and processed by the processor to generate one or more geometric parameters. The geometric parameters may then be output to another vehicle system of the vehicle 200 to generate a virtual projection surface, or the system 100 may generate a virtual projection surface based on the one or more geometric parameters and output the virtual projection surface to another vehicle system of the vehicle 200 for subsequent mapping of image textures.

[0051] In reference Figure 1 and Figure 2 In the depicted embodiment, the processor 102 configured to aggregate object data includes the processor 102 configured to determine an average value of the distance between each object and the vehicle 200. Figure 2 , object 108 is closer to vehicle 200 than object 110, and object 110 is closer to vehicle 200 than object 112. An average distance of these objects determined based on the object data captured by sensor 104 may be calculated by processor 112.

[0052] In some embodiments, the processor may aggregate object data by weighting. The processor may aggregate object data by weighting the distances of objects 108, 110, 112 according to the proximity of each object to the vehicle 200 (e.g., by appropriately giving a weight greater than or less than the distance to object 112 to the distance of object 108). For example, the processor may aggregate object data by weighting objects 108, 110, 112 based on the size and / or number of objects in a given area. The processor may aggregate object data by weighting objects according to one or more priority areas around the vehicle (e.g., objects in the areas in front of and / or behind the vehicle may be emphasized by weighting objects in the areas in front of and / or behind the vehicle higher than objects in the areas on the sides of the vehicle). Such weighting may be used in any suitable combination, and the weighting of object data may be combined with any other suitable aggregation method to aggregate object data.

[0053] In reference Figure 1 and Figure 2 In the depicted embodiment, the processor 102 configured to aggregate object data includes the processor 102 configured to determine the height of each object around the vehicle 200. For example, as determined by the object data captured by the sensor 104, the object 108 may have a height H1, the object 110 may have a height H2, and the object 112 may have a height H3. The heights H of the objects 108, 110, 112 are Figure 2 The processor 102 may also be configured to aggregate the object data to determine the width W of each object in addition to or instead of the height H. Figure 2 , each object 112 has a width W1, which is smaller than the width W2 of the object 108, which is smaller than the width W3 of the object 110. It should be understood that the width W or height H of the object is perceived by one or more corresponding sensors 104, rather than the actual width or height of the object. For example, depending on the orientation of the object relative to the vehicle 200, the object may be perceived as having a different height and / or width.

[0054] In reference Figure 1 and Figure 2 In the depicted embodiment, the processor 102 configured to aggregate object data includes the processor 102 configured to determine the distribution of objects around the vehicle 200. For example, the processor 102 may determine the proportion of the surroundings of the vehicle 200 that are occupied by the objects 108, 110, 112. This may be performed in any suitable manner.

[0055] Reference Figure 3A and Figure 3B, the processor 102 configured to determine the distribution of objects around the vehicle 200 is described in more detail according to an embodiment. Figure 3A Seven different arrangements of objects 314 around vehicle 200 are shown. Objects 314 may include one or more of objects 108, 110, 112 described above, or any other object. Object data captured by sensor 104 (sensor not shown) of vehicle 200 indicates the presence of at least one object 314 in at least one of a plurality of areas around vehicle 200 that are within a plurality of areas around vehicle 200. Figure 3B According to this embodiment, for example, if the object 314 is distributed around 75% of the vehicle 200 or 25% of the vehicle, it can be determined how much of the vehicle the object is distributed around.

[0056] Processor 102 is configured to aggregate the object data to determine a number of areas around the vehicle where the presence of object 314 has been detected, and determine a ratio between the number of areas determined to include at least one object and the number of areas (i.e., the total number or maximum number of areas where sensor 104 can detect object 314) based on the captured object data. Figure 3B In the example of , there are a total of 32 regions (the maximum number of regions), and 17 regions (R5 to F12) contain at least a portion of object 314 - i.e., object 314 is distributed around approximately 53% of the vehicle. Any other suitable ratio may additionally or alternatively be determined, for example, processor 102 may be configured to determine a ratio of the number of regions that include at least one object to the number of regions that do not include at least one object. Such calculations may be computationally more efficient than alternative methods of determining the distribution, which may facilitate improved use of the system in real-time applications.

[0057] Reference Figure 4A and Figure 4B , the properties and parameters of the virtual projection surface 400 are described in more detail.

[0058] Reference Figure 4A, (virtual projection) surface 400 includes ground plane 410. Ground plane 410 represents the surface on which the main vehicle (e.g., vehicle 200) is parked. Ground plane 410 can represent the surface of a road or lane, for example. The size of ground plane 410 can be described by range 430. Range 410 represents the geometric range (i.e., size) of ground plane 410, and can be any suitable parameter of the specific geometry or shape of ground plane 410. Here, ground plane 410 is elliptical in shape and the range defines the semi-major axis of the ellipse. However, as required, the range as defined herein can additionally or alternatively define the semi-minor axis of the ellipse. Optionally, in the case where ground plane 410 is circular, the range can be the radius of the circle. Optionally, in the case where ground plane 410 is rectangular, the range can be the width or length of the rectangle, or the diagonal of the rectangle, or any other suitable definition of the geometric shape of the rectangle. The center 440 of ground plane 410 can be defined as the geometric center of the ground plane.

[0059] Reference Figure 4B , the virtual projection surface 400 includes a curved surface 420 beyond the ground plane 410. The curved surface 420 has a curvature 450. The curved surface can have any suitable curvature. In some embodiments, the curved surface 420 can be parabolic. Here, the curved surface 420 is uniform around the ground plane. In other words, the curvature 450 is constant throughout the entire process around the virtual projection surface.

[0060] As described below, using the system 100, the extent 430 and center 440 of the ground plane 410, and the curvature 450 of the curved surface 420, can be determined from the aggregated data, along with any other suitable geometric parameters of the surface 400. As described above, by taking into account the properties or parameters of the objects surrounding the vehicle, the system 100 can generate a virtual projection surface 400 suitable for mapping the image texture generated from the captured image data so that a single image of the surroundings of the vehicle 200 can be provided with reduced parallax and other distortions of the objects.

[0061] In one embodiment, the range 430 of the ground plane 410 can be determined based on the determined average distance between the objects 314 and the vehicle 200. This arrangement can allow, for example, to reduce the distortion of objects in any image of the vehicle or the representation of the surrounding environment by preventing 3D objects near the vehicle from being mapped to the ground plane 410. Additionally or alternatively, the range 430 of the ground plane 410 can be determined based on the determined distribution of objects 314 around the vehicle 200. This arrangement can allow the ground plane 410 to extend to a position closer to the location where most objects 314 are located. In other words, the point at which the curvature 450 of the virtual projection surface begins is adjusted. This can provide a better representation of aggregated objects without overly distorting isolated or single objects; thus a balanced reduction in object distortion can be achieved.

[0062] In one embodiment, the extent of the ground plane may be positively correlated with the determined average distance, such that on average, if the object 314 is farther from the vehicle 200, the ground plane 410 has a greater extent 430 than if the object is closer to the vehicle. Thus, a more realistic representation of the surroundings of the vehicle 200 may be provided because the curved portion 420 of the virtual projection surface 400 will begin farther from the vehicle. The curvature 450 of the curved portion 420 may also be adjusted or changed so that the overall extent 470 of the surface 400 remains the same regardless of the extent 430 of the ground plane, or the curvature 450 of the curved portion may remain the same so that the extent 470 of the surface 400 increases as the extent 430 of the ground plane 410 increases.

[0063] In one embodiment, the extent of the ground plane may be negatively correlated to the determined distribution, such that for a given average object distance, if objects 314 are distributed around a large portion of the vehicle 200, the ground plane 410 may have a smaller extent 410 than if the objects are distributed around a small portion of the vehicle 200. This may provide a more balanced representation of the vehicle surroundings and help reduce distortion of objects 314. For example, if objects are distributed around 75% of the vehicle, the ground plane may be smaller than if the objects are distributed around 25% of the vehicle, in order to appropriately better represent to the user larger or more tightly clustered objects identified in the vehicle surroundings. This may improve safety.

[0064] Example

[0065] In one embodiment, the extent (E) 430 of the ground plane 410 may be defined as:

[0066] E=O d +(E max ×(1-(O c ×C))

[0067] in:

[0068] -O d is the average distance of objects 314 detected by sensor 104,

[0069] -E max is the maximum extent of the ground plane (this can be predetermined and application specific),

[0070] -O c is the percentage concentration (i.e., distribution) of objects around the vehicle, and

[0071] -C is an adjustable parameter for adjusting the sensitivity to the object distribution, which can be predetermined and application specific.

[0072] For example, when O d =0.5m, E max =10m, C=1.2:

[0073] - If O c =75%, E=1.5m;

[0074] - If O c =50%, E=4.5m;

[0075] - If O c =25%, E=8m.

[0076] Reference Figure 5 , an offset 460 of a center 440 of a virtual projection surface 400 may be determined based on the aggregated object data. The offset 460 defines a translation of the virtual projection surface 400. For example, the surface 400 may be offset by an amount 460 from a center position 440 coinciding with the center of the vehicle 200 to a position 440′ (which defines the center of the offset surface 400′).

[0077] In one embodiment, the processor 102 is configured to determine an offset 460 based on the determined distribution. The offset 460 can be used to "shift" the surface 400 toward most objects so that the curvature starts at a different point without distorting other objects by unnecessarily adjusting the extent of the ground plane. This arrangement can facilitate providing a virtual projection surface that facilitates better representation of most objects near the vehicle and can therefore provide the user with an improved balance in the visual representation of both near and far objects.

[0078] Example

[0079] In one embodiment, the extent (E) 430 of the ground plane 410 may be defined as:

[0080]

[0081] in:

[0082] -O d is the average distance of all objects 314 detected by sensor 104 from vehicle 200,

[0083] - Occupied sensor area is the number of areas around the vehicle where the object 314 is detected,

[0084] -Sensor area is the maximum number of areas in which the sensor 104 can detect the object 314 (i.e., the total number of areas, such as Figure 3B (shown)

[0085] -E max is the maximum extent of the ground plane (this can be predetermined and application specific),

[0086] - C is an adjustable parameter for adjusting the sensitivity to the distribution of objects, which can be predetermined and application specific. In other words, C is a sensitivity factor that affects how much the change in ground plane range varies with the distribution (here, the percentage of sensor area occupied), and

[0087] The adjustable offset is an adjustable parameter that affects the offset 460 of the virtual projection surface 400 .

[0088] Reference Figure 6 , depicts the effect of a change in the curvature 450 of the curved portion 420 of the virtual projection surface 400. Curvature is an important parameter of the virtual projection surface 400. When objects 314 around the vehicle 200 are above ground level 410, the resulting image or representation of the vehicle's surroundings may be distorted; this is because once the image texture representing these objects is mapped to the virtual projection surface 400, these objects will follow the curvature of the virtual projection surface 400. In particular, depending on the curvature 450 of the curved portion 420 of the surface 400, the objects may appear flat, as if they are on the ground, or may appear highly curved. Figure 6 A parabolic shaped virtual projection surface (or parabolic "bowl") for mapping an image texture is shown.

[0089] In particular, Figure 6 A shows the effect of a small curvature in a curved surface relative to the ground plane 410, Figure 6 B shows the effect of moderate curvature in a curved surface relative to ground plane 410, and Figure 6C shows the effect of large curvature in a curved surface relative to the ground plane 410. The object and surrounding scene captured by the image capture device 106 are the same in each scene (top row of images). However, the curvature of the curved surface 450 can have a significant effect on how the captured image data of the object 314 is represented in the resulting image generated based on the image texture mapped to the virtual projection surface 400. In the case of small curvature, the resulting image ( Figure 6 The object 314 is shown as being highly distorted in the bottom row of images of FIG. 4A . However, as the curvature 450 of the projection surface 400 increases relative to the ground plane 410, the distortion decreases. In the case of maximum curvature, in the resulting image ( Figure 6 The bottom row of images of C) shows the objects in a more representative manner.

[0090] The curvature 450 of the virtual projection surface can be determined based on the aggregated object data. In one embodiment, the processor 102 is configured to determine the curvature 450 based on the height (H) of the object 314 around the vehicle 200 determined. If the object is high (large H), increasing the steepness of the curvature of the surface 420 will prevent the object from looking as if it is lying on the ground. However, for shorter objects, the curvature 450 can be reduced. In this way, the curvature 450 of the virtual projection surface 400 onto which the image texture representing the image is mapped can be modified to reduce the curvature of the object in the resulting image. Therefore, distortion can be reduced, and safety can therefore be improved.

[0091] Please refer to the attached Figure 7 Methods according to embodiments of the present invention are described. Method 700 is a method for generating a virtual projection surface, optionally using an imaging system 100 of a vehicle (eg, vehicle 200).

[0092] Method 700 includes capturing 710 object data indicating the presence of at least one object near the vehicle. Operation 710 may occur at any suitable time during operation of the vehicle. When method 700 is performed using vision system 100, object data may be captured by sensor 104, as described above.

[0093] The method includes aggregating 720 the object data captured at step 710, and determining one or more geometric parameters of the virtual projection surface based on the aggregated object data at operation 730. Operations 710, 720, and 730 may be repeated any suitable number of times without performing any other operations. For example, geometric parameters may be continuously calculated based on newly captured object data so that the geometric parameters reflect the current surroundings of the vehicle for possible use in subsequent operations. This may facilitate the use of method 700 in real-time applications because waiting time may be reduced. The process may optionally be repeated until a camera view is selected.

[0094] In other words, before activating the camera view, the geometric parameters are determined (e.g., calculated or selected) as if the camera view were to be activated. This partially dynamic arrangement can facilitate real-time use of the image system without requiring the computing resources required for a fully dynamic system that modifies the virtual projection surface on the fly. In addition, since the perspective and boundaries of the image displayed to the user do not change continuously (which may be distracting and potentially unsafe), user engagement can be improved. To achieve this partially dynamic operation, the sensor can be arranged to capture object data continuously (or periodically at any suitable frequency, for example at a frequency between 0.1 and 150 times a second, optionally at a frequency between 1 and 15 times a second, such as at a frequency of 2 times a second, 5 times a second, 9 times a second, 14 times a second, etc.), and determine one or more geometric parameters based on the aggregated object data.

[0095] After selecting the camera view, the method 700 also includes generating 740 a virtual projection surface based on the one or more geometric parameters determined at operation 730. In some embodiments, the method includes capturing 750 image data indicative of the environment of the vehicle. Operation 750 may occur together with or separately from operation 710. Operation 750 may occur at any suitable time during vehicle operation. The method may also include determining 760 an image texture based on the captured image data and mapping 770 the image texture onto the generated virtual projection surface. The image texture may also be determined based on object data or in consideration of object data; for example, the image texture may be determined based on one or more geometric parameters. The method 700 may also include generating and displaying 780 an image based on the mapped image texture for display to a user.

[0096] If a camera view is not selected, operations 740 to 780 may not be performed and the image may not be displayed to the user. However, it will be understood that in method 700, a camera view may be selected at any suitable time; that is, any suitable combination of operations 710 to 780 may be repeated until a camera view is selected by the user, at which point the remaining operations of method 700 may be performed so that an image may be generated and displayed to the user.

[0097] Please refer to the attached Figure 8 A method according to an embodiment of the invention is described. Method 800 is a method for generating a virtual projection surface, optionally using an imaging system 100 of a vehicle (eg, vehicle 200).

[0098] As described above with respect to method 700, method 800 includes capturing 810 object data indicating the presence of at least one object near the vehicle. Operation 810 may occur at any suitable time during vehicle operation. When method 800 is performed using image system 100, object data may be captured by sensor 104, as described above. The method includes aggregating 820 the object data captured at operation 810, and determining one or more geometric parameters of a virtual projection surface based on the aggregated object data at operation 830.

[0099] The above description of operations 710 to 730 applies to operations 810, 820, and 830, which may be repeated any suitable number of times without performing any other operations. For example, geometric parameters may be continuously calculated based on newly captured object data so that the geometric parameters reflect the current surroundings of the vehicle for possible use in subsequent operations. This may facilitate the use of method 800 in real-time applications because latency may be reduced. The process may optionally be repeated until a camera view is selected.

[0100] The method 800 also includes generating 840 a virtual projection surface based on the one or more geometric parameters determined at operation 830. In some embodiments, operation 840 may be repeated any suitable number of times without performing any other operations. For example, the virtual projection surface may be repeatedly determined or calculated based on newly captured object data so that the surface reflects the current surroundings of the vehicle for possible use in subsequent operations. This may facilitate the use of the method 800 in real-time applications because latency may be further reduced. The process may optionally be repeated until a camera view is selected.

[0101] In some embodiments, the method includes capturing 850 image data indicative of the environment of the vehicle. Operation 850 may occur together with or separately from operation 810. Operation 850 may occur at any suitable time during operation of the vehicle. The method may also include determining 860 an image texture based on the captured image data, and mapping 870 the image texture onto the generated virtual projection surface. The image texture may also be determined based on object data or taking into account object data; for example, the image texture may be determined based on one or more geometric parameters.

[0102] Steps 850, 860, and 870, together with any of steps 810 to 840, may be repeated any suitable number of times without performing any other operations. For example, image textures may be repeatedly mapped onto the generated virtual projection surface based on newly captured object data and newly captured image data so that the mapped image textures reflect the current surroundings of the vehicle for possible use in subsequent operations. In this way, when a user desires to use a camera view of a display of the vehicle, any image texture may be quickly and easily mapped to the generated virtual projection surface. This may facilitate the use of method 800 in real-time applications because waiting time may be reduced. Optionally, these operations may be repeated until a camera view is selected.

[0103] exist Figure 8 In an implementation of , all operations 810 to 870 are repeated until a camera view is selected. However, it will be understood that a camera view may be selected at any suitable time; that is, any suitable combination of operations 810 to 870 may be repeated until a user selects a camera view, at which point the remaining operations of method 800 may be performed so that an image may be generated and displayed to the user.

[0104] The method 800 may also include generating and displaying 880 an image based on the mapped image texture for display to a user, optionally for display following a (user) operation of selecting a camera view. If a camera view is not selected, no image is displayed, although it will be appreciated that an image may still be generated at operation 880. Optionally, the image of operation 880 may be repeatedly generated until a camera view is selected and the generated image may be displayed to the user; this may facilitate use of the method 800 in real-time applications, as latency may be reduced.

[0105] In both methods 700 and 800, it will be appreciated that after a user selects a camera view, any of the operations may be performed (repeatedly) so that the user may be provided with an up-to-date image of their environment. Optionally, the geometric parameters are fixed after the camera view is selected, or determined (e.g., calculated or selected) at a frequency lower than the frequency at which image data is captured. For example, image data may be captured continuously (or periodically at any suitable frequency, such as at a frequency of 0.1 to 150 times a second, optionally at a frequency of 1 to 15 times a second, such as at a frequency of 2 times a second, 5 times a second, 9 times a second, 14 times a second, etc.).

[0106] This arrangement can facilitate real-time use of the image system without requiring the computational resources required for a fully dynamic system that modifies the virtual projection surface on the fly. In addition, because the perspective and boundaries of the image displayed to the user do not continuously change (which could be distracting and potentially unsafe), user engagement can be improved while image data is captured at a high enough rate to provide the user with an image of their current environment, thereby improving safety.

[0107] Please refer to the attached Fig. 9 A vehicle according to an embodiment of the present invention is described. The vehicle may include any of the above-described features of vehicle 200. The vehicle includes an image system 100, which may be arranged in a manner to implement method 700 or method 800. In one embodiment, the vehicle includes a display configured to display an image generated according to the above-described mapped image texture in a camera view.

[0108] It should be understood that various changes and modifications can be made to the present invention without departing from the scope of the present application.

[0109] Certain aspects of the invention are summarized in the following numbered clauses.

[0110] 1. An image system of a host vehicle, the image system comprising:

[0111] one or more image capture devices associated with the host vehicle and configured to capture image data indicative of an environment of the host vehicle;

[0112] a plurality of sensors associated with the host vehicle and configured to capture object data indicative of the presence of at least one object in a vicinity of the host vehicle; and

[0113] a processor communicatively coupled to the one or more image capture devices and the plurality of sensors to receive the captured image data and the captured object data, the processor being configured to:

[0114] aggregating the object data captured by each of the plurality of sensors;

[0115] determining one or more geometric parameters of a virtual projection surface based on the aggregated object data;

[0116] generating a virtual projection surface according to the one or more geometric parameters;

[0117] determining an image texture from the captured image data; and

[0118] The image texture is mapped onto the generated virtual projection surface.

[0119] 2. An imaging system according to clause 1, wherein the one or more geometric parameters define an extent of a ground plane of the virtual projection surface, the ground plane representing a surface on which the host vehicle is located.

[0120] 3. An imaging system according to clause 2, wherein the virtual projection surface is curved beyond the ground plane.

[0121] 4. An imaging system according to clause 3, wherein, in order to determine the one or more geometric parameters, the processor is configured to:

[0122] determining a height of the at least one object; and

[0123] A curvature of the virtual projection surface beyond the ground plane is determined based on the height.

[0124] 5. An image system according to clause 3 or clause 4, wherein the virtual projection surface is a parabola that extends beyond the ground plane.

[0125] 6. An image system according to any one of clauses 2 to 5, wherein, in order to determine the one or more geometric parameters, the processor is configured to:

[0126] determining an average of the distances between each of the at least one object and the host vehicle; and

[0127] The extent of the ground plane is determined based on the determined average distance.

[0128] 7. An imaging system according to clause 5, wherein the range is positively correlated with the determined average distance.

[0129] 8. An imaging system according to any one of clauses 2 to 7, wherein, in order to aggregate the object data, the processor is configured to determine the distribution of the at least one object around the host vehicle, and wherein, in order to determine the one or more geometric parameters, the processor is configured to determine the extent of the ground plane based on the determined distribution.

[0130] 9. An imaging system according to clause 8, wherein the range is negatively correlated with the determined distribution.

[0131] 10. An image system according to any of clauses 2 to 9, wherein the one or more geometric parameters include one or more of a radius, a semi-major axis or a semi-minor axis of the ground plane.

[0132] 11. An image system according to any of clauses 1 to 7, wherein, in order to aggregate the object data, the processor is configured to determine a distribution of the at least one object around the host vehicle.

[0133] 12. An image system according to any one of clauses 8 to 11, wherein the one or more geometric parameters include an offset of the center of the virtual projection surface relative to the center of the host vehicle, and wherein, in order to determine the one or more geometric parameters, the processor is configured to determine the offset based on the determined distribution.

[0134] 13. An image system according to any of clauses 8 to 12, wherein the captured object data indicates the presence of at least one object in at least one of a plurality of areas in the vicinity of the host vehicle, and wherein, to determine the distribution of the at least one object around the host vehicle, the processor is configured to:

[0135] A ratio of the number of regions including at least one object to the number of regions is determined based on the captured object data.

[0136] 14. An image system according to any of the preceding clauses, wherein the processor comprises: one or more electronic processors having an electrical input terminal for receiving captured image data and captured object data; and an electronic storage device electrically coupled to the electronic processor and having instructions stored in the electronic storage device, wherein the processor is configured to map the image texture comprising: the one or more electronic processors being configured to access the storage device and execute the instructions stored in the storage device so that it can operate to determine the image texture, aggregate the object data, and determine the one or more geometric parameters of the virtual projection surface to generate the virtual projection surface based on the one or more geometric parameters, and map the image texture onto the generated virtual projection surface.

[0137] 15. A vehicle comprising an imaging system according to any preceding clause.

[0138] 16. A vehicle according to clause 15, comprising a display configured to display an image generated from the mapped image texture in a camera view.

[0139] 17. The vehicle of clause 16, wherein the processor is configured to receive captured object data and determine the one or more geometric parameters of the virtual projection surface before the display enters the camera view.

[0140] 18. A method for generating a virtual projection surface using an imaging system of a host vehicle, the method comprising:

[0141] capturing image data indicative of an environment of the host vehicle;

[0142] capturing object data indicating the presence of at least one object in proximity to the host vehicle;

[0143] Aggregate the captured object data;

[0144] determining one or more geometric parameters of a virtual projection surface based on the aggregated object data;

[0145] generating a virtual projection surface according to the one or more geometric parameters;

[0146] determining an image texture from the captured image data; and

[0147] The image texture is mapped onto the generated virtual projection surface.

[0148] 19. A non-transitory computer-readable medium comprising computer-readable instructions that, when executed by a processor, cause the performance of the method of clause 18.

Claims

1. An image system of a host vehicle, the image system include: one or more image capture devices associated with the host vehicle and configured to capture image data indicative of an environment of the host vehicle; a plurality of sensors associated with the host vehicle and configured to capture object data indicative of a plurality of objects being present in a vicinity of the host vehicle; as well as a processor communicatively coupled to the one or more image capture devices and the plurality of sensors to receive the captured image data and the captured object data, the processor being configured to: aggregating the object data captured by each of the plurality of sensors; determining geometric parameters of a virtual projection surface according to the properties of the plurality of objects indicated by the aggregated object data; generating a virtual projection surface according to the geometric parameters; determining an image texture based on the captured image data; as well as Mapping the image texture onto the generated virtual projection surface, wherein, to aggregate the object data, the processor is configured to determine a distribution of one or more of the objects around the host vehicle, and wherein, to determine the distribution of the one or more objects, the processor is configured to determine a proportion of the vehicle's surroundings occupied by the objects.

2. The imaging system according to claim 1, in: The geometric parameters define an extent of a ground plane of the virtual projection surface, the ground plane representing a surface on which the host vehicle is located; and The virtual projection surface is curved beyond the ground plane.

3. The imaging system according to claim 2, in, To determine the geometric parameters, the processor is configured to: determining heights of the plurality of objects; and A curvature of the virtual projection surface beyond the ground plane is determined based on the height.

4. The imaging system according to claim 2 or 3, in, The virtual projection surface is a parabola that extends beyond the ground plane.

5. The imaging system according to claim 2 or 3, in, To determine the geometric parameters, the processor is configured to: determining an average of the distances between each of the plurality of objects and the host vehicle; and The extent of the ground plane is determined based on the determined average distance.

6. The imaging system according to claim 2 or 3, in, To determine the geometrical parameter, the processor is configured to determine an extent of the ground plane based on the determined distribution.

7. The imaging system according to claim 2 or 3, in, The geometric parameters include one or more of a radius, a semi-major axis, or a semi-minor axis of the ground plane.

8. The imaging system according to claim 7, in, The geometrical parameter comprises an offset of a center of the virtual projection surface relative to a center of the host vehicle, and wherein, to determine the geometrical parameter, the processor is configured to determine the offset according to the determined distribution.

9. The imaging system according to claim 7, in, The captured object data indicates the presence of at least one object in at least one of a plurality of areas in the vicinity of the host vehicle, and wherein, to determine a distribution of the at least one object around the host vehicle, the processor is configured to: A ratio of the number of regions including at least one object to the number of regions is determined based on the captured object data.

10. The imaging system according to claim 8, in, The captured object data indicates the presence of at least one object in at least one of a plurality of areas in the vicinity of the host vehicle, and wherein, to determine a distribution of the at least one object around the host vehicle, the processor is configured to: A ratio of the number of regions including at least one object to the number of regions is determined based on the captured object data.

11. The imaging system according to any one of claims 1 to 3, in, The processor includes: one or more electronic processors, the electronic processor having an electrical input terminal for receiving captured image data and captured object data; and an electronic storage device, the storage device is electrically coupled to the electronic processor and has instructions stored in the storage device, wherein the processor is configured to map the image texture, including: the one or more electronic processors are configured to access the storage device and execute the instructions stored in the storage device so as to be able to operate to determine the image texture, aggregate the object data, and determine the geometric parameters of the virtual projection surface, so as to generate the virtual projection surface according to the geometric parameters and map the image texture onto the generated virtual projection surface.

12. A vehicle comprising a display and an image system according to any one of claims 1 to 11, the display being configured to display an image generated from the mapped image texture in a camera view.

13. The vehicle according to claim 12, in, The processor is configured to receive captured object data and determine the geometric parameters of the virtual projection surface before the display enters the camera view.

14. A method for generating a virtual projection surface using an image system of a host vehicle, the method include: capturing image data indicative of an environment of the host vehicle; capturing object data indicating the presence of a plurality of objects in a vicinity of the host vehicle; Aggregate the captured object data; determining geometric parameters of a virtual projection surface according to the properties of the plurality of objects indicated by the aggregated object data; generating a virtual projection surface according to the geometric parameters; determining an image texture based on the captured image data; as well as Mapping the image texture onto the generated virtual projection surface, wherein, to aggregate the object data, a distribution of one or more of the objects around the host vehicle is determined, and wherein, to determine the distribution of the one or more objects, a proportion of the vehicle's surroundings occupied by objects is determined.

Citation Information

Patent Citations

  • Bowl-shaped imaging system

    US20150178884A1

  • Viewing system and method for displaying an environment of a vehicle

    CN103295265A

  • Object visualization in bowl-shaped imaging systems

    CN106575432A

  • Systems and methods for producing an image visualization

    CN108140235A

  • Method for recognising function errors of a multi-sensor assembly

    EP2604478A1