System and method for projecting a projection image onto a vehicle surface

Abstract

System with - a motor vehicle (10) and - an unmanned aerial vehicle (12), as well as - a projector unit (14) in / on the unmanned aerial vehicle (12) designed to project a projection image (18) onto a surface; with - a sensor system (16) in or on the motor vehicle (10) and the unmanned aerial vehicle (12) which is designed to determine a relative position of the motor vehicle (10) to the unmanned aerial vehicle (12) or vice versa; and with - a computing unit in or on the motor vehicle (10) or the unmanned aerial vehicle (12) configured to calculate the projection image (18) using the relative position and vehicle-related data, wherein the surface is located on the motor vehicle (10) and the vehicle-related data characterize this surface, and the projector unit (14) configured to project the projection image onto the surface of the motor vehicle (10), wherein part of the sensor system (16) is arranged in the motor vehicle (10) and another part of the sensor system (16) is arranged in the unmanned aerial vehicle (12), wherein the relative position is determined via sensor data fusion from the two parts of the sensor system (16).

Description

[0001] The invention relates to a system for projecting a projection image onto the surface of a vehicle. Furthermore, the invention relates to a corresponding method.

[0002] From WO 2017 / 157863 A1, a system is known that performs one or more support operations for a land vehicle using an unmanned autonomous vehicle, such as a drone.

[0003] From WO 2017 / 041303 A1, systems, methods and devices are known for detecting and tracking one or more moving objects.

[0004] Techniques for controlling drone devices are known from US patent 2018 / 0029522 A1. These techniques include monitoring by a drone device that is operationally connected to a processor and assigned to a vehicle.

[0005] US patent 2016 / 0349746 A1 discloses an unmanned aerial vehicle with a projector on board for projecting information onto objects while the unmanned aerial vehicle is flying.

[0006] US patent 2016 / 0340006 A1 describes a drone that patrols above swimmers to detect a swimmer in distress.

[0007] Currently, there is no way to illuminate the vehicle from above or to trace its contours with projectors. 3D projection mapping is often used as a show effect by original equipment manufacturers (OEMs) or suppliers to showcase the vehicle's versatility or to highlight its contours.

[0008] Projection mapping is only feasible in darkened rooms and with professional equipment. Projection mapping requires positioning the projectors outside the vehicle being projected onto, which is currently only possible when the vehicle is stationary. The invention aims to provide projection mapping for moving vehicles.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are disclosed by the dependent patent claims, the following description, and the figures.

[0010] The invention provides a system comprising a motor vehicle and an unmanned aerial vehicle, as well as a projector unit in / on the unmanned aerial vehicle configured to project a projection image onto a surface. The system further includes sensors in or on the motor vehicle and / or the unmanned aerial vehicle configured to determine the relative position of the motor vehicle to the unmanned aerial vehicle or vice versa, and a processing unit in or on the motor vehicle or the unmanned aerial vehicle configured to calculate the projection image using the relative position and vehicle-related data, wherein the surface is located on the motor vehicle and the vehicle-related data characterize this surface, and the projector unit is configured to project the projection image onto the surface of the motor vehicle.In other words, the system consists of an unmanned aerial vehicle (UAV) equipped with a projector unit that projects an image onto the surface of a vehicle. Calculating the projected image onto the vehicle's surface requires determining the relative position of the vehicle to the UAV, or vice versa. This includes, for example, the distance, the difference in altitude, or, more generally, the coordinates, such as Cartesian or spherical coordinates, within a reference frame of reference for either the vehicle or the UAV. Furthermore, vehicle-related data, such as the vehicle's body dimensions, are necessary for calculating the projected image. The UAV, also known as a drone, can be, in particular, a quadcopter or a multicopter capable of carrying a projector unit.

[0011] The invention offers the advantage that projection mapping can be applied to a vehicle, particularly a moving vehicle, thus highlighting and showcasing the vehicle's contours, for example, during presentations and in road traffic. Additionally, no fixed positioning and alignment of projectors is necessary, saving time and money.

[0012] The invention also includes embodiments which provide additional advantages.

[0013] One embodiment provides for the projector unit to be a video projector or laser projector. The video projector, also known as a beamer, can be a CRT projector, LCD projector (liquid crystal projector), DLP projector (digital light processing), LED projector (light-emitting diode), or LCoS projector (liquid crystal on silicon). The advantage of a video projector or laser projector is that colored and moving images, videos, and animations can be displayed cost-effectively on a surface.

[0014] In another embodiment, the projector unit is further arranged in an automatically adjustable, rotatable and / or tiltable device of the unmanned aerial vehicle. In other words, the projector unit is arranged in an adjustable device, such as a housing, so that the projection direction of the projector unit can be adjusted by the device. This has the advantage that the projection direction is not dependent on the flight movement of the unmanned aerial vehicle. This means that in the event of flight movements or, for example, a gust of wind, the projector unit can be controlled by the automatically adjustable device in such a way that the movements of the unmanned aerial vehicle are compensated for and any shaking of the projected image is minimized.

[0015] Preferably, part of the sensor system is located in the motor vehicle and another part in the unmanned aerial vehicle (UAV), with the relative position being determined via sensor data fusion from both sets of sensors. In other words, the relative position is calculated from a combination of the sensors in the vehicle and the UAV. This offers the advantage of a much more precise determination of the relative position, contributing to increased accuracy of the projected image. The sensor system can include satellite data from a global navigation satellite system, such as GPS, GLONASS, Galileo, or BeiDou; optical or electromagnetic sensors, such as cameras, radar, or lidar; or it can determine the relative position from radio communication data, such as Bluetooth, Wi-Fi, or mobile network data.

[0016] Preferably, the vehicle-related data includes computer-aided design data of the motor vehicle. In other words, the vehicle-related data can be computer-aided design (CAD) data of the motor vehicle. This has the advantage that the exact dimensions of the vehicle are known via the design data, which improves the accuracy of the calculation of the projection image and thus leads to a better projection image.

[0017] One embodiment provides that the sensor system includes an image stabilization device, in particular a gyroscope, to achieve optical image stabilization of the projected image. In other words, the sensor system, especially the sensor system of the unmanned aerial vehicle, includes motion and acceleration sensors whose data are used for optical and / or digital image stabilization of the projected image. This has the advantage of preventing blurring of the projected image and thus resulting in a more stable image.

[0018] Another embodiment provides that the unmanned aerial vehicle's sensor system includes a camera that can supply the processing unit with image data of the vehicle, which is then taken into account when calculating the projected image. This means that the sensor system, particularly the sensor system of the unmanned aerial vehicle, includes a camera that uses images to adapt the projected image to the vehicle's geometry. The camera is preferably a digital video camera. This has the advantage that the projected image can be calculated and adapted directly to real-time data, resulting in an improved projected image.

[0019] Preferably, the processing unit is designed to calculate the projected image using one or more vehicle features and / or markers. In other words, the projected image is adapted to existing vehicle features and / or to the vehicle based on attached markers. Vehicle features include, in particular, the vehicle windows, a vehicle antenna, vehicle lights, or gaps between doors, the hood, or the trunk. Markers on the vehicle are preferably motion capture markers used for optical tracking. The advantage of this is that the processing unit can detect movements with relatively little time expenditure, thus contributing to an improvement of the projected image.

[0020] Another embodiment provides that the movements of the motor vehicle can be synchronized with the movements of the unmanned aerial vehicle. In other words, the unmanned aerial vehicle moves at the same speed and in the same direction as the motor vehicle. This means that when the motor vehicle accelerates, the unmanned aerial vehicle accelerates along with it and also follows the curves of the motor vehicle in such a way that the projection angle of the projector unit onto the motor vehicle remains constant. This has the advantage that the projection by the projector unit remains constant and therefore no new projection image needs to be calculated. This reduces the computing power required for calculating the projection image and further stabilizes the projection image on the motor vehicle.

[0021] The invention also provides a method for projecting a projection image onto a surface. The method determines the relative position of a motor vehicle to an unmanned aerial vehicle (UAV) and uses this relative position and vehicle-related data to calculate the projection image, wherein the surface is located on the motor vehicle and the vehicle-related data characterize this surface. Furthermore, the method projects the projection image from the UAV onto the surface of the motor vehicle. This offers the same advantages and variations as the system.

[0022] The invention also includes combinations of the features of the described embodiments.

[0023] The invention also includes further developments of the method according to the invention, which have features already described in connection with the further developments of the system according to the invention. For this reason, the corresponding further developments of the method according to the invention are not described again here.

[0024] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 a schematic representation of a side view of an embodiment of the system with a motor vehicle and an unmanned aerial vehicle; Fig. 2 a process diagram of an embodiment.

[0025] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0026] In the figures, identical reference symbols denote functionally equivalent elements.

[0027] Fig. Figure 1 shows, by way of example, a motor vehicle 10 and an unmanned aerial vehicle 12, which can be, for example, a drone. Preferably, the unmanned aerial vehicle 12 is a multicopter capable of hovering. The unmanned aerial vehicle 12 can be equipped with a projector unit 14 for projecting an image. Furthermore, the unmanned aerial vehicle can include sensors 16. These sensors can also be located in the motor vehicle 10. The components of the sensors 16 in the motor vehicle 10 and the unmanned aerial vehicle 12 can be different, the same, or partially the same. For example, the sensors in the vehicle can include a GPS sensor and a radar device, while the sensors 16 in the unmanned aerial vehicle 12 can include a GPS sensor, a lidar, and a camera.

[0028] The sensor system 16 can, in particular, determine the relative position of the unmanned aerial vehicle 12 to the motor vehicle 10. For example, a minimum distance can be one meter. A maximum distance can result from the ratio of the brightness of the projector unit 14 to the brightness of the surroundings.

[0029] The unmanned aerial vehicle 12 can be controlled, in particular, by an electronic control device (for example, a navigation system) of the motor vehicle 10, so that the unmanned aerial vehicle 12 automatically follows the motor vehicle 10. The motor vehicle 10 is in constant communication with the unmanned aerial vehicle 12. This communication can take place, for example, via Bluetooth or WLAN. Data, such as sensor data for determining the relative position, can be exchanged via this communication, and control commands can be sent that can cause the projector unit 14 to project a projection image 18 onto a surface of the motor vehicle 10.

[0030] For the projection of the projection image 18 by the projector unit 14, a processing unit (not shown) can be provided in the motor vehicle 10 or the unmanned aerial vehicle 12, which calculates the projection image 18 using the relative position determined by the sensor system 16 and vehicle-related data. The vehicle-related data can be CAD data of the motor vehicle 10, so that the projection image 18 can be precisely fitted to a surface of the motor vehicle 10. Additionally, the sensor system 16 of the unmanned aerial vehicle 12 can include a camera that captures images of the vehicle, preferably in real time, and makes them available to the processing unit. The camera can have the same orientation as the projector unit 14.In addition to the CAD data of the motor vehicle 10, the calculation unit can use the camera images to calculate a digital projection image, which the projector unit 14 can project onto the surface of the motor vehicle 10.

[0031] The projector unit 14 and / or the camera can be arranged in an automatically adjustable device on the unmanned aerial vehicle 12. This makes it easier to compensate for flight movements of the unmanned aerial vehicle 12. For example, the vehicle 10 and the unmanned aerial vehicle 12 can be synchronized, meaning that when the vehicle 10 turns, the unmanned aerial vehicle 12 also turns so that the projection image 18 continues to be projected onto the surface of the vehicle 10. Any tilting of the unmanned aerial vehicle 12 due to the turn can be compensated for by the automatically adjustable device, which is rotatable and / or tiltable. Furthermore, vibrations caused, for example, by a gust of wind can be compensated for.For stabilizing the projected image 18, the sensor system 16 can also include an image stabilization device comprising speed and acceleration sensors, for example, a gyroscope. This allows for optical image stabilization of the projected image 18. To improve the projected image 18, for example, to stabilize it, the processing unit can also calculate the projected image 18 based on vehicle features such as the headlights, side mirrors, vehicle windows, or vehicle doors.

[0032] The unmanned aerial vehicle 12 and the projector unit 14 enable 3D projection mapping to be realized on the surface of the vehicle 10, particularly when the vehicle 10 is in motion. This can be used as a show effect at trade fairs and presentations to demonstrate the versatility of the vehicle or to highlight its contours. The projected image 18 can also be used to enhance the safety of the vehicle 10.

[0033] For example, a lighting system of the vehicle 10, such as a turn signal or brake lights, could fail. If, for instance, the rear brake lights fail, the unmanned aerial vehicle 12 can position itself behind the vehicle 10 in such a way that, during braking, the projector unit 14 projects an image 18 of the brake lights onto the rear surface of the vehicle. This allows following vehicles to be warned even if the brake lights have failed. Thus, increased safety can be achieved.

[0034] However, other functional lighting elements of the vehicle 10, for example a taillight, could also be implemented with a projection image 18 to comply with the legal requirements for vehicle lighting and its signaling systems. For this purpose, a deflecting optic could be provided on the body of the vehicle 10, which redirects a projection image 18 to the corresponding locations on the vehicle 10. This has the advantage that, in the example of the taillight, the unmanned aerial vehicle 12 can also fly in front of and alongside the vehicle 10, and the projection image 18, which is projected onto the vehicle 10 or the vehicle's deflecting optic, is deflected to the rear of the vehicle. Furthermore, the deflecting optic allows for better control of the light's direction of emission, resulting in improved visibility.

[0035] It is also possible for the system to be used with the unmanned aerial vehicle 12 for projecting communication signals for human-machine communication between an autonomously driving vehicle 10 and a human road user. Unlike a human driver, a pedestrian in an autonomously driving vehicle does not receive feedback from the driver, such as eye contact, indicating whether the driver has seen the pedestrian at a crosswalk and is braking. In this case, an autonomously driving vehicle 10 can project an image 18 onto its surface to indicate whether the vehicle has detected the pedestrian and is initiating braking.

[0036] In Fig.Figure 2 shows a process diagram of one embodiment. In step S10, the relative position of a motor vehicle 10 to an unmanned aerial vehicle 12 is determined. Using this relative position, a projection image 18 onto the surface of the motor vehicle 10 can then be calculated in step S12 with vehicle-related data, in particular known surface data obtained from CAD data. Subsequently, in step S14, the projection image 18 can be projected from the unmanned aerial vehicle 12 onto the surface of the motor vehicle 10.

[0037] In another exemplary embodiment, a lighting drone (unmanned aerial vehicle 12) equipped with one or more projectors (projector unit 14) makes it possible not only to implement new lighting functions around the vehicle (motor vehicle 10). A projector (projector unit 14) can also project a wide variety of lighting effects onto the vehicle (motor vehicle 10) using light mapping. This results in a modern, innovative design element, and several lighting effects are possible, such as an extension of the Leaving and Coming Home function.Furthermore, contours can be highlighted on the vehicle using light mapping, allowing for a modified design according to customer requirements, thus increasing vehicle personalization. Communication symbols and warning messages for communicating with pedestrians and cyclists in autonomous vehicles can be projected, and emergency vehicles, such as "police" and "ambulance," can be identified. Additionally, light elements can be integrated into the vehicle without requiring any installation space, such as a sweeping turn signal as a projection, eliminating the need for physical installation.

[0038] An advantageous technical implementation is that the drone (unmanned aerial vehicle 12) can be equipped with one or more projector modules (projector unit 14) or a camera, the position of the drone (unmanned aerial vehicle 12) and the vehicle (motor vehicle 10) and their relationship to each other are known, the CAD dimensions (contours) of the vehicle (motor vehicle 10) are known, communication between the drone and the vehicle is ensured (e.g., Bluetooth, WLAN), and the image captured by the camera is constantly recalculated based on prominent points (e.g., joints, the antenna). This recalculation can then be used to mitigate strong camera shake. The intelligence of the system, i.e., the processing of the camera data and the generation of the projected image, can, for example, take place within the vehicle.

[0039] Overall, this example shows how the invention of projection mapping can be achieved using a flying light assistant to showcase one's own vehicle.

Claims

[1] System with - a motor vehicle (10) and - an unmanned aerial vehicle (12), and - a projector unit (14) in / on the unmanned aerial vehicle (12) which is designed to project a projection image (18) onto a surface; with - a sensor system (16) in or on the motor vehicle (10) and the unmanned aerial vehicle (12), which is designed to determine a relative position of the motor vehicle (10) to the unmanned aerial vehicle (12) or vice versa; and with - a calculation unit in or on the motor vehicle (10) or the unmanned aerial vehicle (12), which is designed to calculate the projection image (18) using the relative position and vehicle-related data, wherein the surface is located on the motor vehicle (10) and the vehicle-related data characterize this surface and the projector unit (14) is designed to project the projection image onto the surface of the motor vehicle (10), wherein part of the sensor system (16) is arranged in the motor vehicle (10) and another part of the sensor system (16) is arranged in the unmanned aerial vehicle (12), wherein the relative position is determined via a sensor data fusion from the two parts of the sensor system (16). [2] System according to claim 1, wherein the projector unit (14) is a video projector or laser projector. [3] System according to one of the preceding claims, wherein the projector unit (14) is further arranged in an automatically adjustable rotatable and / or tiltable device of the unmanned aerial vehicle (12). [4] System according to one of the preceding claims, wherein the vehicle-related data comprise computer-aided design data of the motor vehicle (10). [5] System according to one of the preceding claims, wherein the sensor system (16) comprises an image stabilization device, in particular a gyro sensor, in order to achieve optical image stabilization of the projection image (18). [6] System according to one of the preceding claims, wherein the sensor system (16) of the unmanned aerial vehicle (12) comprises a camera with which image data of the motor vehicle (10) can be provided to the calculation unit, which image data are taken into account when calculating the projection image (18). [7] System according to claim 6, wherein the projection image (18) can be calculated by the calculation unit with the aid of one or more vehicle features and / or markers. [8] System according to one of the preceding claims, wherein movements of the motor vehicle (10) can be synchronized with movements of the unmanned aerial vehicle (12). [9] Method for projecting a projection image onto a surface characterized by , that - a relative position of a motor vehicle to an unmanned aircraft is determined (S10); - the projection image is calculated using the relative position and vehicle-related data (S12), whereby the surface is located on the motor vehicle and the vehicle-related data characterise this surface; - the projection image is projected by the unmanned aerial vehicle onto the surface of the motor vehicle (S14), - wherein one part of a sensor system (16) is arranged in the motor vehicle (10) and another part of the sensor system (16) is arranged in the unmanned aerial vehicle (12), wherein the relative position is determined via a sensor data fusion from the two parts of the sensor system (16).

Images