Methods for processing video signals and motor vehicles

Abstract

The invention relates to a method for processing video signals. Acceleration signals (221) describing the current acceleration of the vehicle are generated from an acceleration sensor (22) of the motor vehicle and mapped to a change factor, wherein there is a positive or negative correlation between the acceleration signals (221) and the change factor. A parameterizable filter is applied to video signals generated by at least one camera (16) that captures at least part of the environment (26) of the motor vehicle (10). This filter shifts and / or selects pixels from still images (32) derived from a video obtained from the video signals, wherein the strength of the filter, which affects the shift and / or selection of pixels, correlates with the change factor. The processed video signals are output on at least one display device (12) of the motor vehicle.

Description

[0001] The invention relates to a method for processing video signals and a motor vehicle equipped to carry out the method.

[0002] A motor vehicle can include at least one display device. This display device can be, for example, a display, a monitor, a transparent display (e.g., mounted on a window of the motor vehicle), a head-up display, a projection onto an interior surface of the motor vehicle, virtual reality (VR) glasses, and / or augmented reality (AR) glasses.

[0003] The display device can show images to an occupant; for example, it can display the vehicle's surroundings. The vehicle's surroundings can be a predefined area around the vehicle.

[0004] To display the vehicle's surroundings on the display device, the video signals can be generated by at least one camera of the vehicle. This camera captures at least a portion of the vehicle's surroundings, the generated video signals are processed by a computing unit, and the processed video signals are output on at least one display device of the vehicle. This means that the vehicle includes at least one camera, which can be directed away from the vehicle so that its field of view includes a portion of the vehicle's surroundings. Each camera can capture video signals, and these signals can be combined to form a video. The video can then be played back via the display device.

[0005] US patent 2024 / 0181885A1 discloses a method in which an image is displayed which is changed in response to an acceleration variable.

[0006] DE 10 2019 220 055 A1 discloses a method for reducing motion sickness, wherein a representation is changed depending on a determined change in the center of gravity.

[0007] DE 10 2019 108 054 A1 discloses a method for displaying a selected display area, wherein a roadway angle characterizing a roadway is determined and, depending on this, the display area is selected and adapted to the roadway angle.

[0008] The purpose of the invention is to allow an occupant of the motor vehicle to intuitively perceive the current driving situation of the motor vehicle.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are described by the dependent patent claims, the following description, and the figures.

[0010] The invention provides a solution comprising a method for processing video signals. This processing involves mapping acceleration signals from a vehicle's accelerometer, describing the vehicle's current acceleration, onto a change factor. This means, for example, that the vehicle includes at least one acceleration sensor by which the vehicle's current acceleration can be determined and signaled using acceleration signals, according to a method known in the art. The acceleration signals can be received and / or processed by another component of the vehicle, such as a control unit and / or a processing unit.

[0011] According to the invention, the acceleration signals are mapped to the rate of change. This is preferably done in a processing unit that is integrated into the vehicle. Mapping the acceleration signals to the rate of change means, in particular, that the rate of change depends on the acceleration signals and thus on the current acceleration of the vehicle. Specifically, the rate of change is a function or the result of such a function, into which the acceleration signals, and thus the current acceleration, are incorporated as a function value and / or parameter. A change in the current acceleration can cause a change in the acceleration signals and thus a change in the rate of change.This means, in particular, that the change factor for a first current acceleration will have a different value than for a second current acceleration if the values ​​of the first current acceleration and the second current acceleration differ from each other.

[0012] There is a positive or negative correlation between the acceleration signals and the change factor. In particular, there is a positive or negative correlation between the current acceleration of the vehicle and the change factor.

[0013] The terms positive and negative correlation are a statistical measure known from the prior art and describe the relationship between a first value of a first quantity and a second value of a second quantity. In the example of the invention, the acceleration signals and / or the current acceleration can be specified as the first quantity, and the rate of change as the second quantity. With a positive correlation between the first and second quantities, the first value is larger as the second value is larger, and / or the second value is larger as the first value is larger. Additionally, the first value is smaller as the second value is smaller, and / or the second value is smaller as the first value is smaller. With a negative correlation between the first and second quantities, the first value is larger as the second value is smaller, and / or the second value is larger as the first value is smaller.Additionally, the first value is smaller the larger the second value is and / or the second value is smaller the larger the first value is.

[0014] It may therefore be intended that with a higher current acceleration, the change factor increases or decreases, especially continuously or in stages.

[0015] A programmable filter is applied to the video signals generated by at least one camera. This filter shifts and / or selects pixels from still images derived from the video extracted from these video signals. Specifically, this means that the video signals from each camera are combined to create a single video that encompasses, or is composed of, the video signals from all cameras.

[0016] It may be provided that a parameterizable filter is applied to the video. The parameterizable filter, in particular, processes and / or edits at least a portion of the video's still images and / or frames, whereby the editing and / or processing may include selecting and / or shifting individual pixels of the respective frame and / or still image. This means that applying the parameterizable filter results in an edited video, which is described by the edited video signals. The edited video may correspond to the video in which pixels were shifted and / or selected and combined to form the edited video. It may be provided that not just one parameterizable filter, but several parameterizable filters are applied to the video, and in particular, that the edited video is only created after the application of the multiple parameterizable filters.For example, several different filters described in the further developments of the invention can be applied, for example, one after the other or simultaneously to the video, in particular to produce the edited video.

[0017] The strength of the filter, which affects the degree or extent of the shift and / or selection of pixels, correlates with the change factor. This means that the shift and / or selection of pixels caused by the filter can depend on and / or correlate with the change factor. The correlation between the filter strength and the change factor can be positive or negative. Additionally or alternatively, the filter strength can be determined by and / or equal to the change factor.

[0018] According to the invention, the video signals processed by the filter are output. The output preferably occurs via and / or on the aforementioned display device of the motor vehicle, in particular via and / or on a display device within the motor vehicle.

[0019] The invention offers the advantage that a vehicle occupant can intuitively perceive the vehicle's acceleration visually. In particular, the invention offers the advantage that image content, especially video, can be presented in a way that takes into account the physical characteristics of human perception and / or the absorption of information, and can be designed to enable, improve, and / or optimize the human perception of the displayed information in a specific way.

[0020] In the present invention, the processing of video signals enables and / or enhances the information displayed regarding the current acceleration of the vehicle for an occupant of the motor vehicle. For example, in a motor vehicle, a driver and / or at least one occupant can be shielded from the environment to such an extent that, in particular, the most comfortable driving experience possible can be achieved. However, this means that a sense of the current acceleration of the vehicle and / or a perception of the current acceleration of the vehicle, especially by the at least one occupant and / or by the driver, is lost, particularly since forces acting due to the current acceleration are shielded and / or weakened by the shielding of the vehicle from the environment.The described visual representation of acceleration, achieved through the video signal processing according to the invention, enables and / or improves the perception of the displayed information, i.e., the current acceleration of the vehicle. This allows the user and / or occupant of the vehicle to grasp the transmitted information, i.e., the current acceleration of the vehicle, quickly, efficiently, and / or intuitively. Thus, video content is presented in a way that takes into account the physical characteristics of human perception, particularly visual perception, for information processing. This processing, especially the visual processing of the video, enables and / or improves the perception of the current acceleration by the user and / or at least one occupant of the vehicle.

[0021] To create the video, a method known from the prior art, particularly in photography, for stitching images can be applied to the video signals and / or individual videos from the individual cameras. For example, stitching can be applied to the individual frames and / or still images of the video signals and / or individual videos from the individual cameras. Additionally or alternatively, the video signals and / or individual videos from all cameras can be combined into a single video by knowing the relative orientations of the cameras to each other and, based on these relative orientations, combining the images and / or individual videos of the cameras into a single video according to a prior art method, such that, in particular, the area depicted in the video corresponds to the combined field of view of all cameras.Additionally or alternatively, object recognition can be used to detect objects in the individual camera videos and to overlap identical objects appearing in multiple individual videos, allowing the individual videos to be combined into a single video. This method is also known from the prior art.

[0022] A prior art document (DE 10 2018 132 926 A1) discloses how to determine display data for the dynamic virtual display in an automated driving mode, taking into account the actual acceleration of the vehicle. A disadvantage of this document is the use of data from a virtual world that is displayed, which reduces the connection to reality. This diminishes the advantage of a better and / or more intuitive perception of acceleration.

[0023] The invention also includes further developments that result in additional advantages.

[0024] A further development involves mapping the current speed of the vehicle, described by speed signals from at least one speed sensor, onto a rate of change. This means that the rate of change depends not only on the acceleration signals but also on the speed signals, and / or that the rate of change correlates with both the acceleration and speed signals. The speed signals can be provided by at least one speed sensor, which is preferably integrated into the vehicle in a known manner. The current speed of the vehicle can be determined using the at least one speed sensor, for example, via a method known from the prior art for determining the speed, in particular the current speed, of the vehicle.The current speed of the motor vehicle can be included in the speed signals and / or the speed signals can describe the current speed of the motor vehicle.

[0025] A positive correlation between the acceleration signals and the change factor also implies a positive correlation between the velocity signals and the change factor. Conversely, a negative correlation between the acceleration signals and the change factor also implies a negative correlation between the velocity signals and the change factor. Specifically, this means that the correlation between the velocity signals and the change factor is equal to the correlation between the acceleration signals and the change factor. This means, in particular, that the change factor is larger and / or smaller when both the current velocity and the current acceleration are higher. Additionally or alternatively, it can mean that the change factor is larger and / or smaller when both the current velocity and the current acceleration are lower.Both a change in current velocity and a change in current acceleration can alter and / or influence the change factor. An increase in current acceleration can change the change factor in the same way and / or in the same direction (increase and / or decrease), but not necessarily by the same amount, as an increase in current velocity. Similarly, a decrease in current acceleration can change the change factor in the same way and / or in the same direction (increase and / or decrease), but not necessarily by the same amount, as a decrease in current velocity. A positive correlation can exist between the acceleration signals and the current acceleration. A positive correlation can also exist between the velocity signals and the velocity.

[0026] The further development offers the advantage that both current acceleration and current speed can be intuitively displayed and / or perceived by a vehicle occupant. In particular, this allows for the perception of both current acceleration and current speed. Advantageously, the invention allows a vehicle occupant to be intuitively informed and / or perceived when a high and / or critical speed and / or a current speed is reached, which in particular exceeds a predefinable reference speed and / or when a predefinable reference speed can be reached due to current acceleration, preferably within a predefinable time interval. In other words, criticality (e.g.,A risk of skidding, sliding, oversteering, or understeering can be signaled in the video content. This allows for a warning to be issued in an intuitive and quickly recognizable manner, for example, regarding excessive speed and / or acceleration.

[0027] A further development involves the filter causing a shift of a section of the video, with the section being displayed on the screen, and the strength of the filter being determined by the speed and / or acceleration of the shift. This can include a detection range of at least one camera being larger than the area displayed and / or output on the screen. For example, only a section of the video can be displayed. A section of a video can be understood as a section being extracted from each and / or at least part of the frames, with only this section being displayed, and / or the sections of the frames and / or still images constituting the video.The filter can cause the image section displayed on the screen to be altered and / or shifted. The filter can change the position of the image section within the video and / or the detection range of the at least one camera and / or the video itself. The speed and / or acceleration of the change and / or shift of the image section is determined by the filter strength, which preferably correlates with the shift factor. This means that a higher shift factor can result in a faster and / or slower shift of the image section within the video, and / or a lower shift factor can result in a faster and / or slower shift of the image section within the video.It may be provided that only a part of the video is displayed on the display device, which may refer to a spatial part, and where a change, in particular a speed and / or acceleration of the change, a selection of the part displayed on the display device and / or image section may be influenced by the filter.

[0028] The advantage of this advanced training is that it allows for the simple depiction of accelerated vehicle movement and / or vehicle movement with a greater acceleration and / or speed than the current acceleration and / or speed. This is achieved by panning and / or shifting the image section from one end of the detection range of the at least one camera and / or video to another end, particularly to the opposite end.Additionally or alternatively, the image section can be panned and / or moved along a segment that is part of a path running from one end of the detection range of the at least one camera and / or the video to another end of the detection range of the at least one camera and / or the video, in particular to an opposite end of the detection range of the at least one camera and / or the video.In particular, by moving the image section within the entire detection range of the at least one camera and / or the video, wherein only the image section and in particular only a partial area and / or part of the video is displayed on the at least one display device, a speed and / or acceleration can be displayed that is greater than the actual speed and / or acceleration, whereby the at least occupant can perceive a higher speed and / or acceleration than the actual speed and / or acceleration.

[0029] Further development involves generating a time-varying model of the environment from video signals generated by multiple cameras. This model recreates the environment depicted in the video signals from multiple perspectives. Specifically, this means that the vehicle is equipped with several cameras, each capturing the environment from different angles and / or perspectives. This implies that the cameras have non-zero angles relative to each other, allowing them to capture the environment from different viewpoints and / or perspectives. By capturing at least one object and / or the environment from these different perspectives, a model, particularly a three-dimensional model, can be generated. This can be achieved, for example, using a method known from the prior art.The model can describe the object and / or the environment from different perspectives and / or from additional perspectives that lie between the perspectives of the different cameras.

[0030] According to the further development, the filter causes a change in perspective on the model. This can mean that an image of the model of the environment is displayed and / or output on the display device from a specific perspective, and the application of the filter causes a change in the image of the model of the environment displayed on the device such that the image displayed after the filter's application shows and / or represents the environment and / or the model of the environment from a different perspective. The strength of the filter is determined by the speed and / or acceleration of the change (change) in perspective. It can be provided that the change and / or alteration of the perspective from which the image of the environment displayed on the device shows the model of the environment and / or the environment is influenced by the strength of the filter.

[0031] The output includes a rendering of the environment from the model in the modified perspective. This rendering process allows, in particular, the creation and / or generation of the image that represents and / or reproduces the environment based on the model and / or the model of the environment.

[0032] The further training offers the advantage that, by changing the perspective display, an acceleration and / or speed of the motor vehicle can be shown on the display device that is greater than the actual speed and / or acceleration of the motor vehicle.

[0033] Further training includes the requirement that the video signals depict an environment located in the direction of travel of the vehicle. This means that at least one camera has a detection range that lies in the direction of travel of the vehicle and / or in front of the vehicle. If the detection range of the at least one camera is larger than an area of ​​the environment located in the direction of travel of the vehicle, those video signals can be selected whose detection range includes part of the environment located in the direction of travel of the vehicle and / or the environment located in the direction of travel of the vehicle. Additionally or alternatively, a preliminary video can be created from the video signals, from which the video is limited by restricting the area depicted in the video to the area of ​​the environment that, as seen from the vehicle, lies in the direction of travel of the vehicle and / or is located there.In particular, according to the training, a video of the surroundings in the direction of travel of the motor vehicle can be provided.

[0034] The filter causes the video to zoom in. Zooming in can mean that an increasingly smaller area and / or section of the video is displayed larger and larger, so that the ever-shrinking area and / or section of the video continues to be displayed on the entire screen of the display device and / or by means of the entire display device. Additionally or alternatively, zooming in can mean that a field of view and / or a section of the video is enlarged. Zooming in can reveal more details and / or shift the focus of the viewer from an overview to specific features and / or objects within the video.

[0035] The strength of the filter is determined by the speed and / or acceleration of the zoom into the video. This can mean that the speed and / or acceleration at which a section of the video image becomes progressively smaller and progressively larger depends on the strength of the filter.

[0036] The further training offers the advantage that speed and / or acceleration can be displayed in a simple way using an environment that lies in the direction of travel of the motor vehicle, making the method particularly suitable for at least one display device that is located in the direction of travel relative to at least one occupant.

[0037] A further development includes the filter additionally distorting the video along lines extending from a central point of the video, which can be displayed and / or positioned in the center of the display device, to an edge of the video. This means that the video has a central point, which can be determined by the fact that, when the video and / or a portion of the video is displayed on the device, the central point is positioned centrally and / or in the middle of the display device. The filter can distort the video along lines extending from the central point to the edge of the video, with the degree of distortion increasing the further one moves away from the central point along at least one of the lines.The distortion can create an image as if it were taken by a camera that has a lens defect of pincushion distortion, which is known from the prior art.

[0038] The strength of the filter is determined by the speed and / or acceleration at which the distortion is changed. Additionally or alternatively, the filter strength can specify the degree of distortion.

[0039] The advantage of this further training is that the speed and / or acceleration of the motor vehicle can be represented and / or perceived by an occupant in a particularly simple and / or effective and / or intuitive way, especially a speed and / or acceleration that may be greater and / or less than the actual speed and / or acceleration of the motor vehicle.

[0040] A further development involves displaying the processed video signals in the background on at least one display device and simulating an artificial horizon in the foreground. The inclination of this artificial horizon correlates with the lateral acceleration experienced by the vehicle during cornering. In other words, the magnitude of the lateral acceleration during cornering can be determined, and the inclination of the artificial horizon can be adjusted accordingly. The artificial horizon can be a known artificial horizon, specifically a line whose inclination can change relative to a horizontal line. According to this further development, the change in inclination can correlate with and / or depend on the lateral acceleration during cornering.

[0041] The advantage of this advanced training is that not only can an occupant of the vehicle intuitively perceive longitudinal acceleration, but lateral acceleration can also be displayed, allowing at least one occupant to perceive it intuitively and / or in a simplified way. This can reduce motion sickness for the observer.

[0042] According to further training, the filter processes the video signals in real time. Real-time processing means that the video is processed and / or edited without any time delays, particularly compared to the recording. The advantage of real-time processing is that the surroundings, which can be perceived by at least one occupant of the vehicle through a window, can also be processed and / or edited and displayed on the screen. This makes the perception of speed and / or acceleration, resulting from the processing of the video signals, more realistic, authentic, and / or intense.The speed and / or acceleration that the occupant is meant to perceive through the processing of the video signals can be presented and / or conveyed more intuitively and / or better and / or more meaningfully through real-time processing.

[0043] Further training that has the advantage of further enhancing an acceleration effect brought about by the application of the filter, thereby simulating an acceleration that is higher than the actual acceleration, includes at least one of the following features. a. Sound effects, in particular a turbine sound, are emitted in the vehicle, their volume and / or pitch correlating with the change factor. This means, in particular, that sounds and / or auditory content are emitted in the vehicle. Specifically, a turbine sound can be reproduced in the vehicle as a sound effect and / or auditory content, whereby, depending on the value of the change factor and / or the change factor, a turbine sound can be selected at a predefined number of revolutions per minute. b. Additionally or alternatively, a simulation of at least one component of a means of transport can be displayed on a simulation display device in the motor vehicle at a given acceleration (current acceleration of the means of transport) and / or speed (current velocity of the means of transport), wherein the acceleration and / or velocity of the means of transport correlates with the change factor. In particular, a means of transport can be selected whose maximum speed and / or maximum acceleration is greater than the maximum speed and / or current velocity and / or maximum acceleration and / or current acceleration of the motor vehicle. For example, an airplane can be selected as the means of transport.At least one component of the means of transport and / or at least one component of the means of transport, which preferably has different optical properties and / or different appearances at different speeds and / or accelerations, can be simulated for different speeds and / or accelerations, wherein, depending on the change factor and / or depending on the change factor, at least one of the simulations can be selected and / or displayed on the simulation display device in the motor vehicle. c. Additionally or alternatively, when cornering, the vehicle's body can be adjusted to sit higher on one side of the curve than on the inside by adjusting the suspension. This may require the vehicle to have air suspension, which allows for adjustable height and / or suspension travel. The system may also measure the vehicle's lateral acceleration during cornering and adjust the suspension accordingly, resulting in one side of the vehicle, particularly the outside side, being positioned higher and / or having greater suspension travel than another side, especially the inside side.One side of the vehicle can be positioned higher than the other by setting a greater suspension travel and / or a different preload on the higher side. Additionally or alternatively, the air springs on the lower side can be at least partially deflated to create the effect of that side being lower. A height difference between the two sides of the vehicle resulting from the suspension setting can depend on and / or correlate with the magnitude of the lateral acceleration, particularly positively correlate with it. d. Additionally or alternatively, haptic feedback, in particular vibrations and / or movements of at least one vehicle seat, may be provided, which correlates with the change factor. This means, in particular, that a driver and / or at least one occupant of the vehicle experiences haptic feedback, the intensity of which depends on the change factor.

[0044] Further development includes storing the video signals and displaying them in slowed-down form on at least one display device. Furthermore, if the current acceleration exceeds a predefined acceleration threshold, the processed video signals are displayed in accelerated form on the display device, in addition to applying the filter. This means, in particular, that the vehicle includes at least one buffer in which the video signals and / or the processed video and / or processed video signals are temporarily stored. The video signals and / or the processed video and / or processed video signals can be retrieved from this buffer, allowing, for example, output on the display device.The display of video signals and / or edited video signals is not in real time, meaning that video signals and / or edited video signals are stored in the buffer but have not yet been played back. This means that the buffer may contain video signals and / or edited videos that have already been recorded but not yet played back. This can be achieved by playing back the video signals and / or edited videos at a speed slower than real-time playback and / or recording speed. Specifically, the video signals are played back slower than they were recorded.

[0045] If the current acceleration exceeds a predefined acceleration threshold, the video signals and / or the processed videos and / or the processed video signals are played back at an accelerated rate in addition to the application of the filter. This means, in particular, that video signals and / or processed videos and / or the processed video signals are retrieved from the buffer on which at least one display device is output and / or played back, whereby the playback can occur at a playback speed that is greater than the slowed-down playback speed and / or the recording speed.

[0046] This offers the advantage that an accelerated and / or faster movement of the motor vehicle can be depicted in a simple and uncomplicated way using the video and / or perceived by at least one occupant.

[0047] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0048] The invention also includes the computing unit for the motor vehicle, which processes the video signals. The computing unit can comprise a data processing device or a processor circuit configured to perform an embodiment of the method according to the invention. For this purpose, the processor circuit can comprise at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor circuit can comprise program code configured to perform the embodiment of the method according to the invention when executed by the processor circuit.The program code can be stored in a data memory of the processor device. The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).

[0049] The invention comprises a motor vehicle comprising at least one display device, at least one camera and an acceleration sensor, wherein the motor vehicle is configured to carry out a method according to the invention.

[0050] The invention also includes further developments of the motor vehicle according to the invention, which have features as already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the motor vehicle according to the invention are not described again here.

[0051] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0052] As a further solution, the invention also includes a computer-readable storage medium comprising program code which, when executed by a computer or a computer network, causes it to execute an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the computer or computer network. However, the storage medium can also be operated, for example, as an app store server and / or cloud server on the internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code can be provided as binary code, assembly code, source code in a programming language (e.g., C), or a program script (e.g., Python). Alternatively, the computer-readable storage medium can be implemented as a signal containing computer-readable data, such as a time-varying voltage signal or a radio signal.

[0053] The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

[0054] The problem is solved by the subject matter of the independent patent claims. Advantageous further developments of the invention are described by the dependent patent claims, the following description, and the figure(s).

[0055] The following are exemplary embodiments of the invention described. This is illustrated by: Fig. 1 an exemplary embodiment of a motor vehicle according to the invention; Fig. 2 an exemplary application of a filter according to the invention; Fig. 3 an exemplary application of a filter according to the invention; Fig. 4 an exemplary application of a filter according to the invention; Fig. 5 an exemplary application of a filter according to the invention; Fig. 6 an exemplary application of a filter according to the invention.

[0056] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0057] In the figures, identical reference symbols denote functionally equivalent elements.

[0058] Fig. Figure 1 shows an example of a motor vehicle 10. The motor vehicle 10 comprises at least one display device 12, at least one computing unit 14, and at least one camera 16, each camera 16 providing video signals. The following is shown as an example: Fig. 1 at least one single camera 161 shown which has a viewing direction, with an example in Fig. 1. The direction of view is given as a direction of travel 18 of the motor vehicle 10. In addition to or alternatively to the at least one single camera 161, the motor vehicle 10 can include at least one camera system 162, which comprises several cameras 16 (in the example of the Fig. 1 the cameras 163, 164 and 165). The multiple cameras 16 of the camera system 162 can be configured to capture the environment of the motor vehicle 10 from several different perspectives. For this purpose, the multiple cameras 16 can each have different angles relative to each other and / or different viewing directions. In Fig. This is exemplified by the fact that the lenses of the multiple cameras 16 each point in different directions. This can result in different detection ranges 20 for the various cameras 16. For example, camera 163 has detection range 203, camera 164 has detection range 204, and camera 165 has detection range 205. In particular, the detection ranges 20 of the cameras can overlap, so that there is a common detection range 202 for the cameras 16 of the camera system 162 and / or for the multiple cameras 16, in which the environment and / or part of the environment of the vehicle 10 is captured from several different perspectives. This means that an environment and / or part of the environment is captured in the common detection range 202 of the multiple cameras 16 from several different perspectives and / or angles.

[0059] Additionally, the motor vehicle 10 includes at least one acceleration sensor 22. The acceleration sensor 22 is configured to determine the current acceleration of the motor vehicle 10, which can describe the acceleration with which the motor vehicle 10 is accelerating at a given time. The acceleration sensor 22 is specifically configured to transmit the current acceleration of the motor vehicle 10 to the processing unit 14 via acceleration signals 221. Additionally, the motor vehicle 10 may include at least one speed sensor 24, which determines the current speed of the motor vehicle 10. The current speed of the motor vehicle 10 can be transmitted to the processing unit 14 as speed signals 241.

[0060] The acceleration signals 221 and / or the speed signals 241, and thus the current acceleration and / or speed of the vehicle 10, can be mapped to a change factor. This means, in particular, that a change factor is generated which depends on the value of the current acceleration and / or speed of the vehicle 10. Specifically, it can be provided that the change factor increases and / or decreases with increasing current acceleration and / or speed, and decreases or increases with decreasing current acceleration and / or speed. Thus, there can be a positive or negative correlation between the change factor and the current acceleration and / or speed.If there is a positive correlation between the change factor and the current acceleration, there may be a positive correlation between the change factor and the current speed, and if there is a negative correlation between the change factor and the current acceleration, there may be a negative correlation between the change factor and the current speed.

[0061] A parameterizable filter is applied to the video signals from at least one camera 16, which the at least one camera 16 has transmitted to, for example, the processing unit 14. The strength of the filter correlates with the gain factor. The filter can involve the selection and / or displacement of pixels in a video created from the video signals of the at least one camera 16, whereby the displacement and / or selection of pixels can depend on the gain factor. The application of the filter, i.e., the processing of the video signals and / or the video, can occur in real time. In particular, real-time processing of the video signals takes place.

[0062] For example, the filter can cause a section 28 of the video displayed on the screen 12 to be shifted. This is demonstrated by... Fig. 2 and Fig. Figure 3 shows an example of an environment 26 as captured by one or more cameras 16. For example, it can be the capture area of ​​a single camera 16, preferably a single camera 161. Additionally or alternatively, it can be a combined capture area, which can result from merging and / or combining the capture areas of several cameras 16, preferably several cameras 163, 164, 165 of a camera system 162. This means, in particular, that the images from several cameras 16 are combined into one image, for example by stitching, and this can be done for each frame.

[0063] From the environment 26, which is captured by the at least one camera 16, a section of the image 28 can be selected and displayed on the display device 12. This means, in particular, that not the environment 26 captured by the at least one camera 16, but only the section of the image 28, is displayed on the display device 12. The section of the image 28 can have a predefinable position relative to the detection range of the at least one camera 16. For example, in Fig. 2 and Fig. 3. The image section 28 is arranged at the edge of the detection area and / or the surroundings 26 in the direction of travel 18. The position of the image section 28 is, in particular, independent of the objects located in the surroundings 26, so that the position of the image section 28 relative to the detection area of ​​the at least one camera 16 does not change over time. The part of the surroundings 26 that lies within the image section 28 and is displayed on the display device 12 can change over time, for example, due to movement of the vehicle 10.

[0064] By applying the filter, it can be provided that the image section 28 shifts, in particular relative to the detection range of the at least one camera 16, so that instead of image section 280 a new image section 281 is displayed, the position of which differs from the position of image section 280 relative to the detection range. Applying the filter can change the position of image section 28 relative to the detection range, preferably by means of a continuous shift of image section 28, as is achieved by Fig. 3 is shown schematically. This means, in particular, that there is no abrupt transition between the image section 280 and the new image section 281, but rather that during the shifting process, intermediate image sections 282 to 285 are displayed on the display device 12, the position of which, relative to the detection area, lies between the position of the image section 280 before the shifting and the position of the new image section 281 after the shifting. Preferably, the image section 28 is shifted against the direction of travel 18, as is also shown by way of example in Fig. 2 and Fig. 3 is shown.

[0065] The speed and / or acceleration of the shift can be determined by the strength of the filter, which correlates with the gain factor.

[0066] Additionally or alternatively, the filter can change the perspective of a model 30 of a part of the environment 26, which was captured by several cameras 16 from different perspectives. This is exemplified in Fig. Figure 4 shows that several cameras 16 can each have non-zero angles relative to each other, so that each camera 16 can capture a part of the environment 26 from different angles and / or perspectives. The part of the environment 26 that lies within a common detection range 202 of several cameras 16 can be perceived from multiple perspectives by the multiple cameras 16, which can have different angles relative to each other. This allows multiple images 32 of the same part of the environment 26 to be provided from different perspectives, as exemplified by images 321, 322, and 323 in Figure 4. Fig. Figure 4 is shown. From the images 32, which depict the same part of the environment 26 from different perspectives, a model 30 of the part of the environment 26 can be created. From the model 30, an image of the part of the environment 26 can be extracted and / or rendered from those perspectives that correspond to the perspective of at least one of the cameras 16 and / or that correspond to a perspective that lies between the perspectives of at least two of the cameras 16.

[0067] It can be provided that an image of model 30 of part of the environment 26 is displayed and / or mapped on the display device 12 from a first perspective. Applying the filter allows the perspective to be changed. This means, in particular, that applying the filter results in an image of model 30 of part of the environment 26 being displayed on the display device 12, showing model 30 from a second perspective that differs from the first. The change in perspective can be continuous, with the speed and / or acceleration at which the perspective changes depending on and / or correlated with the strength of the filter and / or the change factor.

[0068] Additionally or alternatively, the filter can be designed to zoom in on the video, as exemplified in Fig. 5 is shown. This means that the environment 26, which is captured by the at least one camera 16, is zoomed in 34, so that after zooming in 34, only a section 28 of the environment 26 is displayed and / or output on the display device 12 instead of the environment 26. This allows the section 28 of the environment 26 to be displayed enlarged on the display device 12. Preferably, the zooming in 34 is performed on the environment 26, which is captured by a camera 16 whose detection range points in the direction of travel 18 of the motor vehicle 10. The zooming in 34 is preferably continuous, so that during the zooming in 34, a section 28 of the environment 26 displayed on the display device 12 can be shown as an increasingly smaller part of the environment 26 with an increasingly greater magnification. The speed and / or acceleration of zooming in 34 can correlate with the strength of the filter and / or the change factor.

[0069] Additionally or alternatively, the filter can cause a distortion of the video, as exemplified in Fig. 6 is shown. Fig. Figure 6 shows an environment 26, preferably as captured by at least one camera 16 pointing in the direction of travel 18 of the motor vehicle 10. By applying the filter, a distortion 36 of the environment 26 can occur, wherein the speed and / or acceleration at which the distortion 36 changes, and / or the magnitude of the distortion 36, can correlate with the strength of the filter and / or with the change factor.

[0070] The distortion 36 preferably occurs along lines 40 extending from a central point 38 to an edge 42 of the video and / or the environment 26. The central point 38 may be characterized by being centrally located on the display device 12, and / or the central point 38 may be the point and / or pixel that is centrally and / or midway on the display device 12. The lines 40 may each begin at the central point 38 and extend to the edge 42 of the video and / or the environment 26 and / or the display device 12. The lines 40 may meet at the central point 38. In particular, the various lines 40 do not intersect and / or touch each other. Intersecting and / or touching of different lines can only occur at the central point 38. These may be straight lines 401 and / or curved lines 402. Fig. 6 are examples where straight lines 401 are drawn solid and curved lines 402 are drawn dashed. In the example of the Fig. 6. Either the straight lines 401 or the curved lines 402 can be selected. The distortion 36 can involve stretching pixels in the direction indicated by the respective line 40 passing through the pixel. The distortion 36 can increase the further the stretched pixel is from the center point 38 and / or the closer the pixel is to the edge 42.

[0071] Additionally or alternatively, a processed video, created by applying the filter to the video and / or video signals, can be displayed in the background of the display device 12, while an artificial horizon can be displayed in the foreground of the display device 12. The artificial horizon can be a horizontal line and / or a line running parallel to an edge and / or boundary of the display device 12 located at the top and / or bottom in the vertical direction of the vehicle. In addition to the horizontal line, the artificial horizon can have a second line whose inclination relative to the horizontal line correlates with the lateral acceleration of the vehicle 10. In particular, the inclination of the second line relative to the horizontal line is greater the greater the measured lateral acceleration.The part of the second line that is located on the outside of the curve may be positioned above the horizontal line when cornering; the part of the second line that is located on the inside of the curve may be positioned below the horizontal line when cornering.

[0072] To determine the lateral acceleration of the motor vehicle, the motor vehicle 10 can have at least one lateral acceleration sensor which preferably detects an acceleration of the motor vehicle 10 perpendicular to the direction of travel 18 of the motor vehicle 10.

[0073] Additionally or alternatively, the video signals from at least one camera 16 can be stored. This storage can take place in a memory 44 of the vehicle 10. The video signals can be displayed on the display device 12, particularly after being stored in the memory 44 of the vehicle 10, in slow motion. Video signals from the at least one camera 16 that is currently displaying the video signals on the display device 12 can be available, provided that the available video signals have not yet been displayed on the display device 12 and / or represent a point in time that lies in the future, and in particular, in the past, relative to the current point in time.By storing and slowing down playback, video signals can be provided that lie in the future, starting from a current playback time that can describe the time at which the video signals were captured by at least one camera 16.

[0074] If the vehicle 10, in particular the acceleration sensor 22, detects an acceleration exceeding a predefinable acceleration threshold, the video signals can be played back at an accelerated rate. The transition to accelerated playback can be continuous. In particular, the playback speed of the video signals, especially on the display device, is continuously increased, for example, until the acceleration of the vehicle no longer increases. Accelerated playback can involve playing back the video signals at a speed greater than the slowed-down playback speed and / or at a speed corresponding to the recording speed at which the at least one camera 16 captures the video signals and / or at a speed greater than the recording speed. For this purpose, the video signals stored in the memory 44 of the vehicle 10 can be accessed.This allows a playback speed to be achieved that is greater than the recording speed of at least one camera 16.

[0075] Additionally, an acceleration effect that can be caused by the application of the filter can be amplified. This acceleration effect, which can result from the application of the filter, describes in particular the perception by an occupant of the motor vehicle 10 of a different, especially a higher, speed and / or acceleration than the actual speed and / or acceleration.

[0076] The amplification of the acceleration effect can be achieved, for example, by simulating at least one component of a means of transport on a simulation display device in the motor vehicle, where the vehicle acceleration and / or vehicle speed correlates with the amplification factor. For example, at least one component of an aircraft can be simulated and displayed on a second display device, such as at least one transparent display on at least one window of the motor vehicle 10 and / or at least one display in the interior of the motor vehicle 10. The aircraft component could be, for example, at least one landing flap and / or at least one wing and / or at least one engine of the aircraft.The aircraft component can appear different at different speeds and / or accelerations. For example, landing flaps can be extended to different degrees and / or engines can operate with different rotor blade angular velocities. In particular, during positive acceleration (the vehicle's speed increases), at least one aircraft component is simulated during takeoff, and / or during deceleration, at least one aircraft component is simulated during landing.

[0077] Additionally or alternatively, the acceleration effect can be enhanced by displaying a runway and / or landing strip, particularly in the front area of ​​the motor vehicle, for example on the second display device.

[0078] Additionally or alternatively, sound effects, in particular a turbine sound, can be output in the vehicle, their volume and / or pitch correlating with the change factor. For example, the sound of an aircraft turbine can be output in vehicle 10. In particular, the volume of the aircraft turbine sound can change with the acceleration and / or speed of vehicle 10 and / or the volume of the aircraft turbine sound can correlate with the change factor. For example, the volume at which the aircraft turbine sound is output can be changed (increased and / or decreased) when the speed and / or acceleration of the vehicle changes (increased and / or decreased).In particular, if there is a positive change in speed and / or acceleration (the speed and / or acceleration increases), a sound of an aircraft turbine during the start of an aircraft may be emitted inside the vehicle and / or outside the vehicle.

[0079] Additionally or alternatively, when cornering, the vehicle body can be positioned higher on the outside of the curve than on the inside by adjusting the suspension. This can be achieved, for example, by different spring travel on the inside and outside of the curve, such as in the case of air suspension.

[0080] Additionally or alternatively, haptic feedback, in particular vibrations and / or movements of at least one vehicle seat, can correlate with the amplification factor. Thus, haptic elements, such as vehicle seats located in the interior of the vehicle 10, can vibrate, with the intensity of the vibration depending on the amplification factor.

[0081] A particularly preferred embodiment is described below.

[0082] The object of the invention is in particular to display an impression of a flight effect and / or a flight and / or a flight speed on a display device, for example on at least one side window and / or on the windscreen and / or on a screen, in particular a curved screen (curved monitor), in a motor vehicle.

[0083] Images and / or video data can be captured using at least one environmental sensor, and in particular at least three environmental sensors, wherein the environmental sensor is preferably a camera that detects the area around the vehicle. The at least one environmental sensor can be mounted on the side and / or at the front of the vehicle in the direction of travel. The image capture and / or video data can be enhanced, in particular, with information from images and / or image data from at least one roof camera and / or a top-view camera (or a subsystem of a top-view camera) configured to display a top-down view of the area around the vehicle.The image content and / or image recordings and / or video data from the multiple environmental sensors can be mixed, combined via an output channel, and displayed on a display device, for example, on at least one side window and / or on a windshield and / or on a screen and / or on a curved screen in the vehicle and / or on a head-up display in the vehicle. In particular, the impression of a flight effect, especially the impression of a flight effect during takeoff, can be created.

[0084] The following steps can be taken, especially when starting a journey with a motor vehicle: Video data can be acquired by at least one environmental sensor, and in particular by several environmental sensors, preferably arranged side by side and / or one above the other, which capture the area around the vehicle from different angles. The at least one environmental sensor, and in particular all environmental sensors, is preferably at least one camera, which captures the surroundings, i.e., an area outside the vehicle. The video data can be temporarily stored on a data carrier and / or a storage device in the vehicle.

[0085] Because multiple environmental sensors are arranged in a staggered arrangement relative to each other, and / or because the environment is captured from different angles, combining the video data from the individual sensors into a single video can create an acceleration effect. This effect can resemble a camera movement and / or panning motion, and / or can simulate a camera movement and / or panning motion. This acceleration effect can be generated particularly at the start and / or end of a journey in a motor vehicle. The generation of this acceleration effect can be configured as a driving mode in the vehicle.

[0086] The video data and / or image signals from the multiple environmental sensors can be combined and / or merged and / or superimposed to form a video and / or widescreen video feed and / or widescreen video stream.

[0087] Additionally, the video and / or widescreen video feed and / or widescreen video stream can be enriched with information from the surroundings by using a roof-mounted camera and / or top-view camera images that depict the vehicle's environment from a top-down perspective and / or from a bird's-eye view. This can occur particularly when combining and / or overlaying the video signals from the environmental sensors and / or the camera images from one side of a vehicle.

[0088] An acceleration effect, simulating, for example, a camera movement and / or panning motion, can be generated by playing back and outputting the images and / or video from at least one environmental sensor, preferably multiple environmental sensors, particularly multiple cameras, at an accelerated rate. This allows for the creation and / or simulation of a higher speed of movement, especially a speed higher than the actual speed of the vehicle. This can create a flight effect over the current environment.

[0089] The flight effect over the current environment can be created in particular by combining slightly offset camera shots and / or video signals, as well as by accelerating the playback of the image mix and / or video.

[0090] The video can be displayed on a display device. This display device could be, for example, at least one projector in the vehicle interior and / or a transparent display, in particular a transparent OLED (organic light emitting diode) display on at least one side window and / or a display and / or screen in a front area of ​​the vehicle interior and / or an augmented reality projection, in particular onto at least one side window and / or onto a display and / or screen (for example, via augmented reality glasses).

[0091] A flight impression can be created by a combination of real camera images, especially based on the video signals, and / or from the video and / or motion effects and / or faster image output and / or a display on the display device.

[0092] Additionally or alternatively, an artificial horizon can be displayed on the display device, for example on a curved screen and / or on a head-up display, which pans along with a representation and / or an edited video output on the display device when the vehicle turns right and / or left.

[0093] Additionally or alternatively, a display and / or simulation of landing flaps, for example on an aircraft, can be provided. Landing flaps can increase the lift of, for example, an aircraft. The visual display of the landing flaps can be shown on the display device, for example on at least one window of the vehicle and / or on at least one display inside the vehicle. The visual display of the landing flaps can make flight behavior and / or a flight effect more realistic.

[0094] Additionally or alternatively, at least one landing strip and / or runway, for example at the start of a journey and / or upon reaching a destination of the journey, can be displayed on at least one display device and / or part of the display device, for example on a curved monitor and / or a head-up display.

[0095] Additionally or alternatively, a takeoff and / or landing sound of an aircraft turbine, or something similar to an aircraft turbine, can be played in the vehicle. The takeoff and / or landing sound can mimic the sound of an aircraft (jet) during takeoff and / or landing.

[0096] Additionally or alternatively, the motor vehicle may include a lighting system that, in color and / or position, can correspond to at least one landing light and / or position light of an aircraft. "Relative to position" can mean that the side on the aircraft corresponds to the side on the motor vehicle and / or that, along the direction of travel and / or flight, the relative position to a center point (point designating the spatial center of the vehicle and / or aircraft) and an edge point (point designating an outer edge and / or end of the vehicle and / or aircraft) of the motor vehicle and / or aircraft is the same. The lighting system can be activated for a predetermined duration, for example, less than one minute, and in particular less than 30 seconds, when the vehicle is locked and / or unlocked.

[0097] Additionally or alternatively, at least one aircraft component, for example at least one wing and / or at least one landing flap and / or at least one engine, can be simulated and displayed on a display device in the interior of the vehicle.

[0098] Additionally or alternatively, haptic feedback can be provided, particularly for a flight effect, such as a realistic flight effect and / or a realistic feeling of flying. The haptic feedback can involve at least one haptic element of the vehicle interior (interior of the motor vehicle), for example, at least one seat, and / or include vibrations and / or movements.

[0099] Additionally or alternatively, at least one camera can capture the underside of the vehicle (i.e., the lower exterior surface of the vehicle when viewed from above), and / or the road surface on which the vehicle is traveling, and / or the movement of at least one tire of the vehicle, and display and / or reproduce this information on at least one display device inside the vehicle. This can, for example, create the effect of a high-speed flyover.

[0100] Additionally or alternatively, a sunrise and / or a sunset and / or a starry sky above the clouds can be displayed on at least one window of the motor vehicle, for example by means of a transparent display.

[0101] Overall, the examples show how a flight experience can be provided for a motor vehicle, especially for the Landjet, for example via a supersonic mode and / or a flight mode.

Claims

[1] Method for processing video signals, in which the video signals are generated by at least one camera (16) of a motor vehicle, wherein the at least one camera (16) captures at least a part of an environment (26) of the motor vehicle, the generated video signals are processed by a computing unit (14) and the processed video signals are output on at least one display device (12) of the motor vehicle, characterized by , that a. acceleration signals (221) describing the current acceleration of the motor vehicle from an acceleration sensor (22) of the motor vehicle b. are mapped to a change factor, with a positive or negative correlation between the acceleration signals (221) and the change factor, c. a parameterizable filter is applied to the video signals generated by the at least one camera (16), which causes a shift and / or selection of pixels from still images (32) derived from a video obtained from the video signals, wherein a strength of the filter affecting the shift and / or selection of pixels correlates with the change factor, and d. the output of the video signals processed by the filter takes place. [2] Method according to claim 1, wherein speed signals (241) describing the current speed of the motor vehicle from at least one speed sensor (24) are additionally mapped to the change factor and, in the case of a positive correlation between the acceleration signals (221) and the change factor, there is also a positive correlation between the speed signals (241) and the change factor and, in the case of a negative correlation between the acceleration signals (221) and the change factor, there is also a negative correlation between the speed signals (241) and the change factor. [3] Method according to one of the preceding claims, wherein the filter causes a displacement of an image section (28) from the video, wherein the image section (28) is output on the display device (12) and wherein the strength of the filter is given by a speed and / or acceleration of the displacement of the image section (28). [4] Method according to one of the preceding claims, wherein a time-changing model (30) of the environment (26) is determined from the video signals of several cameras (16, 162), which replicates the environment (26) depicted on the video signals from several perspectives, and the filter causes a change in perspective on the model (30), wherein the strength of the filter is given by a speed and / or acceleration of the change in perspective, and the output comprises rendering the environment (26) from the model (30) in the changed perspective. [5] Method according to one of the preceding claims, wherein the video signals describe an environment (26) located in the direction of travel (18) of the vehicle and the filter causes a zoom (34) into the video, wherein the strength of the filter is given by a speed and / or acceleration of the zoom (34) into the video. [6] Method according to claim 5, wherein the filter additionally causes a distortion (36) of the video along lines (40) extending from a center point (38) of the video, which is displayed and / or arranged centrally on the display device (12), to an edge (42) of the video, wherein the strength of the filter is given by a speed and / or acceleration with which the distortion (36) is changed and / or wherein the strength of the filter indicates a strength of the distortion (36). [7] Method according to claim 5 or 6, wherein the output of the processed video signals takes place in the background on the at least one display device (12) and an artificial horizon is simulated in the foreground, the inclination of which correlates with a lateral acceleration during cornering. [8] Method according to any of the preceding claims, wherein real-time processing of the video signals is carried out by the filter. [9] A method according to any of the preceding claims, wherein an acceleration effect brought about by the application of the filter, by which a simulation of an acceleration higher than the actual acceleration is carried out, is enhanced by a. Sound effects, in particular a turbine sound, are emitted in the motor vehicle, wherein their volume and / or pitch correlates with the change factor, and / or b. on a simulation display device in a motor vehicle, a simulation of at least one component of a means of transport is simulated at a means of transport acceleration and / or means of transport speed, wherein the means of transport acceleration and / or the means of transport speed correlate with the amplification factor, and / or c. when the motor vehicle is cornering, the vehicle body is higher on the outside of the curve than on the inside of the curve due to adjustment of the suspension and / or d. haptic feedback, in particular vibrations and / or movements of at least one vehicle seat, correlates with the change factor. [10] Method according to one of the preceding claims, wherein the video signals are stored and slowed down before being displayed on the at least one display device (12) and, in the event of an actual acceleration exceeding a predefinable acceleration threshold, an accelerated playback of the video signals takes place in addition to the application of the filter. [11] Motor vehicle comprising at least one display device (12), at least one camera (16) and an acceleration sensor (22), wherein the motor vehicle is configured to perform a method according to one of the preceding claims.

Images