METHOD FOR DISPLAYING AUGMENTED REALITY AND DEVICES FOR APPLYING THE METHOD

Description

[0001] The invention relates to a method for computer-aided representation of augmented reality. The invention also relates to a representation device for augmented reality. Furthermore, the invention relates to a vehicle with a representation device for augmented reality. Finally, the invention relates to a computer program product and a provision device for this computer program product, wherein the computer program product is equipped with program instructions for implementing this method.

[0002] Document KR 2019 007 8944 A relates to an augmented reality head-up display system for a rail vehicle. The display system comprises a first and a second head-up display installed in the front and rear driver's cabs of a rail vehicle, respectively, a detection unit for detecting the surroundings of the rail vehicle to generate route information, and a control unit for outputting an image displayed using augmented reality on the first and second head-up displays.

[0003] The advantage of augmented reality is that it allows the train driver to be provided with information about the route ahead that goes beyond what is already visible through the vehicle's windshield. This additional visual information could, for example, include railway signals that are not physically present in the real world. This allows for savings in physical track elements while maintaining an appropriate level of safety.

[0004] However, augmented reality displays carry the risk that an incorrect display could lead to misunderstandings by the train driver regarding train traffic. This could result in dangerous accidents that must be avoided at all costs.

[0005] Document EP 3121638 A1 describes a head-up display system configured for a vehicle and a method for ensuring the safety of information displayed by means of a head-up display, wherein the HUD system comprises a visualization system.

[0006] Document WO 2019 / 016102 A1 relates to a secure display device having an input port for receiving an input signal with at least one piece of information to be displayed. The secure display device comprises a plurality of display matrices arranged one above the other and at least one transparent display area, wherein each display matrix of the plurality is associated with a control means for controlling the display matrix.

[0007] The object of the invention is therefore to provide a method and devices suitable for implementing the method, which can generate augmented reality in the driver's cab of rail vehicles while meeting the safety requirements applicable to rail traffic during operation. Furthermore, the object of the invention is to provide a computer program product and a device for providing this computer program product, with which the aforementioned method can be implemented.

[0008] This object is achieved according to the invention with the subject matter of the claim (method) specified at the outset in that the first head-up display and the second head-up display are arranged in two superimposed layers, displaying image information that complements one another to form an image.

[0009] The field of vision intended for the driver is the field of vision available to the driver at their control station to perceive the vehicle's surroundings. This is preferably the vehicle's windshield, which can be equipped with the two head-up displays.

[0010] The two head-up displays, preferably on the inside of the vehicle's windshield, can display additional visual information, for example, with the help of projectors, containing signals or warnings that are not physically present in the physical reality (outside world). This additional visual information is then superimposed on the actual view from the vehicle, thus creating augmented reality.

[0011] The head-up displays themselves are transparent, allowing the driver to perceive the reality outside the vehicle. The information provided by augmented reality (hereinafter also abbreviated to AR) is thus superimposed on the information directly accessible to the driver through, for example, the vehicle's windshield. However, the head-up displays can also display virtual reality.

[0012] Virtual reality (hereinafter referred to as VR) is the representation of reality (also referred to as physical reality) in a real-time, computer-generated, interactive virtual environment. The level of detail required for the VR creation depends on the specific application. Generally, VR is created in a three-dimensional space and represents its physical properties, particularly its topography, in a simplified manner.

[0013] Particularly in rail transport, the level of detail of the representation can often be simplified depending on the application, since the rail vehicle cannot leave the track. Therefore, two-dimensional or even one-dimensional virtual realities can also be used. These can be advantageously created from simple models of reality, such as a route map or a timetable.

[0014] A prerequisite for using VR is knowledge of the viewer's location and viewing direction. This requires localization. When VR is used for rail transport, certain simplifications arise. For example, a train driver's field of view is clearly defined through the windshield, so the VR image section to be displayed can be determined by locating the train.

[0015] If VR is to be displayed on an output device, image data must be generated to create the image. The VR display allows a viewer to intuitively grasp the VR.

[0016] The creation of augmented reality (hereinafter referred to as AR) within the meaning of the invention occurs when additional image information is superimposed on a previously generated virtual reality (VR) or the reality to be viewed (physical reality). The additional image information consists of image elements and can include, for example, symbols, text information, and image motifs that can be perceived by the viewer.

[0017] To display AR, symbols or text information, for example, can be displayed via a suitable output device. Another option is to modify existing image elements in the VR. In the latter case, the image data must first be transformed so that it can be displayed as image elements. The additional image information is then presented on an output device, particularly a display. The transformed image elements can then be superimposed on a displayed VR or the physical reality to be viewed, thus creating an expanded information content of the AR.

[0018] The invention utilizes the combination of the reproduced image information to create a single image so that errors in the augmented reality display can be immediately and intuitively recognized by the viewer (e.g., a train driver). This requires that errors in the display of part of the information lead to image errors that are easily perceived intuitively. To achieve this, the two layers, each formed by one of the head-up displays, are advantageously available.

[0019] The functional principle of improved perceptibility of display errors is based on the fact that two overlapping layers can be used to display image errors, because image errors will, in the vast majority of cases, only occur in one of the two layers at a time. The probability that a display error goes unnoticed because both head-up displays fail simultaneously is significantly lower than the probability that such an error occurs in only one of the two layers. According to the invention, this effect significantly reduces the likelihood of overlooking errors in the augmented reality display and thus provoking potential malfunctions or even accidents during vehicle operation.

[0020] A further advantage of the display provided by two head-up displays is that, in many cases, the information can still be perceived by the user even if there are errors. This creates double security. The error in the augmented reality display can be detected, reported, and, if necessary, corrected, even if it does not yet negatively impact traffic operations. The information is supplemented by meaningful additions during the user's perception and can, in many cases, lead to the correct decision, possibly after confirmation by a dispatcher. This also increases the reliability of vehicle operation when errors occur.

[0021] In the context of the invention, "computer-aided" or "computer-implemented" can be understood to mean an implementation of the method in which at least one computer or processor carries out at least one method step of the method.

[0022] The term "computer" or "computer" covers all electronic devices with data processing capabilities. Computers can include, for example, personal computers, servers, handheld computers, mobile devices, and other communication devices that process data in a computerized manner, processors, and other electronic devices for data processing, which can preferably also be connected to a network.

[0023] In the context of the invention, a "processor" can be understood as, for example, a converter, a sensor for generating measurement signals, or an electronic circuit. A processor can be, in particular, a central processing unit (CPU), a microprocessor, a microcontroller, or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can also be understood as a virtualized processor or a soft CPU.

[0024] In the context of the invention, a "storage unit" can be understood as meaning, for example, a computer-readable memory in the form of a random-access memory (RAM) or data storage device (hard disk or data carrier).

[0025] "Interfaces" can be implemented in hardware, for example, as a wired or wireless connection, and / or in software, for example, as interaction between individual program modules or program parts of one or more computer programs.

[0026] "Program modules" are understood to mean individual functional units that enable a program sequence of method steps according to the invention. These functional units can be implemented in a single computer program or in several communicating computer programs. The interfaces implemented in this way can be implemented in software within a single processor or in hardware if multiple processors are used.

[0027] According to one embodiment of the invention, it is provided that the image information reproduces image elements which are constructed from first segments and second segments complementary to the first image segments, wherein the first segments are displayed by the first head-up display and the second segments are displayed by the second head-up display.

[0028] Complementary segments of the image elements within the meaning of the invention are image segments that stand alone and only together with the respective complementary image segments form an image element. The complementary segments thus abut one another, creating direct boundaries to the neighboring image segments of the other layer. Should image errors occur in one layer, they will be clearly visible through the adjacent image segments belonging to the other layer and still functioning. This is because the viewer is particularly sensitive to image errors that occur in directly adjacent segments.

[0029] Possible image errors could be the missing display of a segment or an offset of neighboring (complementary) segments belonging to different layers. These are advantageously made intuitively clear to the viewer. The viewer can react and, for example, report the error to a train dispatcher and, if necessary, request correct information or request the correction of a technical error related to the image error.

[0030] According to one embodiment of the invention, the image elements are generated by a pattern of stripes, the stripes being separated from one another by spaces.

[0031] The advantage of displaying image elements in a pattern of stripes is that a relative shift between the two layers leads to the formation of interference patterns. Interference is easily perceptible to the human eye and will immediately be noticed by a viewer even with small shifts. Therefore, stripe patterns are particularly suitable for supporting the intuitive perception of image defects.

[0032] According to this embodiment of the invention, the stripe patterns of one and the other layer are designed to complement each other. Thus, gaps between the stripes are not displayed, as both stripe patterns complement each other. Only a shift between the two layers in this case leads to the formation of gaps between the stripes, which are immediately noticeable to the viewer.

[0033] A failure in the display of the stripes in one layer will also be immediately noticeable. The complementary formation of the stripes will also result in visible gaps that the viewer will notice.

[0034] According to one embodiment of the invention, the stripes and spaces form a pattern of concentric circles in groups.

[0035] The above remarks regarding stripes apply to concentric circles. However, concentric circles offer the additional advantage that displacements between the two layers can be detected equally well in all directions, since the same interference pattern always appears in each of the circles, regardless of the direction of displacement.

[0036] For all of the above-mentioned types of image information display in two (or more) layers, image errors are already noticeable when the image information can still be perceived by the viewer due to the minimal overall impact of the image error. Image errors indicate a faulty display, even if the viewer may still be able to perceive the image information. Therefore, an image error is advantageously identified at a relatively early stage and can be used as an opportunity to perform maintenance on the head-up display before further, and especially more serious, malfunctions occur.

[0037] According to one embodiment of the invention, it is provided that the image information displayed by the first head-up display and the second head-up display is checked for image errors in a computer-assisted manner.

[0038] In addition to the intuitive assessment by the viewer, the method according to the invention can also be used for automated control of the displayed image or image to be displayed. For this purpose, the image-generating image data or the generated image must be checked. This advantageously supports and automates the operation of the vehicle and the associated detection of errors in the display. For example, a warning signal can be issued if image errors are automatically detected during the method, prompting the viewer of the image to recognize the image error and initiate appropriate measures.

[0039] According to one embodiment of the invention, the computer-aided check is carried out by taking a picture of the displayed image information with a camera and checking the image for image errors.

[0040] This embodiment of the invention is suitable for checking the actual image. This has the advantage that image errors caused by the hardware can also be detected. An offset between the two displayed layers can occur, for example, if the projectors projecting the images onto the head-up display are no longer correctly aligned. Another possibility is that defective pixels are present in the (active, i.e., self-illuminating) head-up display.

[0041] According to one embodiment of the invention, it is provided that the data for generating the image information are checked for errors.

[0042] In this embodiment of the invention, the data to be used to display the image elements is checked. These can, for example, be compared with standardized data sets stored in a memory device. This method is particularly suitable for detecting errors that occur when displaying symbols or text and other standardized image elements. Here, it is easy to compare the data sets with standardized data sets. The advantage is that image errors can also be detected that would otherwise lead to correct reproduction of the (faulty) image information by the hardware.

[0043] According to one embodiment of the invention, if image errors are detected, an error output is provided.

[0044] As already mentioned, the error output supports the derivation of measures that should be taken following the detection of image errors for safety reasons. For example, in the case of minor image errors that do not compromise the information content of the head-up display, maintenance measures can be derived that should be implemented in a timely manner to prevent the image errors from becoming more severe (for example, an increasing offset in the display of the layers, which would require projectors to be realigned to the head-up display).

[0045] In the event of major faults, measures must be taken to ensure the continued safe and reliable operation of the vehicle. One measure could be, for example, having an external agency take over or support the vehicle's operation. A drastic measure could be a shutdown if the safety risk of continued operation is deemed too high. After the head-up display has been repaired, operations can then be resumed or the vehicle replaced.

[0046] According to one embodiment of the invention, the error output is provided on the first and / or second head-up display.

[0047] This has the advantage that no additional display medium is required for error output. However, this requires that the head-up displays are still at least partially functional.

[0048] According to one embodiment of the invention, the error is output by issuing a warning signal.

[0049] The warning signal can be issued visually, for example via a warning light, or acoustically, for example via a loudspeaker. An output device that is independent of the head-up display's display can be used to output the warning signal. This has the advantage that the warning signal can also be issued if the head-up display is no longer capable of doing so due to the fault. Furthermore, a vehicle user will be more aware of a warning signal that differs from the usual display media.

[0050] According to one embodiment of the invention, it is provided that the image information is provided by projecting light by means of a first projector onto the first head-up display and a second projector onto the second head-up display, wherein the first head-up display reflects only the light of the first projector and the second head-up display reflects only the light of the second projector.

[0051] This type of head-up display is particularly well-suited for retrofitting in vehicles. The projectors can be mounted in a suitable location in the driver's cab or cockpit. Furthermore, the vehicle's windshield, for example, must be coated with suitable materials.

[0052] According to one embodiment of the invention, it is provided that the first head-up display and the second head-up display have active pixels which can be activated to display the image information.

[0053] This type of head-up display has the advantage of being very resistant to shifting of the layers. The head-up display is also relatively easy to read in different lighting conditions.

[0054] According to one embodiment of the invention, it is provided that a secure computer is used to operate the method.

[0055] The use of secure computers has the advantage that the high safety requirements, for example, in rail traffic can be met even when operating head-up displays. The operation of the head-up displays is thus based on an infrastructure of secure computers that are commonly provided in railway operations (e.g., according to the SIL-3 standard). The verification of the image elements displayed on the head-up displays is also performed by a secure computer, allowing the automatic detection of display errors to be carried out safely.

[0056] According to one embodiment of the invention, it is provided that the components involved in the method are subjected to a functional test at intervals.

[0057] Regularly checking the components of the display system has the advantage of further increasing reliability. The goal is to guarantee the improved safety provided by the ability to detect display errors, while still keeping the probability of detected failures as low as possible. This improves the availability and reliability of the equipped vehicle.

[0058] The stated object is alternatively also achieved according to the invention with the subject matter of the claim specified at the outset (display device for augmented reality) in that the display device is designed to carry out a method according to one of the preceding claims.

[0059] The stated object is alternatively also achieved according to the invention with the subject matter of the claim specified at the outset (vehicle with a display device for augmented reality) in that the display device is designed to carry out a method according to one of the preceding claims.

[0060] Both devices allow the advantages already explained in connection with the method described in more detail above to be achieved. The statements regarding the method according to the invention also apply accordingly to the device according to the invention.

[0061] Furthermore, a computer program product with program instructions for carrying out the said method according to the invention and / or its embodiments is claimed, wherein the method according to the invention and / or its embodiments can be carried out by means of the computer program product.

[0062] Furthermore, a provision device for storing and / or providing the computer program product is claimed. The provision device is, for example, a storage unit that stores and / or provides the computer program product. Alternatively and / or additionally, the provision device is, for example, a network service, a computer system, a server system, in particular a distributed, for example, cloud-based computer system and / or virtual computer system, which stores and / or provides the computer program product, preferably in the form of a data stream.

[0063] The provision takes place in the form of a program data block as a file, in particular as a download file, or as a data stream, in particular as a download data stream, of the computer program product. However, this provision can also take place, for example, as a partial download consisting of multiple parts. Such a computer program product is, for example, read into a system using the provision device, so that the method according to the invention is executed on a computer.

[0064] Further details of the invention are described below with reference to the drawings. Identical or corresponding elements of the drawings are provided with the same reference numerals and are explained several times only to the extent that differences arise between the individual figures.

[0065] The exemplary embodiments explained below are preferred embodiments of the invention.

[0066] Furthermore, the described components can also be combined with the features of the invention described above, as long as they are within the scope of the appended claims.

[0067] They show: Figure 1 an embodiment of the device according to the invention (display device and vehicle with display device) with its functional relationships schematically, Figure 2 an embodiment of head-up displays according to the invention schematically, wherein these generate image elements by complementary representation, Figures 3 and 4 the creation of a picture element in the form of a number from complementary picture components, represented by the head-up display according to Figure 2 , schematically, the representation according to Figure 3 does not contain any image defects and the representation is in accordance with Figure 4 has an image defect, Figure 5a comparative example, which is not the subject of the invention, for head-up displays, whereby these generate image elements by additive (redundant) representation, Figure 6 a comparative example, which is not the subject of the invention, for a picture element in which an offset of the layers of the head-up displays according to Figure 5 become clear through interference patterns, Figure 7 a comparative example, which is not the subject of the invention, for a picture element in which an offset of the layers of the head-up displays according to Figure 5 can be identified by color, Figure 8 an embodiment of a computer infrastructure of the device according to Figure 1 as a block diagram, where the individual functional units contain program modules, each of which can run in one or more processors and the interfaces can accordingly be implemented in software or hardware, Figure 9an embodiment of the method according to the invention as a flow chart, wherein the individual method steps can be implemented individually or in groups by program modules and wherein the functional units and interfaces according to Figure 8 are indicated as examples.

[0068] According to Figure 1 A vehicle FZ is shown, the driver's cab of which is equipped with a windscreen FS so that a driver ZF can visually view the route in front of the vehicle FZ. The windscreen FS is also equipped with a first head-up display HUD1 and a second head-up display HUD2, which form a first layer LY1 and a second layer LY2 on the windscreen FS. In the embodiment according to Figure 1A transparent image is generated on the first layer LY1 by a first projector PJ1 and on the second layer LY2 by a second projector PJ2. The projection for image generation is indicated by dot-dash lines. A camera CM is also provided, which can record the generated images as well as the surroundings beyond the windshield FS.

[0069] The vehicle FZ is equipped with a first computer CP1. This computer controls the process for creating augmented reality in the vehicle FZ. For this purpose, the first head-up display HUD1 is connected to the first computer CP1 via a first interface S1, and the second head-up display HUD2 is connected via a second interface S2. Likewise, the first projector PJ1 is connected to the first computer CP1 via a third interface S3, the second projector PJ2 via a fourth interface S4, and the camera CM via a fifth interface S5.

[0070] The vehicle FZ is also equipped with a first antenna A1. The first antenna A1 symbolizes an antenna system that can receive various signals. For example, it is possible to exchange data with a second computer CP2 in a control center LZ via a second antenna A2 using a sixth interface S6. Furthermore, the first antenna A1 can exchange positioning signals from a satellite ST using a seventh interface S7.

[0071] In Figure 1 The letters A to E indicate subareas of the device according to the invention, which fulfill different functions for the method according to the invention. The functions in the method can already be indicated by these steps, but are explained in more detail below. The areas are: Area A: Multi-channel

[0072] Consistent multi-channel structure of a) Information generation (Secure Computer CP1), b) Projection (two projectors PJ1 and PJ2), c) Reflection (two semi-transparent foils as layers LY1 and LY2 in the front screen FS), d) Image display (only when taken together do the partial reflection TR1 of the first layer and the partial reflection TR2 of the second layer superimposed on the real image RB, which penetrates the front screen, result in the safe display of an image and warnings are possible in the event of deviations), e) Readback (two-channel playback of the information from the foils to the Secure Computer with the option of a safety-related reaction) and f) Tests of the components involved by the camera CM. Area B: Projection by projectors PJ1, PJ2

[0073] Two projectors project partial images onto two semi-transparent optically active films (layers LY1, LY2) embedded in the windscreen. These can be virtual track elements of a physically no longer existing infrastructure, or indications or markings of real track elements. Area C: Reflection on semi-transparent optically active films on or in the windscreen

[0074] The semi-transparent optically active foils only react to the respective projector PJ1, PJ2 and reflect a partial image towards the train driver ZF. Area D: Addition of the complementary or redundant partial images and the real image to form a meaningful whole.

[0075] Only with correct partial images and correct reflection do they complement each other to form a meaningful overall image. Additionally, it is possible to use interference effects to ensure that small deviations in the optical information result in visual warnings, see also Figure 6 . Area E: Reading back the displayed information

[0076] The display information from both slides is read back into the secure computer independently of each other. An appropriate security response can then be implemented based on a detected error. Area F: Recording the displayed information by the camera CM

[0077] The recorded display result is read back into the secure computer. An appropriate security response can then be implemented based on a detected image error.

[0078] In Figure 2 and 5The head-up displays HUD1, HUD2 with their layers LY1, LY2 are shown. The display can be used as part of the arrangement according to Figure 1 can be understood. Also visible are the first computer CP1 as well as the first interface S1 and the second interface S2.

[0079] The difference between Figure 2 and Figure 5 is that the two head-up displays HUD1, HUD2 in the case of Figure 2 Create complementary partial images via the layers LY1, LY2 and according to Figure 5 create redundant partial images. This is done in Figure 2indicated by the fact that the first partial reflection TR1 originates from the first layer LY1 and the second partial reflection TR2 originates from the second layer LY2. For the first layer LY1 and the second layer LY2, only the partial areas are represented by a line that are involved in the respective generation of the complementary partial image (this does not mean that the layers LY1, LY2 themselves must be interrupted, but merely that they are activated in complementary surface areas; in this respect, the representation is according to Figure 2 schematic).

[0080] The head-up displays HUD1, HUD2 according to Figure 2 and Figure 5 are according to the embodiment according to Figure 1 with reflective layers LY1, LY2. These are passive layers that are illuminated by the projectors PJ1, PJ2 according to Figure 1must be illuminated to generate the image. However, active layers LY1, LY2 are also conceivable, which generate the image based on active elements in the layers LY1, LY2. In this case, the layers can also be self-luminous, in which case no partial reflection TR1, TR2 is generated, but rather partial emissions, namely a first partial emission TE1, generated by the first layer LY1, and a second partial emission TE2, generated by the second layer LY2. The other configuration of the head-up displays HUD1, HUD2 remains the same. Projectors PJ1, PJ2 according to Figure 1 are then not required. Since the Figures 2 and 5 Since they would not differ structurally, the relevant arrows are each marked with the reference symbols TR1, TE1 as well as with the reference symbols TR2, TE2.

[0081] Instead of partial reflections TR1, TR2, partial emissions TE1, TE2 can also be generated by the two layers LY1, LY2. In this case, no projectors PJ1, PJ2 are used, as in Figure 1 shown, is used to create the image as a reflection, but the layers LY1, LY2 are active and can make individual image points (pixels) light up. This is how the image is created. In the following, the statements will therefore be made both for the partial reflection TR1, TR2 and analogously for the partial emissions TR1, TR2, insofar as these two functional principles have no impact on the statements made with regard to the representation. Unless it is important, the following will therefore refer to partial images TR1, TE1 and TR2, TE2, whereby these can be created by partial reflections TR one, TR two or by partial immissions TE1, TE2.

[0082] In Figure 5 is in contrast to Figure 2The entire area of the first layer LY1 and the second layer LY2 is active. This means that image elements are redundantly formed by both layers LY1 and LY2. This is indicated by the fact that the partial images TR1, TE1 and TR2, TE2 of the second layer LY2 are combined in an arrow, since the two partial images overlap.

[0083] The two Figures 2 and 5 can be seen how the real image RB, which penetrates both semi-transparent layers LY1, LY2, together with the partial images TR1, TE1 and TE2, TR2 results in the image which the train driver ZF (not shown) can perceive as augmented reality AR.

[0084] In Figure 3 For example, the number 2 is shown as an image element, which is generated complementarily by the partial image TR1, TE1 and the partial image TR2, TE2. The addition of both partial images (indicated by a plus sign) results in the complete number 2 behind the indicated equal sign.

[0085] The representation according to Figure 4 shows the representation of paragraph 2 according to Figure 3 , where an image error BF1 occurs. This consists in the fact that a segment SG (cf. Figure 3 ) of the partial image TR1, TE1 is not displayed. As can be seen from the overall representation of number 2, this segment is missing from the line representing number 2. The representation also provides a marker MK, which is generated by the first computer CP1 as soon as the error is detected, in order to alert the train driver ZF to this image error BF1. The marker can, for example, consist of a frame surrounding the image error BF, as shown, but can also have other designs, e.g., a colored, partially transparent marker, etc.

[0086] In Figure 6 A picture element BE2 is shown in the form of concentric circles, which are arranged according to the Figure 5The representation principle shown consists of redundant image data from both the first layer LY1 and the second layer LY2. If an image error occurs because these layers are shifted against each other, the Figure 6 The image aberration BF2 shown here results in interference patterns due to the shifted concentric circles. These are easily perceptible to the human eye, which is very sensitive to interference patterns. Therefore, when the two layers are shifted relative to each other, the two partial images TR1, TE1 and TR2, TE2 can be seen.

[0087] In Figure 7 another image element BE3 is shown, which is also designed according to the representation concept according to Figure 5was formed. Because the image information is redundant here, too, an offset in the layers LY1, LY2 causes the two partial images TR1, TE1 and TR2, TE2 to diverge. This is noticeable to the human eye due to the color inhomogeneity of the surface.

[0088] In Figure 7 The additively mixed color of image element 3 (without errors) is represented by cross-hatching. The two individual partial images therefore have simply hatched colors. Each hatching represents a color, with the cross-hatching created by an additive color mixture of the two individual colors. Of course, brightness overlays can also be used instead of color overlays, which can also be indicated by the hatching.

[0089] In Figure 8 The configuration can be done according to Figure 1, in particular recognize the interaction of the individual processors P1...P6 of the individual hardware components. A fourth processor P4 of the first computer CP1 controls a first processor P1 in the first projector PJ1 via the third interface S3 and a second processor P2 in the second projector PJ2 via the fourth interface S4, so that these two projectors project the image onto the first head-up display HUD1 and the second head-up display HUD2. The first layer LY1 of the first head-up display HUD1 has a fifth processor P5 and the second layer LY2 of the second head-up display HUD2 has a sixth processor P6. These two processors are able to report the generated image data back to the fourth processor P4 via the first interface S1 and the second interface S2 so that it can be examined for image errors.The actual image generated on the head-up display HUD1 and HUD2 can also be checked by the camera CM, which is equipped with a third processor P3. The third processor P3 can also transmit the camera image to the fourth processor P4 via the interface S5.

[0090] The fourth processor P4 is capable of comparing the returned image data (via P1, P2, P5, P6, or P3) with image patterns stored in a first memory unit SE1. This allows the generated images to be examined for image defects.

[0091] To select the required sample images from the first storage unit SE1, the first computer CP1 requires location information, which is determined via the seventh interface S7 using a satellite ST, for example, a GPS satellite. Furthermore, the second computer CP2 in the control center LZ can be used via the sixth interface S6 to transmit timetables and route data stored in a second storage unit SE2. These timetables and route data also make it possible to select the correct sample data from the first storage unit SE1.

[0092] The procedure can be Figure 9After the process has started, the first computer CP1 is activated. This searches for a GPS signal, for example, which the satellite ST outputs in an output step POS OUT and which is read in by the first computer CP1 in an input step POS IN (in principle, communication could also take place with a so-called Radio Block Center (RBC) - depending on the purpose of the connection, not shown). This is used for a localization step LOC for the vehicle in which the first computer CP1 is installed. For this purpose, the data of a route plan MAP, which is stored in the first storage unit SE1, is also used. In a generation step GEN, data for the augmented reality AR is generated, whereby the data of a timetable FP, which is available in the second storage unit SE, is also used.This is followed by an output step AR OUT, in which data for the augmented reality AR is output and, after the two head-up displays have started, is read into the fifth processor P5 (input step AR IN) and the sixth processor P6 (AR IN).

[0093] If the head-up displays HUD1 and HUD2 are active displays, the information from the first layer LY1 and the second layer LY2 can be output by processors P5 and P6 in a first display step DSP1 and a second display step DSP2. This creates an AR in the manner already described, by overlaying the displays on the real image visible through the windshield FS.

[0094] In the variant in which the first layer LY and the second layer LY2 are reflection layers, the first projector PJ1 can be controlled by the fifth processor P5 via an eighth interface S8 such that the first projector PJ1 is controlled by the first processor P1. Likewise, the sixth processor P6 is connected to the second processor P2 via a ninth interface S9, so that the sixth processor P6 can control the second projector PJ2. In this way, the image is output by the head-up displays HUD1, HUD2 via the processors P1, P2 in the display step DSP1, DSP2.

[0095] The processors P5, P6 (alternatively the processors P1, P2) can further transfer the said data to the first computer CP1 in an output step for display data DSP OUT, which then reads this data in an input step for display data DSP IN. Additionally or alternatively, after the output step for the AR data AR OUT, the processor P3 in the camera CM can be activated, allowing the camera to perform a recording step CTP for the displayed data HUD1, HUD2. This is the image that the train driver ZF also sees. The data recorded by the camera CM is also transferred to the first computer CP1.

[0096] The collected data, i.e. the display data and the recorded displayed data, are evaluated in a test step (CHK) for the display to detect any image errors. This is followed by a query step (ERR?), which asks whether errors were detected. If this is not the case, the process starts again in the first computer (CP1) with the input step for the position data (POS IN). If, however, an error is detected, an output step (ERR OUT) for the data is carried out, which leads to a similar input step (ERR IN) in the control center (LZ). In the second computer (CP2) used there, an MSR measure is derived from this, indicating how to respond to the error.

[0097] Data from the dynamic timetable FP, which is stored in the second storage unit SE2, must also be included here (the dynamic timetable is understood to be the regularly updated data that reflects actual train traffic). The MSR measure then leads to a correction step COR, which is carried out by the first computer CP1 and aims to ensure that the error does not affect traffic safety. A subsequent query step ERR? checks whether the error continues to have an impact. If so, feedback is provided and a repeated output step ERR OUT for the error is output. If the error no longer exists, however, an output step QT OUT is carried out to abort the correction process, which leads to an input step QT IN in the second computer CP2. The MSR measure is therefore aborted and the process begins again with the input step POS IN for the position data. List of reference symbols

[0098] LZControl Center FZVehicle FSWindscreen RBreal Image ZFOperator A1 ... A2Antenna CP1 ... CP2Computer P1 ... P6Processor HUD1 ... HUD2Head-up Display LY1 ... LY2Layer TR1 ... TR2Partial Reflection TE1 ... TE2Partial Emission PJ1 ... PJ2Projector TR1 ... TR2Partial Reflection CMCamera S1 ... S13Interface BE1 ... BE3 Image element SGSegment BF1 ... BF3 Image error MKMarker POS OUTOutput step position data POS INSput step position data LOCLocalization step MAPRoute plan GENGeneration step for AR AR OUTOutput step for AR data AR INInput step for AR data DSP1 ... DSP2Display step for AR DSP OUTOutput step for display data DSP INInput step for display data CTPAcquisition step for displayed data CHKCheck step for display ERR?Query step for errors ERR OUTOutput step for errors ERR INInput step for errors FPSchedule MSRMeasure CORCorrection step QT OUTOutput step for cancellation QT INInput step for cancellation

Claims

1. Method for computer-assisted displaying of an augmented reality (AR) by way of a first head-up display (HUD1) and a second head-up display (HUD2), which are arranged in a vehicle (FZ) in the field of view intended for a vehicle driver, wherein the first head-up display (HUD1) and the second head-up display (HUD2) • are arranged in two layers (LY1, LY2) that overlap one another, characterised in that • the first head-up display (HUD1) and the second head-up display (HUD2) display image information which supplement one another to form an image, • the image information renders image elements (BE1 ... BE3), which are constructed of complementary first segments and second segments, wherein first segments and second segments stand on their own in each case, first and second segments come up against one another, and thus together form the respective image element (BE1 ... BE3), • wherein the first segments are displayed by the first head-up display (HUD1) and the second segments are displayed by the second head-up display (HUD2).

2. Method according to claim 1, characterised in that the image elements (BE1 ... BE3) are generated by a pattern of strips, wherein the strips are separated from one another by intermediate spaces.

3. Method according to claim 2, characterised in that the strips and intermediate spaces form a pattern of concentric circles in groups.

4. Method according to one of the preceding claims, characterised in that a review of the image information displayed by the first head-up display (HUD1) and the second head-up display (HUD2) for image errors (BF1 ... BF3) takes place on a computer-assisted basis.

5. Method according to claim 4, characterised in that the computer-assisted review is performed by • an image of the displayed image information being recorded using a camera (CM), and • the image is reviewed for image errors (BF1 ... BF3).

6. Method according to one of the preceding claims, characterised in that the data for generating the image information is reviewed for errors.

7. Method according to claim 4 to 6, characterised in that if image errors (BF1 ... BF3) are ascertained, an error output (ERR OUT) takes place.

8. Method according to claim 7, characterised in that the error output (ERR OUT) takes place on the first and / or second head-up display (HUD1, HUD2).

9. Method according to claim 7 or 8, characterised in that the error output (ERR OUT) takes place by outputting a warning signal.

10. Method according to one of the preceding claims, characterised in that the image information takes place by way of a projection of light by means of a first projector (PJ1) onto the first head-up display (HUD1) and a second projector (PJ2) onto the second head-up display (HUD2), wherein the first head-up display (HUD1) only reflects the light of the first projector (PJ1) and the second head-up display (HUD2) only reflects the light of the second projector (PJ2).

11. Method according to one of the preceding claims, characterised in that the first head-up display (HUD1) and the second head-up display (HUD2) have active image points, which can be activated to render the image information.

12. Method according to one of the preceding claims, characterised in that a secure computer (CP1) is used for the operation of the method.

13. Method according to one of the preceding claims, characterised in that the components involved in the method are subjected to a functional test at time intervals.

14. Display apparatus for an augmented reality (AR), characterised in that the display apparatus is configured to perform a method according to one of the preceding claims.

15. Vehicle (FZ) with a display apparatus for an augmented reality (AR), characterised in that the display apparatus is configured to perform a method according to one of claims 1 - 13.

16. Computer program product with program commands for performing the method according to one of claims 1 - 13.

17. Provision apparatus for the computer program product according to the last-mentioned claim, wherein the provision apparatus stores and / or provides the computer program product.