METHOD AND DEVICE FOR PICTURING THE INTERIOR OF A VEHICLE, AND VEHICLE AND COMPUTER PROGRAM PRODUCT

Abstract

A method for capturing images of a vehicle's interior comprises: capturing the vehicle's interior using an IR camera (infrared camera) to generate initial image data representing the vehicle's interior; performing a comparison of the initial image data with: a) Camera image data generated by optically capturing the interior of the vehicle using a visible spectrum camera and / or b) Information that characterizes a human body, human body parts, articles of clothing or vehicle components, or that is suitable to assist in the identification or recognition of a human body, human body parts, articles of clothing (underwear) or vehicle components, to obtain matching information from the comparison; the method further includes the generation of second image data based on the first image data and / or the camera image data, taking into account the matching information. The procedure can utilize nudity detection, body keypoints, and inpainting.

Description

[0001] The present invention relates to a method and a device for capturing images of the interior of a vehicle, as well as a vehicle with such a device, and a corresponding computer program or computer program product.

[0002] Vehicles equipped with cameras for capturing images of the vehicle interior are already known in the prior art. In some cases, IR cameras (short for "infrared cameras") are used. For example, DE 10 2008 037 815 A1 discloses a device equipped with an IR camera for capturing at least a portion of the passenger compartment. The data captured by the IR camera enables a visual representation of the passenger compartment on a display. The IR camera is intended to display a thermal image of the passenger compartment, including the surface temperature of the passengers.

[0003] The inventors of the present application have recognized that, for example, polyester clothing is at least partially or substantially transparent to infrared radiation (IR radiation), at least at certain wavelengths. Although, according to the aforementioned DE 10 2008 037 815 A1, the intention is to generate a thermal image in which areas with different surface temperatures are displayed in different colors, so that more precise details, for example of body parts of vehicle occupants, are not necessarily recognizable, it cannot be ruled out that, unintentionally, at least contours of body parts, for example in the genital area or of underwear, become visible.

[0004] Furthermore, the inventors believe it would be desirable to be able to display not just a thermal image of the vehicle interior, but a more realistic representation. However, the inventors also recognize that optical scanning, for example with an RGB camera, does not always provide a satisfactory representation of the vehicle interior in darkness.

[0005] Against this background, it is a task of the present revelation to at least mitigate the problems discussed above.

[0006] A solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the solution are the subject of the dependent claims.

[0007] A first aspect of the present disclosure relates to a method for capturing an image of the interior of a vehicle, comprising: Capturing the interior of the vehicle using an IR camera to generate initial image data that represents a depiction of the vehicle's interior;

[0008] Performing a comparison of the initial image data with: a) Camera image data generated by optically capturing the interior of the vehicle using a visible spectrum camera and / or b) Information that characterizes a human body, human body parts, articles of clothing or vehicle components, or that is suitable to assist in the identification or recognition of a human body, human body parts, articles of clothing or vehicle components, to obtain matching information from the comparison; the method further includes the generation of second image data based on the first image data and / or the camera image data, taking into account the matching information. For the purposes of this disclosure, "initial image data" means data or information which – possibly after suitable processing – - enable a display of the vehicle's interior when played back on a suitable display device such as a vehicle display, for example the so-called Central Information Display (CID), or on a smartphone that may be (wirelessly) connected to the vehicle.

[0009] Based on the first aspect, it is therefore proposed to use an IR camera after all, whereby the (first) image data generated by the IR camera are generally not (or not necessarily) output 1:1, for example, to a display device suitable for image reproduction. Instead, second image data is generated, whereby the first image data generated by the IR camera ultimately influences the generation of the second image data. In this way, a visual representation of the vehicle's interior on a display device becomes possible, which is generally more realistic than a thermal image. At the same time, the unwanted reproduction of details captured by the IR camera (e.g., body parts in the genital area or underwear) on a display device can be avoided.

[0010] For the purposes of this disclosure, the term "capturing the interior of the vehicle" also includes capturing only a (possibly relatively small) part of the interior of the vehicle.

[0011] The camera for the visible spectrum could, for example, be an RGB camera.

[0012] The following describes various exemplary embodiments of the method, which, unless expressly excluded or technically impossible, can be combined with each other as desired and with the second, third and / or fourth aspects of the present disclosure described below.

[0013] The procedure may also exhibit: Use the matching information to determine whether a modification should be applied to the initial image data and / or the camera image data.

[0014] The matching information can therefore be used to determine whether a modification of the first image data and / or the camera image data is even necessary. It's also possible that the matching information will indicate that either the first image data or the camera image data can be used (essentially) unchanged as the second image data, meaning that no modification of the first image data and / or the camera image data is required. In many cases, however, the matching information will indicate that such a modification is necessary.

[0015] Determining whether a modification should be applied to the initial image data and / or the camera image data may involve the following: Determine whether the matching information meets one or more given criteria, and if so: Using the matching information to determine the modification to be applied to the initial image data and / or the camera image data, and

[0016] Generating the second image data by applying the modification to the first image data and / or the camera image data.

[0017] The given criterion or criteria may depend on whether camera image data or the aforementioned information (information relating to a human body, human body parts, articles of clothing, or vehicle components) is used for comparison with the initial image data. Non-restrictive examples of applicable modifications are explained below.

[0018] If the above-explained determination of whether the matching information meets a given criterion or several given criteria, i.e., one or more criteria that ultimately lead to the determination and application of a modification, yields the result that the given criterion or several given criteria are not met, the first image data and / or the camera image data can be used as second image data without modification.

[0019] In the disclosed procedure, the given criterion or one or more of the given criteria may relate to the extent to which the representation of the vehicle's interior represented by the first image data differs from a representation of the vehicle's interior represented by the camera image data.

[0020] Using established image processing techniques, the initial image data and the camera image data (or the representations they each depict) can be compared to identify deviations, and potentially also the severity of such deviations. However, this requires that suitable camera image data is actually available. A situation in which camera image data is available but unsuitable for direct display on a playback device would arise, for example, if, in near darkness with dim ambient light, the interior of a vehicle is just barely detectable by a camera operating in the visible spectrum, but would appear so dark on a playback device without significant brightening that the human eye would only be able to discern a few details.In this case, while the overall image could be brightened during playback, this could also lead to a very unnatural appearance. Furthermore, the image quality could be (significantly) reduced by high image noise. For example, according to the present disclosure, in such a case, it is possible to determine a measure of the deviation between the initial image data and the camera image data when comparing them. If the measure of deviation meets one or more criteria, for example, if the measure of deviation exceeds a certain threshold, this indicates that the IR camera has captured details that the camera operating in the visible spectrum did not, such as underwear worn under outerwear made of polyester.

[0021] In the disclosed method, a difference between the representation of the vehicle's interior as depicted by the first image data and the representation of the vehicle's interior as depicted by the camera image data can be determined using edge detection.

[0022] Techniques for edge detection are known in themselves and will therefore not be explained further.

[0023] It is known that images generated using an IR camera often do not display color accurately. A comparison between the initial image data and the camera image data could therefore reveal a significant discrepancy. While, according to the present disclosure, such discrepancies can be detected and used for corresponding corrections in the image reproduction, edge detection can also be used to detect image content that is captured by the IR camera but not by the camera operating in the visible spectrum. To return to the example of underwear worn under polyester outerwear: The camera operating in the visible spectrum would typically only capture the outerwear. The IR camera, on the other hand, would also capture the underwear.A comparison of the initial image data with the camera image data, which includes edge detection, could thus identify one or more edges of the underwear. Such an edge would not be visible in the camera image data. The additional edge present in the initial image data could therefore indicate underwear worn under polyester outerwear and be detected accordingly using the disclosed method.

[0024] In the disclosed method, the given criterion or one or more of the given criteria may relate to whether the representation of the vehicle's interior, as provided by the initial image data, includes one or more elements of the following categories: one or more of predefined body parts or body sections, in particular the genital area; or one or more of a predefined set of articles of clothing, in particular underwear; or Components of the vehicle, in particular a seat belt.

[0025] One possible implementation would, for example, provide that relevant information or data regarding the aforementioned elements (body parts, body sections, articles of clothing, vehicle components) can be retrieved and compared with the initial image data to determine whether a representation of the initial image data contains one or more of the aforementioned elements. Data regarding the aforementioned elements could, for example, be stored in the vehicle itself, in particular in a device designed to carry out the disclosed method, or in another vehicle component. However, the data could also be retrieved, for example, via a wireless interface, such as the internet.

[0026] If the comparison reveals that a representation of the first image data contains one of the aforementioned elements, the corresponding image content can then be modified appropriately.

[0027] As part of the matching process, a so-called "nudity detection" method can be used. The procedures involved are generally known and therefore will not be explained in detail.

[0028] Determining whether a representation of the initial image data contains a vehicle component could be limited to detecting a seat belt, particularly one extending in front of a person. A seat belt extending in front of a person can sometimes appear distracting in an image captured with an IR camera due to its color representation (usually white). Detecting the seat belt extending in front of the person allows for modification of the corresponding image content. For example, a seat belt extending in front of a person can be identified by detecting two parallel edges extending diagonally across the person's torso.

[0029] A seatbelt that does not extend in front of a person, as well as other vehicle parts, may appear less intrusive, so that no modification of the corresponding image content might be necessary with regard to these vehicle parts.

[0030] In the disclosed procedure, determining whether the matching information meets one or more given criteria can be done with the aid of Body Keypoints.

[0031] The use of body keypoints can help ensure that the process selectively modifies image content relating to people. Other image content, such as vehicle seats, the vehicle headliner, etc., could be excluded from modification, even if the comparison with the initial image data reveals, for example, that an article of clothing, identified as underwear, is lying on a vehicle seat.

[0032] Furthermore, by using body keypoints, the procedure can be focused on modifications in areas of the human body that are considered intimate. For example, the procedure can be configured to suppress modification of the initial image data or the camera image data if the comparison of the initial image data with the camera image data indicates that the initial image data differs from the camera image data in terms of its content, for example, in the area of ​​a person's forearm. Conversely, a modification would be performed if the detected differences were associated with the intimate area.

[0033] The procedure can be configured so that the identified modification only affects a sub-area of ​​the representation of the vehicle's interior represented by the first image data or the representation of the vehicle's interior represented by the camera image data.

[0034] Such a sub-area can, for example, concern a part of the body, a body part, in particular the intimate area, a piece of clothing, in particular underwear, or components of the vehicle.

[0035] In the disclosed method, the modification can cause the portion of the representation of the vehicle's interior to appear in at least one of the following ways when displayed on a screen: - opaque - with a predetermined pattern - replaced by inpainting with different image content - replaced by adjacent image content or an essentially continuous continuation of the same.

[0036] In general terms, the modification can result in the interior of the vehicle being presented in a way that is pleasing or familiar to the human eye, or at least comes closer to this goal than if no modification were applied.

[0037] When displayed as an opaque area or with a predetermined pattern, the corresponding area is, in effect, covered with one or more colors. This can be done more or less without regard to other image content.

[0038] Using inpainting and with continuous processing, the affected area can be replaced with other image content in a meaningful and natural way. This can also involve accessing other information, such as information stored in an external database. In particular, deep neural networks or a GAN (Generative Adversarial Network) can be used for this purpose.

[0039] The second image data can be generated in such a way that the second image data represents a modified representation of the vehicle's interior.

[0040] In this way, a user receives a useful image of the vehicle's interior, while avoiding the problem of displaying unwanted image content.

[0041] Alternatively, the second image data can be generated in such a way that it does not represent—not even a modified—the vehicle's interior. When the second image data is displayed on a screen, for example, a message can be shown indicating to the user that displaying a representation of the vehicle's interior is not possible (at that time).

[0042] The method or corresponding device could also be configurable accordingly, i.e., the method or corresponding device can be determined during manufacturing or by a user of the vehicle as to whether, in the presence of people or image content that would normally be undesirable without modifications (body parts of the intimate area, underwear, etc.), a modified representation of the vehicle interior should be displayed, or the information that the representation of the vehicle interior is not possible.

[0043] In the disclosed method, the generation of the initial image data and the camera image data may involve capturing the interior of the vehicle from the same position or substantially the same position and / or along the same optical axis or substantially along the same optical axis.

[0044] This can simplify the comparison between the initial image data and the camera image data, thus reducing the computational effort and potentially delivering more accurate or satisfactory results than generating different positions or along different optical axes.

[0045] A second aspect of the present disclosure relates to a computer program with a program code or a computer program product with program code stored on a medium readable by a computer, wherein the program code contains instructions which, when executed by a computer, cause it to perform one of the procedures described above.

[0046] A third aspect of the present disclosure relates to a device for capturing an image of the interior of a vehicle, comprising: an IR camera to generate initial image data representing the interior of the vehicle; and a data processing device for performing a comparison of the initial image data with: a) Camera image data generated by optically capturing the interior of the vehicle using a visible spectrum camera and / or b) Information that characterizes a human body, human body parts, articles of clothing or vehicle components, or that is suitable to assist in the identification or recognition of a human body, human body parts, articles of clothing or vehicle components, to obtain matching information from the comparison; wherein the data processing device is configured to generate second image data based on the first image data and / or the camera image data, taking into account the matching information.

[0047] The device may already be installed in a vehicle or may be available separately for later installation in a vehicle. It should also be noted that the device and the methods described above may be configured to generate the second set of image data without displaying this image data on a playback device by the device or within the framework of the method. However, the present disclosure also proposes a device or method that includes the playback of the image data by the device or the method.

[0048] A fourth aspect of the present disclosure relates to a vehicle comprising a device as described above.

[0049] A previously described computer program or computer program product may optionally be used in conjunction with an optical detection device according to the third aspect of the present disclosure or in a vehicle according to the fourth aspect of the present disclosure.

[0050] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example, the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules. The modules can, in particular, be configured, or at least be usable, in such a way that they function in the sense of distributed computing (i.e., distributed computing)."Distributed computing" is performed on different devices (computers or processor units) that are geographically separated and connected via a data network.

[0051] The features and advantages described in relation to the first aspect of the revelation and its advantageous design also apply, at least where technically appropriate, to the other aspects of the revelation and their advantageous design, and vice versa.

[0052] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.

[0053] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive "or" and not an exclusive "or". For example, a condition A or B is satisfied by each of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0054] The terms "ein" or "eine," as used here, are defined as "one or more" or "at least one." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0055] The term "plural", as used here, as well as any other variant thereof, is to be understood in the sense of "two or more".

[0056] The term "connected" and any variant thereof is to be understood in the sense of a connection for data transmission or transmission of electrical and / or optical signals - at least when the context does not indicate that it is a purely mechanical connection.

[0057] The terms "configured" or "set up" to perform a specific function (and their respective variations) are used here to mean that the device in question is already in a configuration or setting in which it can perform the function, or at least that it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predefined configurations or operating modes, allowing configuration to be performed by selecting one of these configurations or operating modes.

[0058] Further features, advantages and applications of the revelation will be revealed in the following description in connection with the figures, in which the same reference signs are consistently used for the same or corresponding elements of the revelation.

[0059] For the sake of clarity, the figures are at least partially schematic or greatly simplified. Fig. Figure 1 schematically shows an interior view of a motor vehicle according to one embodiment of the present disclosure. Fig. Figure 2 schematically shows a person. Fig. Figure 3 shows a schematic representation of the person from Fig. 2 according to one interpretation of the present disclosure. Fig. Figure 4 schematically shows a device according to one embodiment of the present disclosure. Fig. Figure 5 schematically shows a part of a device according to one embodiment of the present disclosure. Fig. Figure 6 schematically shows another representation of the person from Fig. 2 according to one interpretation of the present disclosure. Fig. Figure 7 schematically shows another representation of the person from Fig. 2 according to one interpretation of the present disclosure. Fig. Figure 8 schematically shows a display device according to one embodiment of the present disclosure. Fig. Figure 9 shows a flowchart to illustrate a process according to one embodiment of the present disclosure.

[0060] Fig. Figure 1 schematically shows an interior view of a motor vehicle 1 according to one embodiment of the present disclosure. The in Fig. The view shown in Figure 1 corresponds approximately to the view from a central position within the vehicle 1 looking forward through the windshield 12 of the vehicle 1. The vehicle 1 has, as is typical, a rearview mirror 11 and a display 16, such as a CID 16, which is integrated into the dashboard 13 of the vehicle 1. The vehicle 1 also has an on-board computer 17. This can, for example, be integrated into the dashboard 13 in such a way that it is not visible from the vehicle interior. A passenger 4 is seated in the right front seat. A steering wheel (without reference markings) is indicated in the area of ​​the driver's seat.

[0061] Vehicle 1 has an IR camera 2, which in the example shown is attached to the underside of the rearview mirror 11. However, the IR camera 2 could also be mounted at other (suitable) locations on vehicle 1. In addition to the IR camera 2, vehicle 1 also has a camera 3 for the visible spectrum. In the example shown, this camera is also attached to the underside of the rearview mirror 11, but it could also be mounted at other (suitable) locations on vehicle 1. In the example shown, the IR camera 2 and the RGB camera 3 are positioned very close to each other. The detection areas (fields of view) of the IR camera 2 and the RGB camera 3 overlap and are ideally as congruent as possible.

[0062] Fig. Figure 2 schematically shows a person 4, for example, the one in Fig. Person 4 is shown in the passenger seat, or in another seat of the vehicle 1, for example, a rear seat. In the example shown, person 4 is wearing a blouse 18 made of polyester or a polyester-containing material as outerwear 18. Fig. Figure 2 shows person 4 as they might appear on a display 16 if person 4 had been recorded by a camera 3 for the visible area.

[0063] Fig. Figure 3 shows a schematic representation of person 4. Fig. 2 according to one interpretation of the present disclosure. Fig. Figure 4 shows person 4 as they might be displayed on a display 16 if person 4 had been recorded by the IR camera 2 and the (first) image data 5 supplied by the IR camera 2 (see Fig. 4 and the following explanation) would not have been edited before being displayed on the screen 16. Because polyester is transparent to IR radiation – at least partially or in a limited area of ​​the IR spectrum – the underwear 8 of person 4 could potentially be visible through the polyester blouse 18 on the screen 16, which might be undesirable for person 4 and possibly for a viewer of the screen 16.

[0064] Fig. Figure 4 schematically shows a device 24 according to an embodiment of the present disclosure. In the example shown, the device 24 comprises an IR camera 2, a camera 3 for the visible range, for example an RGB camera 3, and a data processing device 10. For example, the Fig. The on-board computer 17 shown in Figure 1 can serve as a data processing device 10, or the data processing device 10 can be integrated into the on-board computer 17. For data transmission, the data processing device 10 is connected to the IR camera 2 and the RGB camera 3 either wired or wirelessly. In the Fig. In the example shown in Figure 4, the data processing device 10 is further connected to a database or storage device 9 and a display 16 for data transmission.

[0065] The data processing device 10 can have a wireless interface 14 for wireless connection to the IR camera 2 and / or to the RGB camera 3 and / or to the database or storage device 9 and the display 16. In one variant, one or more of the aforementioned data transmission connections of the data processing device 10 are implemented via wired connections (solid lines in Fig. 4).

[0066] It will now also be on Fig. 9. Referenced. Fig. Figure 9 shows a flowchart illustrating a method according to one embodiment of the present disclosure. After the start 30 of the method, in step 31 the interior of the vehicle 1 is captured using the IR camera 2 to generate initial image data 5, which represent a depiction of the interior of the vehicle 1. As shown by Fig. As explained in section 3, these initial image data 5 can represent a person 4 including their underwear 8, and possibly also a representation of body parts in the intimate area of ​​this person 4. The initial image data 5 are forwarded from the IR camera 2 to the data processing device 10.

[0067] In step 32, the data processing device 10 performs a comparison. During this comparison, the initial image data 5 is compared with camera image data 6 from the RGB camera 3. For this purpose, the RGB camera 3 also captured the interior of the vehicle 1 to generate camera image data 6. The camera image data 6 is also forwarded to the data processing device 10 so that the comparison between the initial image data 5 and the camera image data 6 can take place there.

[0068] When comparing the first image data 5 with the camera image data 6, the data processing device 10 determines differences between them in order to obtain comparison information 7 from the comparison. Various methods, known per se, for determining differences between image data can be used here. A suitable example uses edge detection in the first image data 5 and the camera image data 6. One based on the in Fig. 3. The depicted representation of the person shown in 4. The applied edge detection would, for example, detect all in Fig. 3. Determine the edges shown, including the outlines or other edges or contours of the underwear. 8. One on the in Fig. The representation of person 2 shown, however, would only detect the edge detection applied to person 4, as shown in the image. Fig. 2. Determine the outlines or other edges or contours shown, but not the outlines or other edges or contours of the underwear. 8. A comparison performed using edge detection (so to speak, “edges of the Fig. 3" minus "edges of the Fig. 2") would therefore provide as matching information 7 exactly the outlines or other edges or contours of the underwear 8.

[0069] In a subsequent step 33, second image data 21 are then generated, based on the first image data 5 and / or the camera image data 6, taking into account the matching information 7. The procedure can then end (34).

[0070] The data processing device 10 can also generate the second image data 21. The second image data 21 generated in this way can then be transmitted from the data processing device 10 to the display device 16, where it can be displayed.

[0071] As described above, the matching information 7 refers to differences between the initial image data 5 and the camera image data 6.

[0072] The generation of the second image data 21 can be carried out, for example, as follows: If the data processing device 10 determines that the camera image data 6 is suitable for display on the display device 16 without modifications (for example, if the image content is rated as sufficiently bright), the camera image data 6 can be used unchanged as the second image data 21 or passed on to the display device 16. However, if the data processing device 10 determines that the camera image data 6 is not suitable for display on the display device 16 without modifications (for example, if the image content is rated as insufficiently bright), the data processing device 10 can use the matching information 7 to modify the camera image data 6 and / or the first image data 5. In particular, the data processing device 10 can assess the extent to which the camera image data 6 or the first image data 5 are suitable for display on the display device 16 without modifications.The first image data 5 are to be used as the basis for the second image data 21 – for example, based on an assessment of how well details are recognizable in the camera image data 6, which in turn is at least partially based on the brightness of the image content represented by the camera image data 6. Depending on the situation, the second image data 21 can result, for example, as a mixture of the first image data 5 and the camera image data 6. Including the first image data 5 can make details that are not easily recognizable based solely on the camera image data 6 clearer. On the other hand, the data processing device 10 also takes the comparison information 7 into account. To return to the example of edge detection: The data processing device 10 may detect a group of edges that are present in both the first image data 5 and the camera image data 6, but are too dark in the camera image data 6.The data processing device 10 then uses, at least for the image areas corresponding to these edges, the first image data 5 or an enhanced version of the camera image data 6 as the second image data 21 for transmission to the display device 16. However, if the data processing device 10 detects another group of edges that are present in the first image data 5 but not in the camera image data 6, this could indicate areas of the vehicle interior whose depiction might not be desired, for example, underwear 8 of vehicle occupants or parts of the genital area. Therefore, for the image areas corresponding to these edges, the first image data 5 are not used to generate the second image data 21, or only with additional modifications.

[0073] According to the present disclosure, several possibilities are provided for the modifications to be applied in order to generate the second image data 21. These are described below.

[0074] Fig. Figure 6 shows a first variant in which the image area to be modified – here corresponding to the upper body of person 4 – is covered opaquely, for example in the form of an opaque, monochrome rectangle 19. Other shapes are also possible, for example a shape that corresponds to the outline of person 4 or their upper body.

[0075] Fig. Figure 7 shows a second variant, which is a modification of the design according to Fig. This can be viewed in image 6. Here, the image area to be modified is not covered by a solid-color rectangle, but by a rectangle 20 with a predefined pattern. Other shapes are also possible here.

[0076] In another variant, the image area to be modified is not defined as in the Fig. 6 and Fig. 7 is replaced by a geometric shape whose content has little or nothing to do with the image content captured by IR camera 2 or camera 3 for the visible range; rather, the image area to be modified is replaced in such a way that it resembles the adjacent, unmodified image area, for example, with regard to color, pattern, etc. This would, for example, apply to the image area in Fig. The image content shown in section 3 means that the image content, insofar as it relates to the underwear 8, is replaced by the image content of the blouse 18 adjacent to the underwear 8, so that the modified image content corresponds to the one shown in Fig. 2 resembles the depicted image content.

[0077] It has already been mentioned that, as in Fig. As shown in Figure 4, the data processing device 10 is also connected to a database or storage device 9. This database or storage device 9 can support the recognition of specific image content, for example, body parts or genital areas and underwear. This can be used as follows: First, differences between the initial image data 5 and the camera image data 6, or the image content represented therein, are determined as described above. The differences thus determined are then potential candidates for one or more image areas to be modified. Using the database or storage device 9, it is then determined whether the potential candidates relate to potentially undesirable image content (genital area, underwear), i.e., whether they form a shape similar to known shapes of genital areas or underwear.If this is the case, the corresponding image areas are modified as described above (for example, made opaque in a single color, covered with a pattern, or by continuously extending the adjacent image content). If the potential candidates do not relate to potentially undesirable image content, there is no reason for such modification, so the image content corresponding to these potential candidates, represented by the first image data 5, can be used in whole or in part when generating the second image data 21.

[0078] Additionally or alternatively, the potential candidates are checked to see if they are located in areas or positions, particularly in relation to Person 4, that are consistent with typical positions of body parts in the genital area or underwear. So-called body keypoints can be identified and used for this purpose. Suitable techniques for identifying body keypoints are already known and are therefore not described in detail. Thus, if it is determined that a potential candidate is located, for example, on Person 4's forearm, it is determined that there is no reason to modify the corresponding image area.

[0079] Modification can also be achieved using so-called "inpainting." For example, if the data processing device 10 determines that person 4 is wearing a blouse 18, but underwear 8 is also visible in the area of ​​blouse 18 represented by the first image data 5, inpainting could replace the entire area of ​​blouse 18 in the first image data 5 with a suitable garment (outerwear). Suitable inpainting techniques are already known and will therefore not be described in detail.

[0080] The database or storage device 9 can either be integrated into or connected to the data processing device 10. In particular, the database or storage device 9 can be wirelessly connected to the data processing device 10, for example via the Internet or another network. The database or storage device 9 can, in particular, be AI-supported.

[0081] In one variant, the actual determination of whether the potential candidates for image areas to be modified relate to unwanted image content does not take place in the data processing device 10, but externally, for example in a server that is wirelessly connected to the vehicle 1.

[0082] In another variant, which is a modification of device 24 according to Fig. 4 can be viewed, the database or storage device 9 and its connection to the data processing device 10 are omitted, i.e. image content that was determined as a difference between the first image data 5 and the camera image data 6 is not additionally examined to determine whether it is actually unwanted image content, but this image content is replaced or otherwise modified as described above.

[0083] In another variant, which is also referred to as a modification of device 24 according to Fig. Since the image data can be viewed at 4, no comparison of the first image data 5 with camera image data 6 takes place. Instead, only a comparison of the first image data 5 with information takes place that—with the aid of the database or storage device 9 or the like—enables the detection of potentially undesirable image content of the type described above. Within this framework, it can be determined, for example, whether the first image data 5 relates to human body parts or articles of clothing such as underwear. The image content thus determined can then be replaced or otherwise modified as described above. According to this variant, the device 24 does not need to have a camera 3 for the visible spectrum.

[0084] In another variant, which is also referred to as a modification of device 24 according to Fig. As can be seen in section 4, the device 24 does not have a display device 16. In this variant, the data processing device 10 generates (and can output) the second image data 21 as described above. The second image data 21 can then be received by a display device that is not part of the device 24, and corresponding image content can be displayed.

[0085] Regarding the display unit 16 and its connection with other components of the in Fig. The device shown in section 24 also offers several possibilities, which are described below.

[0086] According to one variant, the data processing device 10 generates second image data 21 and forwards it to the display device 16 as described above, so that the corresponding image content can be displayed by the display device 16, in particular without any further content processing by the display device 16. This case is represented by the solid line 21 in Fig. Figure 4 illustrates this. In this case, no further image data or other information, such as the initial image data 5, the camera image data 6, or the calibration information 7, needs to be transmitted to the display device 16. Therefore, in this case, the dashed lines 5 and 6 and the dotted line 7 can / would be displayed. Fig. 4 are omitted.

[0087] According to another variant, the first image data 5 and / or the camera image data 6 as well as the alignment information 7 are transmitted to the display device 16, as described in Fig. 4 is illustrated by the dashed lines 5 and 6 and the dash-dotted line 7. The second image data 21 then does not need to be transmitted to the display device 16 and may not even need to be generated by the data processing device 10. Instead, the display device 16 generates the second image data 21 itself, namely based on the first image data 5 and / or the camera image data 6, taking into account the matching information 7. In this case, the solid line 21 could or would thus be Fig. 4 are omitted.

[0088] Fig. Figure 5 schematically shows a part of a device 24 according to one embodiment of the present disclosure. The one in Fig. The part of the device 24 shown in section 5 can be a part of the Fig. Device 24 shown in section 4. Fig. Figure 5 shows only the IR camera 2, the camera 3 for the visible spectrum, and a mirror 15, as well as the optical axes belonging to the IR camera 2 and the camera 3. The mirror 15 can be partially transparent to both visible light and the infrared range, so that both cameras 2 and 3 can capture the vehicle interior in their respective wavelength ranges. The cameras 2 and 3 and the mirror 15 are arranged and aligned such that the optical axes emanating from each camera 2 and 3 initially meet opposite sides of the partially transparent mirror 15 at an angle (for example, approximately 45°), but then essentially coincide as a common optical axis 22. This allows the two cameras 2 and 3 to capture the vehicle interior from the same or essentially the same viewing angle and, if necessary, provide congruent image content.

[0089] In one variant of this, the mirror 15 is designed to be (essentially) transparent to visible light and (essentially) reflect infrared radiation, so that the two cameras 2, 3 essentially only receive radiation of their respective spectrum.

[0090] Fig.Figure 8 schematically shows a display device 16 according to one embodiment of the present disclosure. According to this embodiment, the display device 16 does not show image content that essentially corresponds to the first image data 5 or the camera image data 6, but rather it (merely) displays information 23 from which a user or viewer of the display device 16 can deduce that a view of the vehicle interior cannot be displayed, for example, if undesirable image content has been detected and modifying the image content is not possible, or if – for example, due to corresponding settings of the device 24 – it is not desirable for modified image content to be displayed.

[0091] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide guidance for those skilled in the art in implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents.The features described herein may be combined with one another in any way, unless expressly excluded or technically impossible. Likewise, features described primarily in connection with one aspect disclosed herein may also represent features of the other aspects disclosed herein. Furthermore, all aspects and features disclosed herein, either individually or in combination, are to be considered aspects of the present invention. Reference symbol list 1 vehicle 2 IR cameras 3 cameras for the visible area 4th person, passenger 5 first image data 6 camera image data 7 Matching information 8 Underwear 9 Database / Storage Device 10 Data processing device 11 Rearview mirrors 12 Windscreen 13 Dashboard 14 wireless interfaces 15 mirrors 16 Display unit / Display / CID 17 On-board computers 18 Outerwear 19 opaque area 20 patterned area 21 second image data 22 optical axis 23 Information 24 Device for image acquisition 30-34 process steps QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2008 037 815 A1 [0002, 0003]

Claims

[1] Method for capturing an image of the interior of a vehicle (1), comprising: Capturing the interior of the vehicle (1) using an IR camera (2) to generate initial image data (5) that represent a representation of the interior of the vehicle (1); Performing a comparison of the first image data (5) with: a) Camera image data (6) generated by optically capturing the interior of the vehicle (1) using a camera (3) for the visible spectrum and / or b) Information that characterizes a human body, human body parts, articles of clothing (8) or components of the vehicle (1) or is suitable to assist in the identification or recognition of a human body, human body parts, articles of clothing (8) or components of the vehicle (1), to obtain matching information (7) from the matching; wherein the method further comprises generating second image data (21) based on the first image data (5) and / or the camera image data (6) taking into account the matching information (7). [2] Method according to claim 1, further comprising: Using the matching information (7) to determine whether a modification should be applied to the first image data (5) and / or the camera image data (6). [3] Method according to claim 2, wherein determining whether a modification is to be applied to the first image data (5) and / or the camera image data (6) comprises: Determine whether the matching information (7) meets one or more given criteria, and if so: Using the matching information (7) to determine the modification to be applied to the first image data (5) and / or the camera image data (6), and Generating the second image data (21) by applying the modification to the first image data (5) and / or the camera image data (6). [4] Method according to claim 3, wherein the given criterion or one or more of the given criteria relate to the extent to which the representation of the interior of the vehicle (1) represented by the first image data (5) differs from a representation of the interior of the vehicle (1) represented by the camera image data (6). [5] Method according to claim 4, wherein a difference between the representation of the interior of the vehicle (1) represented by the first image data (5) and the representation of the interior of the vehicle (1) represented by the camera image data (6) is determined by means of edge detection. [6] Method according to any one of claims 3 to 5, wherein the given criterion or one or more of the given criteria relate to whether the representation of the interior of the vehicle (1) represented by the first image data (5) includes one or more elements of the following categories: one or more of predefined body parts or body sections, in particular the genital area; or one or more of predefined articles of clothing, in particular underwear (8); or Components of the vehicle (1), in particular a safety belt. [7] Method according to any one of claims 3 to 6, wherein determining whether the matching information (7) meets one or more given criteria is carried out using Body Keypoints. [8] Method according to any one of claims 2 to 7, wherein the determined modification relates only to a partial area of ​​the representation of the interior of the vehicle (1) represented by the first image data (5) or the representation of the interior of the vehicle (1) represented by the camera image data (6). [9] Method according to claim 8, wherein the modification causes the portion of the representation of the interior of the vehicle (1) to appear on a display device (16) in at least one of the following ways: - opaque - with a predetermined pattern - replaced by inpainting with different image content - replaced by adjacent image content or an essentially continuous continuation of the same. [10] Method according to one of the preceding claims, wherein the generation of the second image data (21) is carried out in such a way that the second image data (21) represent a modified representation of the interior of the vehicle (1). [11] Method according to any of the preceding claims, wherein the generation of the first image data (5) and the camera image data (6) comprises capturing the interior of the vehicle (1) from the same position or substantially the same position and / or along the same optical axis (22) or substantially along the same optical axis (22). [12] computer program with a program code or Computer program product with program code stored on a medium readable by a computer, wherein the program code includes instructions which, when executed by a computer, cause the computer to execute the method according to one of the preceding claims. [13] Device (24) for capturing an image of the interior of a vehicle (1), comprising: an IR camera (2) for generating initial image data (5) representing a depiction of the interior of the vehicle (1); and a data processing device (10) for performing a comparison of the first image data (5) with: a) Camera image data (6) generated by optically capturing the interior of the vehicle (1) using a camera (3) for the visible spectrum and / or b) Information that characterizes a human body, human body parts, articles of clothing (8) or components of the vehicle or that is suitable to assist in the identification or recognition of a human body, human body parts, articles of clothing (8) or components of the vehicle, to obtain matching information (7) from the matching; wherein the data processing device (10) is configured to generate second image data (21) based on the first image data (5) and / or the camera image data (6) taking into account the matching information (7). [14] Vehicle (1) comprising a device (24) according to claim 13.

Images