Method for detecting the operating state of a vehicle component of a motor vehicle, computer program and / or computer-readable medium, data processing device and motor vehicle

Abstract

A method for detecting the operating state of a vehicle component of a motor vehicle, wherein the motor vehicle has a component installed in its passenger compartment with a first surface, wherein light rays striking the first surface are reflected towards a second surface arranged in the passenger compartment and can be imaged on it as a reflection pattern, the method comprising: receiving a sequence of temporally successive images, each image representing a reflection pattern imaged onto the second surface; determining, based on the sequence of images, at least one parameter of a vibration of the first surface; comparing the vibration parameter with a predefined threshold; and generating a maintenance message if the vibration parameter falls below and / or exceeds the threshold.

Description

[0001] The present disclosure relates to a method for detecting the operating state of a vehicle component of a motor vehicle. The disclosure also relates to a computer program and / or computer-readable medium, a data processing device, and a motor vehicle.

[0002] To ensure the safety and functionality of a motor vehicle, various technical methods for checking its components are known. In this context, for example, DE 10 2016 116 671 A1 discloses a method for vehicle diagnostics based on vehicle sounds and vibrations. Here, sensors for detecting sounds and vibrations are installed in the immediate vicinity of the respective vehicle component. However, integrating such a sensor system into the vehicle is complex and costly.

[0003] DE 103 30 946 A1 discloses a measuring device for recording the movement trace of observation points on objects that are subjected to a periodic oscillation, comprising at least one measuring mark for optically marking at least one observation point on the objects and an electronic observation camera, wherein the measuring device is designed such that, depending on the frequency of the periodic oscillation, the exposure time of the observation camera for recording at least one complete movement trace executed in one oscillation period or the exposure time and image repetition frequency of successive image recordings for recording different parts of a movement trace can be adjusted.

[0004] DE 10 2018 103 485 A1 relates to a method for monitoring the condition of a machine, wherein video data of at least one part of the machine are acquired by means of a video camera arranged on or in a housing in a fixed spatial relationship to the machine, the video data are evaluated frequency-selectively with regard to machine rotation frequency-related vibrations by means of a data evaluation unit arranged in the housing in order to record current vibration states of the machine in a frequency range up to a maximum of 150 Hz for the purpose of machine condition monitoring, the recorded current vibration states are compared with target vibration states and / or previously recorded vibration states with regard to at least one machine-specific characteristic value by means of an evaluation unit arranged in the housing in order to evaluate the current vibration states with regard to their permissibility, and a result of the evaluation is output.

[0005] DE 10 2022 121 614 A1 discloses a method for recognizing the configuration of a vehicle's interior. In this method, image data is acquired using at least one image acquisition device. The image data represents a view of the vehicle's interior, and the view includes at least one stationary part of the vehicle and at least one movable component of the vehicle's interior, each at least partially. The image data is then evaluated. The at least one stationary part of the vehicle is recognized in order to assign the view to a corresponding area of ​​the vehicle's interior. Furthermore, the configuration of the at least one movable component of the interior is recognized within the area of ​​the vehicle's interior assigned to the view.

[0006] DE 10 2022 105 681 A1 a method for determining the vibration of a ventilation system at a measuring point.

[0007] DE 10 2022 201 725 A1 relates to a method for manufacturing a vibration detection system with at least one acceleration sensor arranged in an oscillating area of ​​a device.

[0008] KR 10 2018 0 049 395 A discloses a method and a device for diagnosing a sound source by 3D micro-vibration measurement of an object based on an invisible infrared multi-point laser beam and an infrared camera image.

[0009] EP 3 237 836 B1 discloses a system and method for detecting surface vibrations.

[0010] US 2022 / 0224861A1 discloses a device for monitoring vehicle occupants.

[0011] US 2021 / 0341620A1 discloses a system and method for detecting vibrations of one or more objects in a scene, which includes at least one light source that projects light onto the scene in a structured light pattern.

[0012] US 2023 / 0029930A1 discloses systems and methods for imaging, measuring an object and characterizing a sample.

[0013] Against the background of this prior art, one objective of the present disclosure is to specify a method suitable for enriching the prior art and improving at least the aforementioned aspects of the prior art. In particular, the objective of the disclosure is to detect an operating state of a vehicle component using simple sensors and / or by using existing sensors.

[0014] The problem is solved by the features of the independent claims. The dependent claims contain further developments of the disclosure.

[0015] The problem is then solved according to one aspect of the disclosure by a method for detecting an operating state of a vehicle component of a motor vehicle, wherein the motor vehicle has a component installed in its passenger compartment with a first surface, wherein light rays striking the first surface can be reflected towards a second surface arranged in the passenger compartment and can be imaged on it as a reflection pattern, wherein the method comprises: - Receiving a sequence of temporally successive images, where each image represents a reflection pattern mapped onto the second surface, - Determine, based on the sequence of images, at least one parameter of a vibration of the first surface, - Comparing the vibration parameter with a predefined threshold value, - Generating a maintenance message if the vibration parameter falls below and / or exceeds the threshold value.

[0016] It was discovered that vibrations from components installed in the passenger compartment, occurring during vehicle operation, can be used to determine the operating condition of that component or of other vehicle components mechanically coupled to it. Vibrations can also be referred to as oscillations. Excessive vibration of a component, for example, can indicate faulty assembly. Furthermore, excessive vibration of a component can indicate a defect in another vehicle component mechanically interacting with it. For instance, excessive vibration of a display actively cooled by a fan could be due to improper display mounting. Alternatively, the vibration of the display could also be due to a defect in the fan.

[0017] Furthermore, it was discovered that light striking a reflective surface in the passenger compartment—such as a chrome-plated surface, a display surface, and / or a window—can be projected onto another surface as a reflection pattern. Vibration of the reflective surface, for example, due to the running engine or another operating component, is accompanied by a corresponding vibration of the reflection pattern. This reflection pattern can be recorded using a camera. The vibration parameters can be determined by analyzing the recorded image data, for example, using an image analysis algorithm. These parameters include, in particular, the frequency and / or amplitude of the vibration. By comparing the recorded vibration parameter with a predefined threshold value, a deviation of the vibration behavior from a target value or range can be identified.If a deviation is detected, this indicates a possible defect in the component or another vehicle component. In this case, generating a corresponding maintenance notification enables early repair.

[0018] Determining at least one parameter of the vibration of the first surface can be achieved through image analysis of a sequence of images, for example, using an image analysis algorithm. When the component vibrates, periodic changes in the reflection pattern can be detected and characterized in the successive images. From this, the frequency and / or amplitude of the component's vibration can be determined. Determining at least one parameter of the vibration of the first surface can be done using classical analysis methods, such as Fourier analysis. Alternatively or additionally, determining at least one parameter of the vibration of the first surface can be done using an artificial neural network, especially one that has been pre-trained with a large number of references.

[0019] For a specific vehicle, suitable threshold values ​​can be empirically determined based on its driving condition and stored, for example, in the vehicle's on-board computer. Driving conditions could include, for example, "stationary with engine running" and / or "driving at a constant speed." Since the nature of the road surface has a significant influence on vibration behavior, relevant information can be considered when determining and / or assigning a threshold value, for example, to the driving condition "driving at a constant speed." For instance, the route traveled can be determined using the navigation system, and the corresponding road surface characteristics can be retrieved from a database.

[0020] In particular, the generated maintenance message - output to the driver via a user interface module of the vehicle, for example as an optical, acoustic and / or haptic signal, - be stored in the on-board computer so that the maintenance message can be read out, for example, as part of an on-board diagnostic procedure, and / or - transmitted to a workshop via a telecommunications network.

[0021] For the purposes of this disclosure, light rays are understood to be electromagnetic rays, in particular in the form of visible light, infrared radiation, and / or ultraviolet radiation. The light rays incident on the first surface can be generated by a vehicle-side light source. Alternatively, light sources located outside the vehicle can be used (e.g., headlights, sunlight, streetlights, etc.).

[0022] The first surface is, in particular, a reflective surface, such as a chrome-plated surface, a glass surface, or another smooth surface. The second surface exhibits, in particular, a diffusely scattering reflection, similar to a projection screen. The second surface is formed, for example, by fabric-covered parts, such as a seat and / or the headliner. The sequence of temporally successive images can be, or may have been, recorded using a camera pointed at the second surface.

[0023] Optionally, the method includes recording a sequence of temporally successive images using a camera located in the passenger compartment and directed at the second surface. In this way, the sequence of images can be recorded as needed using an in-vehicle camera. In some vehicles, it is advantageous to use an existing interior camera for this purpose. Preferably, the recording of the image sequence is only carried out when a specific driving condition exists (e.g., stationary with the engine running).

[0024] Optionally, the method includes irradiating the first surface with light beams using a light source located in the passenger compartment. This creates a well-defined measurement setup with known distances and / or angles between the light source, reflecting surface, imaging surface, and, if applicable, a camera, which simplifies the determination of at least one vibration parameter. Preferably, in some applications, non-visible light, i.e., infrared and / or ultraviolet radiation, is used in conjunction with a suitable camera, as this allows the method to be carried out without disturbing the driver. Preferably, the irradiation of the first surface with light beams is only performed when a specific driving condition is present (e.g., stationary with the engine running).

[0025] Optionally, the first surface is irradiated with light beams using infrared radiation. This makes the irradiation imperceptible to the vehicle's occupant and therefore undisturbed. Furthermore, the resulting reflection pattern is independent of daylight. It was also recognized that vehicles may already be equipped with an infrared camera, for example, for driver monitoring, and that the sequence of images can therefore be captured by this infrared camera.

[0026] Optionally, the specific vibration parameter can be compared with a predefined threshold, taking into account the vehicle's driving conditions. In most cases, the component's vibration behavior will change depending on the driving conditions, so comparing the specific vibration parameter with the predefined threshold is more meaningful if both values ​​apply to the same driving conditions. Different thresholds can be predefined for different driving conditions. The driving conditions can be recorded during the image sequence or subsequently read out, for example, from the on-board computer.

[0027] This increases the probability that a deviation in the vibration parameter is due to a defect in a vehicle component.

[0028] Optionally, the procedure includes: receiving information about which other vehicle components, which can mechanically affect the component, were in operation while the image sequence was being recorded, with at least some of this information being processed and / or provided when generating the maintenance message. This information can be recorded during the image sequence recording or subsequently read out, for example, from the on-board computer. In this way, the maintenance message contains a list of potential sources of defect, which facilitates the identification of the defective vehicle component during repair work. The other vehicle components that can mechanically affect the component can be components that are located near the component, mechanically coupled to it, and / or interact with it.Such an assignment of components to other vehicle components can be empirically determined in advance, depending on the vehicle model, and stored, for example, in the on-board computer. For instance, in some vehicle models, a vibration of the dashboard can indicate a defect in the air conditioning compressor.

[0029] The above can be summarized in other words, and with reference to a specific form that is described as not limiting for the present disclosure, as follows: The purpose of this technology is to detect vibrations in the vehicle's interior using simple sensors or the existing camera. Interior vibrations can indicate defects or loose components. Detecting these allows for early repairs. Vibration detection works as follows: A light source inside the vehicle shines onto a reflective surface, such as a display, a window, or a chrome component. Alternatively, sunlight striking the reflective surface from outside can be used. This creates a reflection or pattern, for example, on a seat surface or the headliner, which is detected and recorded by a camera. The shift of the reflected pattern is tracked between two images, and the amplitude and frequency are measured. The amplitude and frequency are then stored along with the current driving state, e.g., the vehicle is stationary with the engine running.The amplitude and frequency are compared to a respective target value, which is determined empirically, for example, under the same driving conditions. A deviation too large from the target value, i.e., exceeding or falling below an assigned threshold, can indicate a defect in the component where the reflection occurs, such as an incorrect assembly condition. However, the deviation can also indicate a defect in another vehicle component that is mechanically coupled to the vibrating component. For example, the reflection occurs at a display that is actively cooled. The fan is defective and generates vibrations at the display. In another example, the reflection occurs at parts of the dashboard, where the associated defective air conditioning compressor generates strong vibrations at the dashboard.Excessive deviations in amplitude or frequency are stored and, together with the active auxiliary devices (fan or air conditioning compressor), sent to the manufacturer in the form of a maintenance message, who can then trigger a service message in the vehicle.

[0030] Optionally, the parameter characterizes the frequency and / or amplitude of the vibration. This allows for a comprehensive characterization of the vibration. The frequency can represent the speed of change in the reflection pattern. The amplitude can represent the magnitude of the change in the reflection pattern.

[0031] According to one aspect of the disclosure, a computer program and / or a computer-readable medium is provided. The computer program and / or the computer-readable medium includes instructions that, when executed by a data processing device, cause the device to perform the method according to the disclosure and / or steps thereof. Optionally, the computer program and / or the computer-readable medium includes instructions that, when executed by a data processing device, cause the device to perform the process steps described as advantageous or optional in order to achieve an associated technical effect.

[0032] According to one aspect of the disclosure, a data processing device is provided for a motor vehicle. The data processing device is configured to perform the procedure described above. Optionally, the data processing device is configured to perform a procedure step described as advantageous or optional and / or to implement a procedure feature in order to achieve an associated technical effect.

[0033] According to one aspect of the disclosure, a motor vehicle comprising the data processing device described above is provided. Optionally, the data processing device of the motor vehicle and / or the motor vehicle itself is configured to perform a process step described as advantageous or optional and / or to implement a process feature in order to achieve an associated technical effect.

[0034] One embodiment of each is described below with reference to the figures. Fig. Figure 1 schematically shows a motor vehicle according to one aspect of the revelation; Fig. Figure 2 schematically shows a flowchart of a procedure according to one aspect of the revelation; and Fig. Figure 3 shows a schematic representation of a computer program and / or computer-readable medium according to one aspect of the revelation.

[0035] Fig. Figure 1 schematically shows a motor vehicle 50 according to one aspect of the revelation.

[0036] Motor vehicle 50 is a passenger car. Motor vehicle 50 is equipped to carry out steps related to Fig. The procedure 100 described in Section 2 is to be carried out. For this purpose, the motor vehicle 50 has a data processing device 51 and a communication interface 52 connected to it. The data processing device 51 can be the vehicle's on-board computer. The motor vehicle 50 has a passenger compartment 60 in which a component 61 with a first surface 62 is installed. Light rays 71 striking the first surface 62 are reflected towards a second surface 63 located in the passenger compartment 60 and are projected there as a reflection pattern 72. In this case, the motor vehicle 50 has a light source 70 located in the passenger compartment 60 for illuminating the first surface with light rays 71. The second surface 63 is, for example, the headliner 66, a seat, or another suitable component located in the passenger compartment 60.A vibration of component 60, including its reflective first surface 62, is accompanied, in particular, by periodic changes in the reflection pattern 72 on the second surface 63. The motor vehicle 50 has a camera 64 directed at the second surface 63, which is configured to record a sequence of temporally successive images of the reflection pattern 72. The camera 64 is connected to the data processing device 51 via communication technology and can transmit the recorded image data to it. By analyzing the recorded image data, for example, using an image evaluation algorithm running on the data processing device 51, one or more parameters of the vibration of component 60, in particular the frequency and / or amplitude of the vibration, can be determined.By comparing the measured vibration parameter with a predefined threshold value, a deviation of the vibration behavior of component 61 from a target value or range can be detected. If a deviation is detected, this indicates a possible defect in component 61 or in another vehicle component 65 that mechanically interacts with component 61. In this case, the data processing device 51 generates a maintenance message and transmits it to the communication interface 52.

[0037] The communication interface 52 can be a user interface module configured for interaction with the driver. For example, the maintenance message can be output to the driver as a visual, acoustic, and / or haptic signal via the user interface module. Alternatively or additionally, the communication interface 52 can be configured to transmit the maintenance message to a workshop via a telecommunications network. Furthermore, the communication interface 52 can be configured, either alternatively or additionally, to be connected to an external testing device, allowing the maintenance message stored in the data processing device 51 to be read by the testing device.

[0038] Fig. Figure 2 schematically shows a flowchart of a procedure 100 according to one aspect of the disclosure. The procedure 100 according to Fig. 3 is a method 100 for detecting an operating state of a vehicle component 61, 65 of a motor vehicle 50, wherein the motor vehicle 50 has a component 61 installed in its passenger compartment 60 with a first surface 62, wherein light rays 71 striking the first surface 62 are reflected in the direction of a second surface 63 arranged in the passenger compartment 60 and can be imaged on it as a reflection pattern 72. Such a motor vehicle 50 is, for example, in Fig. 1 shown. Fig. 2 is referred to in the Fig. 1 described.

[0039] Procedure 100 according to Fig. 2 indicates: Receiving 130 a sequence of temporally successive images, each image representing a reflection pattern 72 projected onto the second surface 63; Determining 140 from the sequence of images at least one parameter of a vibration of the first surface 62; Comparing 150 the vibration parameter with a predefined threshold; and Generating 160 a maintenance message if the vibration parameter falls below and / or exceeds the threshold. Optionally, the parameter characterizes a frequency and / or amplitude of the vibration.

[0040] According to one aspect of the disclosure, the method involves: recording 120 of the sequence of temporally successive images using a camera 64 arranged in the passenger compartment 60 and directed towards the second surface 63.

[0041] According to one aspect of the disclosure, the method comprises: irradiating 110 of the first surface 62 with light rays 71 by means of a light source 70 arranged in the passenger compartment 60. The irradiation 110 of the first surface 62 with light rays 71 is optionally carried out by means of infrared radiation.

[0042] According to one aspect of the disclosure, the comparison 140 of the specified parameter of the vibration with a predefined threshold is carried out taking into account the driving condition of the motor vehicle 50.

[0043] According to one aspect of the disclosure, the method involves: receiving information about which other vehicle components 65 that can mechanically act on component 61 were in operation while the sequence of images was recorded, wherein at least some of this information is processed and / or provided when generating 160 the maintenance message, e.g. by being included in the maintenance message.

[0044] Fig. Figure 3 shows a schematic representation of a computer program and / or computer-readable medium 200 according to one aspect of the disclosure. The computer program and / or computer-readable medium 200 comprises instructions 201 which, when the program or instructions are executed by a data processing device 51, cause the device to execute the method 100 and / or the steps of the method 100 according to Fig. 3 to be carried out.

[0045] The commands 201 can be in the form of program code in any code or language, in particular code suitable for controlling and / or monitoring motor vehicles 50. The computer program and / or computer-readable medium 200 can be or comprise any digital data storage device, such as a USB flash drive, hard drive, CD-ROM, SD card, or SSD card. The computer program need not necessarily be stored on such a computer-readable storage medium, but can also be accessed via the Internet or otherwise. Reference symbol (part of the description) 50 motor vehicles 51 Data processing device 52 Communication interface 60 passenger compartment 61 Component 62 first surface of the component 61 63 second surface 64 camera 65 additional vehicle components 66 Headliner 70 light source 71 light rays 72 reflection patterns 100 procedures 110 Irradiate 120 Record 130 Received 150 Determine 150 comparisons 160 Produce 200 computer program and / or computer-readable medium 201 commands

Claims

[1] Method (100) for detecting an operating state of a vehicle component (61, 65) of a motor vehicle (50), wherein the motor vehicle (50) has a component (61) installed in its passenger compartment (60) with a first surface (62), wherein light rays (71) striking the first surface (62) can be reflected in the direction of a second surface (63) arranged in the passenger compartment (60) and can be imaged on it as a reflection pattern (72), wherein the method (100) comprises: - Receiving (130) a sequence of temporally successive images, wherein each image represents a reflection pattern (72) mapped onto the second surface (63); - Determine (140) from the sequence of images at least one parameter of a vibration of the first surface (62); - Comparing (150) the vibration parameter with a predefined threshold value; and - Generating (160) a maintenance message if the vibration parameter falls below and / or exceeds the threshold value. [2] Method (100) according to claim 1, wherein the method (100) comprises: - Recording (120) the sequence of temporally successive images using a camera (64) arranged in the passenger compartment (60) and directed at the second surface (63). [3] Method (100) according to claim 1 or 2, wherein the method comprises: - Irradiating (110) the first surface (62) with light rays (71) by means of a light source (70) arranged in the passenger compartment (60). [4] Method (100) according to claim 3, wherein the irradiation (110) of the first surface (62) with light rays (71) is carried out using infrared radiation. [5] Method (100) according to one of the preceding claims, wherein the comparison (140) of the determined parameter of the vibration with a predefined threshold value is carried out taking into account the driving condition of the motor vehicle (50). [6] Method (100) according to any of the preceding claims, wherein the method (100) comprises: - Receiving information about which other vehicle components (65) that can mechanically act on the component (61) were in operation while the sequence of images was being recorded, whereby at least some of this information is processed and / or provided when generating (160) the maintenance message. [7] Method (100) according to any of the preceding claims, wherein the parameter characterizes a frequency and / or amplitude of the vibration. [8] Computer-readable medium (200) comprising instructions (201) which, when executed by a data processing device (51), cause the device to perform the method (100) and / or the steps of the method (100) according to any one of claims 1 to 7. [9] Data processing device (51) for a motor vehicle (50), wherein the data processing device (51) is configured to perform the method (100) according to any one of claims 1 to 7. [10] Motor vehicle (50) comprising the data processing device (51) according to claim 9.

Images