Vehicle anomaly detection method and device and vehicle

Abstract

The invention provides a vehicle anomaly detection method and device and a vehicle, and is applied to the technical field of intelligent detection. The vehicle anomaly detection method comprises the steps that audio data collected by multiple audio collection devices are acquired, the multiple audio collection devices are arranged in different directions of a vehicle, and the audio data represent target audio generated by at least one part of the vehicle; based on the detection model, performing anomaly detection on each target audio in the audio data to obtain a detection result; wherein the detection result comprises abnormal information of the component corresponding to the at least one target audio.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent detection technology, and in particular to a method and device for detecting abnormality of a vehicle, and a vehicle. Background Art

[0002] In the daily use of vehicles, it is inevitable that various mechanical and electronic components will fail and produce abnormal noises. If a component fails and produces abnormal noises while the user is using the vehicle, the user cannot immediately know the vehicle failure, resulting in a lack of detection and processing of vehicle abnormalities. Summary of the invention

[0003] In view of this, the present disclosure provides a method and device for detecting abnormality of a vehicle, and a vehicle.

[0004] According to a first aspect of the present disclosure, a method for detecting abnormalities in a vehicle is provided, comprising: acquiring audio data collected by multiple audio collection devices, wherein the multiple audio collection devices are arranged at different positions of a vehicle, and the audio data represents target audio generated by at least one component of the vehicle; based on a detection model, performing abnormality detection on each target audio in the audio data to obtain a detection result; wherein the detection result includes abnormal information of a component corresponding to at least one target audio.

[0005] According to an embodiment of the present disclosure, the abnormal information includes spatial information; based on the detection model, abnormality detection is performed on each target audio in the audio data to obtain a detection result, including: processing the audio data based on the first detection model to obtain a first detection result, wherein the first detection result characterizes the spatial information of the component corresponding to the target audio; wherein the first detection model is used to determine the first orientation information of the component that generates the target audio based on the audio data.

[0006] According to an embodiment of the present disclosure, the abnormal information includes attribute information; based on the detection model, an abnormality detection is performed on each target audio in the audio data to obtain a detection result, including: processing the audio data based on the second detection model to obtain a second detection result, the second detection result characterizing the attribute information of the component corresponding to the target audio; wherein the second detection model is used to determine the attribute information of the component that generates the target audio from an abnormality database based on the audio data, the abnormality database includes multiple data pairs, each data pair includes the attribute information of the component and the abnormal audio generated by the abnormality of the component.

[0007] According to an embodiment of the present disclosure, the abnormal information includes spatial information and attribute information; based on the detection model, an abnormality detection is performed on each target audio in the audio data to obtain a detection result, which also includes: processing the audio data based on the first detection model to obtain a first detection result, the first detection result characterizing the spatial information of the component corresponding to the target audio, wherein the first detection model is used to determine the first orientation information of the component that generates the target audio based on the audio data; processing the audio data based on the second detection model to obtain a second detection result, the second detection result characterizing the attribute information of the component corresponding to the target audio, wherein the second detection model is used to determine the attribute information of the component that generates the target audio from an abnormal database based on the audio data, the abnormal database including multiple data pairs, each data pair including the attribute information of the component and abnormal audio generated by the abnormal component; and determining the detection result based on the first detection result and the second detection result.

[0008] According to an embodiment of the present disclosure, audio data is processed based on a first detection model to obtain a first detection result, including: the first detection model determines, based on the audio data, associated information of an audio acquisition device that collects target audio, the associated information including direction information of the audio acquisition device and audio signal strength of the target audio collected by the audio acquisition device; the first detection model obtains the first detection result based on the associated information.

[0009] According to an embodiment of the present disclosure, the second detection model is obtained by using the following operations: acquiring multiple sample data; wherein the multiple sample data include multiple abnormal audios of a vehicle and a target component corresponding to each abnormal audio; inputting the multiple sample data into a neural network to obtain a predicted component for each abnormal audio; and adjusting the parameters of the neural network according to the difference between the predicted component and the target component until the neural network converges.

[0010] According to an embodiment of the present disclosure, obtaining audio data collected by multiple audio collection devices includes: obtaining target audio generated by at least one component collected by the multiple audio collection devices; obtaining first spatial position information of the multiple audio collection devices relative to a vehicle, wherein the first spatial position information represents coordinate information of the audio collection devices in a unit coordinate system of the vehicle; obtaining second spatial position information of at least one component relative to the vehicle, wherein the second spatial position information represents coordinate information of the component in a unit coordinate system of the vehicle; and encoding at least one target audio based on the first spatial position information and the second spatial position information to generate audio data.

[0011] According to an embodiment of the present disclosure, audio data is processed based on a first detection model to obtain a first detection result, including: based on the first detection model, performing anomaly detection on each target audio in the first audio data collected by multiple audio collection devices in the first part to obtain a third detection result, the third detection result characterizing second position information of a component generating the target audio, the accuracy of the first position information being better than that of the second position information; determining the second part based on the third detection result, the second part belonging to the first part; based on the first detection model, performing anomaly detection on each target audio in the second audio data collected by multiple audio collection devices in the second part to obtain the first detection result.

[0012] A second aspect of the present disclosure provides a vehicle abnormality detection device, comprising: an acquisition module, used to acquire audio data collected by multiple audio acquisition devices, wherein the multiple audio acquisition devices are arranged at different positions of the vehicle, and the audio data represents the target audio generated by at least one component of the vehicle; and a detection module, used to perform abnormality detection on each target audio in the audio data based on a detection model to obtain a detection result; wherein the detection result includes abnormal information of the component corresponding to the at least one target audio.

[0013] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned vehicle abnormality detection method.

[0014] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned vehicle abnormality detection method.

[0015] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned vehicle abnormality detection method when executed by a processor.

[0016] The fifth aspect of the present disclosure also provides a vehicle, comprising: a plurality of audio collection devices; and the above-mentioned vehicle abnormality detection device.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1A A flow chart of a method for detecting abnormality of a vehicle according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 1B The following schematically shows the layout of the audio acquisition device according to an embodiment of the present disclosure;

[0021] Figure 2 A flow chart of a method for detecting abnormality of a vehicle according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 3 A flow chart of a method for detecting abnormality of a vehicle according to another embodiment of the present disclosure is schematically shown;

[0023] Figure 4 A flowchart of a second detection model training method according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 5 A flow chart of a method for detecting abnormality of a vehicle according to another embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A flow chart of a method for detecting abnormality of a vehicle according to yet another embodiment of the present disclosure is schematically shown;

[0026] Figure 7 A schematic diagram of a structure of a vehicle abnormality detection device according to an embodiment of the present disclosure is shown; and

[0027] Figure 8 A block diagram of an electronic device suitable for implementing a method for detecting abnormality in a vehicle according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] The embodiments of the present disclosure provide a method, device and vehicle for detecting abnormality of a vehicle. Before introducing the technical solution provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.

[0033] In the related art, during the daily use of vehicles, it is inevitable that various mechanical and electronic components will fail and produce abnormal noises. If a component fails and produces abnormal noises while the user is using the vehicle, the user cannot immediately know the vehicle failure, resulting in a lack of detection and processing of vehicle abnormalities.

[0034] For example, cars have become a part of people's lives. With the development of science and technology, cars are becoming more and more intelligent and comfortable, becoming the third living space for human beings, bringing better experience to people's commuting or traveling.

[0035] The car itself is an electromechanical consumable. As time goes by, the aging and failure of various mechanical and electronic components are inevitable. In addition, if the car is in complex road conditions or traffic accidents (such as scratches, collisions, etc.), the driver cannot immediately know the situation of the vehicle. For example, when the engine, chassis, or gearbox suddenly makes strange noises while driving; when foreign objects (such as stones) collide with the car body; when the car briefly collides with other cars while driving (for example, the rearview mirror collides), etc., most people cannot judge in time what is wrong with the vehicle.

[0036] The usual solutions to abnormal noises in vehicles are: the first is to rely on manual processing. That is, start the vehicle, and an experienced engineer will roughly determine the approximate location of the abnormal noise, and then use the detection device to "listen carefully" nearby to determine the specific location of the abnormal noise. If it is a transient sound, check the body and components at the corresponding position to determine whether there is any abnormality. This processing method is more common in the C-end, that is, 4S stores and repair shops. This solution requires the support of professionals, such as vehicle design engineers or maintenance engineers, and the location and cause of the abnormal noise generated by the user of the vehicle during use cannot be known. The second is to set the location where the abnormal noise may occur from the perspective of vehicle engineering, create or collect abnormal noise audio, and assist the quality control of the vehicle offline in the vehicle factory, which can quickly locate the fault location and facilitate subsequent processing. This solution is a solution for specific situations, such as quality control of parts that may cause abnormal noise, location of the cause after the abnormal noise occurs, etc., and cannot cover a variety of situations of abnormal noise caused by vehicle failure.

[0037] The embodiment of the present disclosure provides a method for detecting abnormalities in a vehicle, including: obtaining audio data collected by multiple audio collection devices, wherein the multiple audio collection devices are arranged at different positions of the vehicle, and the audio data represents the target audio generated by at least one component of the vehicle; based on the detection model, performing abnormality detection on each target audio in the audio data to obtain a detection result; wherein the detection result includes abnormal information of the component corresponding to at least one target audio. By detecting the operation of the vehicle in real time, it is possible to provide alarms, suggestions, backtracking and other functions in a timely manner when an abnormality occurs, which greatly enhances the degree of intelligent use of the vehicle, and facilitates users to timely understand the body condition of the vehicle, and to perform abnormality detection and processing in a timely manner.

[0038] The following will be passed Figure 1A to Figure 6 The vehicle abnormality detection method of the disclosed embodiment is described in detail.

[0039] Figure 1A The flowchart of the vehicle abnormality detection method according to the embodiment of the present disclosure is schematically shown. Figure 1B The following schematically shows the layout of an audio acquisition device according to an embodiment of the present disclosure.

[0040] like Figure 1A As shown, the vehicle abnormality detection method of this embodiment includes operations S210 to S220.

[0041] In operation S210, audio data collected by a plurality of audio collecting devices is acquired, wherein the plurality of audio collecting devices are arranged at different positions of a vehicle, and the audio data represents a target audio generated by at least one component of the vehicle.

[0042] For example, the transportation tool may be a car, a motorcycle, a ship, an airplane, etc. The audio collection device may be a microphone, or an electronic device having a microphone, such as a recorder. Multiple audio collection devices are arranged at different positions of the delivery tool to collect audio generated by different parts of the delivery tool from multiple angles. Figure 1B , multiple microphones are placed at different locations in the car to form a microphone array. The microphones in the microphone array can be deployed in the engine compartment, trunk, bottom of the car, around the vehicle, and on the roof of the car.

[0043] It should be noted that the number of audio acquisition devices deployed depends on the type of designed tasks. For example, in order to detect the working status of a vehicle in real time, microphones are installed at the main mechanical parts of the vehicle, such as the engine box, water tank, exhaust pipe, wheels, etc. At the same time, the acoustic structure design needs to be considered when deploying the audio acquisition device, such as reducing the impact of unnecessary wind noise, etc., and avoiding exposing the audio acquisition device directly to the surface of the vehicle. The audio acquisition device can be installed near the mechanical parts of the vehicle, mainly collecting audio from the vibration of the vehicle body, mechanical parts, etc. In principle, the more audio acquisition devices there are, the higher the spatial resolution and the more accurate the anomaly detection.

[0044] The audio data may be the sound synthesized from the audio collected by multiple audio collection devices at the same time. For example, the audio data is obtained by synthesizing the audio collected by multiple microphones at the same time. When a component produces an abnormal sound, multiple microphones collect the sound of this abnormal sound and synthesize the audio data. When multiple components produce abnormal sounds at the same time, multiple microphones collect the sounds of multiple abnormal sounds and synthesize the audio data.

[0045] The target audio may be an abnormal sound generated during the operation of different components in a vehicle. It should be known that vibration sounds may be generated during the operation of different components in a vehicle. For example, when a car is in use, the engine (such as the engine) generates vibrations, and the microphone arranged at the engine will also record the sound of the engine's vibrations during the sound collection process. If an engine failure generates a buzzing sound, the sound collected by the microphone includes the sound of the engine's vibration and the buzzing sound, and the target audio is the buzzing sound generated by the abnormal engine.

[0046] Multiple audio acquisition devices record multiple channels of target audio and finally output a 3D audio stream (audio data).

[0047] It should be noted that audio collection devices at close distances may collect different volumes of the sound produced by a certain component at the same time. For example, if the first microphone placed at the engine and the second microphone placed at the water tank produce an abnormal buzzing sound, both the first microphone and the second microphone may collect the buzzing sound, but the volume of the buzzing sound collected by the first microphone may be higher than that collected by the second microphone.

[0048] In operation S220, anomaly detection is performed on each target audio in the audio data based on the detection model to obtain a detection result.

[0049] The detection result includes abnormal information of at least one component corresponding to the target audio.

[0050] Exemplarily, the detection model is used to detect and analyze the audio data to obtain the component that produces the abnormal sound. The abnormal information may include one or more abnormal data. In the case where the abnormal information includes multiple abnormal data, the types of the multiple abnormal data are different. The multiple abnormal data can facilitate the vehicle user to better locate the abnormality and know the cause of the abnormality.

[0051] For example, the abnormal information of a component may include: the location of the abnormal noise, for example, the detection model analyzes the audio data and finds that the location where the abnormal noise is generated is at the left front of the vehicle. The abnormal information of a component may also include the component that generates the abnormal noise, for example, the detection model analyzes the audio data and finds that the component that generates the abnormal noise is a wheel. The abnormal information of a component may also include the cause of the abnormal noise, for example, the detection model analyzes the audio data and finds that the cause of the abnormal noise is a deformation of the axle. The location, component and cause of the abnormal noise may also be included at the same time. For example, the detection model analyzes the audio data and finds that the cause of the abnormal noise is a deformation of the axle of the front left wheel.

[0052] It can be understood that by using the detection model to analyze the audio data generated during the operation of the vehicle in real time, it can provide timely alarm, suggestions, backtracking and other functions when an abnormality occurs, greatly enhancing the level of intelligent use of delivery tools, making it convenient for users to understand the body condition of the vehicle in time and perform abnormality detection and processing in time.

[0053] As described above, in the case where the abnormal information includes spatial information, operation S220 performs abnormal detection on each target audio in the audio data based on the detection model to obtain a detection result. In an achievable manner, the operation may further include operation S221a.

[0054] In operation S221a, the audio data is processed based on the first detection model to obtain a first detection result, wherein the first detection result represents spatial information of a component corresponding to the target audio.

[0055] The first detection model is used to determine first position information of a component that generates the target audio based on the audio data.

[0056] Exemplarily, the detection model includes a first detection model. The first detection model can be used to analyze audio data to obtain first position information of a component that generates an abnormal sound. For example, the sound (including abnormal sound) collected by multiple microphones of a vehicle is synthesized into audio and then input into the first detection model, and the first detection model analyzes the synthesized audio to obtain the tail of the vehicle where the abnormal sound is generated. The first detection model can also be used to analyze audio data to obtain second position information of the component that generates the abnormal sound. The positioning effect of the second position information is better than the positioning effect of the first position information. For example, the sound (including abnormal sound) collected by multiple microphones of a vehicle is synthesized into audio and then input into the first detection model, and the first detection model analyzes the synthesized audio to obtain the trunk of the vehicle where the abnormal sound is generated.

[0057] It can be understood that by analyzing the audio data through the first detection model to obtain the location where the abnormal sound is generated, it is convenient for the user or maintenance personnel of the vehicle to quickly locate the problem of the vehicle and determine the abnormal problem more quickly.

[0058] As described above, in the case where the abnormal information includes attribute information, operation S220 performs abnormal detection on each target audio in the audio data based on the detection model to obtain a detection result. In another achievable manner, the operation may further include operation S221b.

[0059] In operation S221b, the audio data is processed based on the second detection model to obtain a second detection result, where the second detection result represents attribute information of a component corresponding to the target audio.

[0060] Among them, the second detection model is used to determine the attribute information of the component that generates the target audio from the abnormal database based on the audio data. The abnormal database includes multiple data pairs, each data pair includes the attribute information of the component and the abnormal audio generated by the abnormal component.

[0061] Exemplarily, the detection model includes a second detection model. The second detection model can be used to analyze audio data to obtain the component that produces the abnormal sound. For example, the sound (including the abnormal sound) collected by multiple microphones of the vehicle is synthesized into audio and input into the second detection model. The second detection model analyzes the synthesized audio and obtains that the abnormal sound is generated by the engine (engine) of the vehicle. The second detection model can also be used to analyze audio data to obtain the component that produces the abnormal sound and the reason. For example, the sound (including the abnormal sound) collected by multiple microphones of the vehicle is synthesized into audio and input into the second detection model. The second detection model analyzes the synthesized audio and obtains that the abnormal sound is generated by the loose belt of the engine (engine) of the vehicle.

[0062] The second detection model has a database, which includes multiple data pairs, each of which includes a component and an audio of an abnormal sound generated by the component due to a specific reason. The target audio is matched with the abnormal audio of different data pairs in the database. If the match is consistent, the corresponding data pair is obtained, and the component corresponding to the abnormal audio and the specific reason for the abnormal audio generated by the component can be obtained.

[0063] It can be understood that by analyzing the audio data through the second detection model to obtain the component causing the abnormality, it is convenient for the user or maintenance personnel of the vehicle to quickly locate the problem of the vehicle and determine the abnormal problem more quickly.

[0064] Figure 2 A flow chart of a vehicle abnormality detection method according to yet another embodiment of the present disclosure is schematically shown.

[0065] As described above, in the case where the abnormal information includes spatial information and attribute information, operation S220 is to perform abnormal detection on each target audio in the audio data based on the detection model to obtain a detection result. In another possible implementation, Figure 2 As shown, the operation may further include operation S221c to operation S223c.

[0066] In operation S221c, the audio data is processed based on the first detection model to obtain a first detection result, which represents spatial information of a component corresponding to the target audio, wherein the first detection model is used to determine first position information of the component generating the target audio based on the audio data.

[0067] In operation S222c, the audio data is processed based on the second detection model to obtain a second detection result, which represents the attribute information of the component corresponding to the target audio, wherein the second detection model is used to determine the attribute information of the component that generates the target audio from an abnormality database based on the audio data, and the abnormality database includes multiple data pairs, each data pair includes the attribute information of the component and abnormal audio generated by the abnormality of the component.

[0068] In operation S223c, a detection result is determined according to the first detection result and the second detection result.

[0069] Exemplarily, the detection model includes a first detection model and a second detection model. The abnormal information obtained by the first detection model from analyzing the audio data can refer to the description of the above operation S221a; the abnormal information obtained by the second detection model from analyzing the audio data can refer to the description of the above operation S221b, which will not be repeated here.

[0070] It should be noted that the first detection model and the second detection model can simultaneously analyze the audio data to obtain the first detection result and the second detection result respectively, and determine the abnormal problem of the component according to the first detection result and the second detection result. It is also possible that the first detection model first analyzes the audio data to obtain the first detection result, and the second detection model then analyzes the audio data to obtain the second detection result, and determines the abnormal problem of the component according to the first detection result and the second detection result. It is also possible that the second detection model first analyzes the audio data to obtain the second detection result, and the first detection model then analyzes the audio data to obtain the first detection result, and determines the abnormal problem of the component according to the first detection result and the second detection result.

[0071] For example, the sounds (including abnormal sounds) collected by multiple microphones of a vehicle are synthesized into audio and then input into the first detection model. The first detection model analyzes the synthesized audio to obtain the location of the abnormal sound, which is above the rear of the vehicle. The second detection model analyzes the synthesized audio to obtain that the abnormal sound is caused by the rust on the hinges of the vehicle. Therefore, according to the detection results of the two detection models, it can be concluded that the abnormality is caused by the rust on the hinges of the vehicle trunk.

[0072] In the case of a vehicle with multiple identical parts, the first detection result obtained by the first detection model analysis and the second detection result obtained by the second detection model analysis can assist each other, making the obtained detection results more accurate. For example, the sounds (including abnormal sounds) collected by multiple microphones of the vehicle are synthesized into audio and then input into the first detection model. The first detection model analyzes the synthesized audio to obtain the left front of the vehicle where the abnormal sound is generated. The second detection model analyzes the synthesized audio to obtain that the abnormal sound is generated by the deformation of the axle of the vehicle. Therefore, according to the detection results of the two detection models, it can be obtained that the abnormality is caused by the deformation of the axle of the left front wheel of the vehicle.

[0073] It can be understood that the location of the abnormal sound can be obtained by analyzing the audio data through the first detection model, and the abnormal component or the cause of the abnormality can be obtained by analyzing the audio data through the second detection model, which makes it easier for the user or maintenance personnel of the vehicle to quickly locate the problem of the vehicle and determine the abnormal problem more quickly.

[0074] In other embodiments, the first detection result and the audio data may be simultaneously input into the second detection model to obtain the detection result.

[0075] Figure 3 A flow chart of a method for detecting abnormality of a vehicle according to another embodiment of the present disclosure is schematically shown.

[0076] As described above, in operation S221a, the audio data is processed based on the first detection model to obtain a first detection result. In one possible implementation, Figure 3 As shown, the operation may further include operation S2211a to operation S2212a.

[0077] In operation S2211a, the first detection model determines association information of an audio collection device that collects target audio according to the audio data, where the association information includes direction information of the audio collection device and audio signal strength of the target audio collected by the audio collection device.

[0078] In operation S2212a, the first detection model obtains a first detection result according to the associated information.

[0079] Exemplarily, the audio data (3D audio) is input into the first detection model, which analyzes the audio data to obtain the time when the same audio reaches each audio collection device and the audio signal strength. Finally, based on the time difference (time difference measurement) and sound intensity difference (intensity difference measurement) when the same sound reaches each audio collection device, the direction and distance of the collected sound are converted into the specific location of the sound source, and the location information of the component that generates the abnormal sound can be determined.

[0080] Figure 4 The flowchart of the second model training method according to an embodiment of the present disclosure is schematically shown.

[0081] As described above, for the second detection model, in one possible implementation, Figure 4 As shown, the second detection model can be trained through the following operations S310 to S330.

[0082] In operation S310 , a plurality of sample data are acquired; wherein the plurality of sample data include a plurality of abnormal audios of a vehicle and a target component corresponding to each abnormal audio.

[0083] In operation S320, a plurality of sample data are input into a neural network to obtain a prediction component of each abnormal audio.

[0084] In operation S330, parameters of the neural network are adjusted according to the difference between the predicted component and the target component until the neural network converges.

[0085] Exemplarily, the multiple sample data may be audio of abnormal noises generated during the operation of different mechanical parts of a vehicle. It may be audio of abnormal noises generated during the operation of a mechanical part, for example, the gurgling sound generated by abnormal operation of a water pump; the hissing sound generated by abnormal muffler; the squeaking sound generated by deformation of axle. It may also be audio of mixed abnormal noises generated during the operation of multiple mechanical parts. For example, an audio contains both the gurgling sound generated by abnormal operation of a water pump and the hissing sound generated by abnormal muffler. The sounds generated by abnormal mechanical parts of multiple vehicles of the same model can be collected as sample data. The abnormal audios in the multiple sample data are input into a neural network to obtain a predicted part. According to the difference between the predicted part and the target part (the part that actually generates the abnormal noise), the parameters of the neural network are continuously adjusted to find the possible minimum value of the difference, and the neural network converges, thereby obtaining a trained second detection model.

[0086] It should be known that convergence usually means that the model parameters are updated to a stable state, in which the model has a good fit to the training data and can generalize well to new, unseen data. Specifically, convergence means that the model's loss function (a function that measures the difference between the model's predictions and the actual values) has found a possible minimum or local minimum.

[0087] Using the trained second detection model, the target audio is input into the second detection model. The second detection model uses the database to match the target audio with the abnormal audio in the audio data, so as to obtain the attribute information of the abnormal component corresponding to the target audio.

[0088] Figure 5 A flow chart of a method for detecting abnormality of a vehicle according to another embodiment of the present disclosure is schematically shown.

[0089] As described above, in operation S210, audio data collected by multiple audio collection devices is obtained. In one possible implementation, Figure 5 As shown, the operation may further include operation S211 to operation S214.

[0090] In operation S211, target audio generated by at least one component collected by a plurality of audio collecting devices is acquired.

[0091] In operation S212, first spatial position information of the plurality of audio collection devices relative to the vehicle is obtained, wherein the first spatial position information represents coordinate information of the audio collection devices in a unit coordinate system of the vehicle.

[0092] In operation S213, second spatial position information of at least one component relative to the vehicle is acquired, wherein the second spatial position information represents coordinate information of the component in a unit coordinate system of the vehicle.

[0093] In operation S214, at least one target audio is encoded based on the first spatial orientation information and the second spatial orientation information to generate audio data.

[0094] Exemplarily, the audio data may be a three-dimensional sound field map formed by multi-channel audio recorded by an audio acquisition device. A 3D coordinate system is established with the exact center of the vehicle's three-dimensional space. The position information of each microphone in space (i.e., the first spatial orientation information) is determined and represented by the first coordinate information. The position of the sound source (such as a component) in three-dimensional space (i.e., the second spatial orientation information) is determined and represented by the second coordinate information. Multiple microphones collect sound at the same time. The collected sound is combined with the position information of the microphone and the position information of the component to generate a three-dimensional sound field map through calculation and processing. The multi-channel audio collected by the microphone is synthesized to obtain audio data, and each moment corresponds to an audio data and a three-dimensional sound field map.

[0095] Figure 6 The flowchart of a vehicle abnormality detection method according to yet another embodiment of the present disclosure is schematically shown.

[0096] As described above, in operation S221a, the audio data is processed based on the first detection model to obtain a first detection result. In another possible implementation, Figure 6 As shown, the operation may further include operation S2211b to operation S2213b.

[0097] In operation S2211b, based on the first detection model, anomaly detection is performed on each target audio in the first audio data collected by multiple audio collection devices in the first local area to obtain a third detection result, which characterizes the second position information of the component that generates the target audio, and the accuracy of the first position information is better than that of the second position information.

[0098] In operation S2212b, a second part is determined according to the third detection result, and the second part belongs to the first part.

[0099] In operation S2213b, based on the first detection model, anomaly detection is performed on each target audio in the second audio data collected by the multiple audio collection devices in the second part to obtain a first detection result.

[0100] Exemplarily, when the spatial information is obtained by analyzing the audio data using the first detection model, detection can be performed in multiple stages, thereby gradually determining the location of the abnormal sound component. For example, in the first audio analysis, the audio data collected by the microphones with a larger density (the first part) are first analyzed to obtain the approximate location information of the abnormal sound. Then, the microphone position required for the next analysis (the second part) is determined based on the approximate location information of the abnormal sound, and the audio data collected by the microphones in the second part are analyzed to obtain the specific location of the abnormal sound.

[0101] For example, a microphone array is arranged every 30 cm on the vehicle. During the abnormality detection process, if the axle of the left front wheel of the vehicle is deformed and produces an abnormal sound, the audio data collected by the microphones (first part) at an interval of 60 cm are first analyzed to obtain the position where the abnormal sound is roughly in the left front of the head of the vehicle (second position information). Then, the audio data collected by multiple microphones (second part, where the microphones can be arranged at an interval of 30 cm) arranged in the left front of the head of the vehicle are analyzed to obtain the abnormal sound generated at the axle position of the left front wheel of the vehicle.

[0102] It should be noted that the embodiments of the present disclosure do not impose any specific limitation on the number of times the local area is divided. The local area can be divided into multiple gradients according to the density of different audio acquisition devices, so as to achieve multi-layer detection of abnormal sound audio data.

[0103] It is understandable that firstly filtering the categories of audio data collected from different parts with a larger density and then performing specific location analysis based on the audio data collected with a smaller density can simplify the calculation of abnormal data and improve the efficiency of abnormality detection.

[0104] In some embodiments, the above-mentioned vehicle abnormality detection method further includes the operation of mapping the audio signals collected by multiple audio collection devices in a vehicle unit coordinate system and performing signal enhancement to obtain audio data.

[0105] For example, a stereo model is built for the microphone array. The position of the microphone and the propagation characteristics of the audio signal are mapped to a grid. Signal processing techniques (such as beamforming, filtering, and echo cancellation) are used to selectively enhance the audio signal in a specific direction or area in the grid space, thereby reducing background noise or other interference. Virtual sound field technology (such as Head Related Transfer Functions, HRTF, head-related transfer function) is used to reconstruct the spatial sense of sound and the audio data after sound enhancement. Anomaly detection analysis is performed on the enhanced audio data, which is more conducive to the detection model to quickly analyze and detect the audio data.

[0106] Based on the above-mentioned vehicle abnormality detection method, the present disclosure also provides a vehicle abnormality detection device. Figure 7 The device is described in detail.

[0107] Figure 7 The structure block diagram of the vehicle abnormality detection device according to an embodiment of the present disclosure is schematically shown.

[0108] like Figure 7 As shown, the vehicle abnormality detection device 400 of this embodiment includes an acquisition module 410 and a detection module 420 .

[0109] The acquisition module 410 is used to acquire audio data collected by multiple audio collection devices, wherein the multiple audio collection devices are arranged at different positions of the vehicle, and the audio data represents the target audio generated by at least one component of the vehicle. In one embodiment, the acquisition module 410 can be used to perform the operation S410 described above, which will not be repeated here.

[0110] The detection module 420 is used to perform anomaly detection on each target audio in the audio data based on the detection model to obtain a detection result. In one embodiment, the detection module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0111] The detection result includes abnormal information of at least one component corresponding to the target audio.

[0112] According to an embodiment of the present disclosure, any multiple modules in the acquisition module 410 and the detection module 420 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 410 and the detection module 420 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 410 and the detection module 420 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0113] In some embodiments, the detection module includes a first detection submodule. When the abnormal information includes spatial information, the first detection submodule is used to process the audio data based on the first detection model to obtain a first detection result, wherein the first detection result represents the spatial information of the component corresponding to the target audio; wherein the first detection model is used to determine the first orientation information of the component generating the target audio according to the audio data.

[0114] In some other embodiments, the detection module includes a second detection submodule. In the case where the abnormal information includes attribute information, the second detection submodule is used to process the audio data based on the second detection model to obtain a second detection result, and the second detection result represents the attribute information of the component corresponding to the target audio; wherein the second detection model is used to determine the attribute information of the component that generates the target audio from the abnormal database according to the audio data, and the abnormal database includes multiple data pairs, each data pair includes the attribute information of the component and the abnormal audio generated by the abnormal component.

[0115] In some further embodiments, the detection module includes a first detection submodule, a second detection submodule and a first determination submodule. The first detection submodule is used to process the audio data based on the first detection model to obtain a first detection result, the first detection result characterizing the spatial information of the component corresponding to the target audio, wherein the first detection model is used to determine the first orientation information of the component generating the target audio according to the audio data. The second detection submodule is used to process the audio data based on the second detection model to obtain a second detection result, the second detection result characterizing the attribute information of the component corresponding to the target audio, wherein the second detection model is used to determine the attribute information of the component generating the target audio from an abnormal database according to the audio data, the abnormal database including a plurality of data pairs, each data pair including the attribute information of the component and the abnormal audio generated by the abnormal component. The first determination submodule is used to determine the detection result according to the first detection result and the second detection result.

[0116] In some embodiments, the first detection submodule may be a first detection model, which determines, based on the audio data, associated information of an audio acquisition device that acquires the target audio, the associated information including direction information of the audio acquisition device and audio signal strength of the target audio acquired by the audio acquisition device; the first detection model obtains a first detection result based on the associated information.

[0117] In some embodiments, the second detection model is obtained using the following operations: acquiring multiple sample data; wherein the multiple sample data include multiple abnormal audios of a vehicle and a target component corresponding to each abnormal audio; inputting the multiple sample data into a neural network to obtain a predicted component for each abnormal audio; and adjusting the parameters of the neural network according to the difference between the predicted component and the target component until the neural network converges.

[0118] In some embodiments, the acquisition module includes a first acquisition submodule, a second acquisition submodule, a third acquisition submodule and a generation submodule. The first acquisition submodule is used to acquire the target audio generated by at least one component collected by multiple audio acquisition devices. The second acquisition submodule is used to acquire the first spatial orientation information of the multiple audio acquisition devices relative to the vehicle, wherein the first spatial orientation information represents the coordinate information of the audio acquisition device in the unit coordinate system of the vehicle. The third acquisition submodule is used to acquire the second spatial orientation information of at least one component relative to the vehicle, wherein the second spatial orientation information represents the coordinate information of the component in the unit coordinate system of the vehicle. The generation submodule is used to encode at least one target audio based on the first spatial orientation information and the second spatial orientation information to generate audio data.

[0119] In some embodiments, the first submodule includes a first detection subunit, a second determination subunit, and a second detection subunit. The first detection subunit is used to perform anomaly detection on each target audio in the first audio data collected by multiple audio collection devices in the first part based on the first detection model, and obtain a third detection result, the third detection result characterizes the second position information of the component that generates the target audio, and the accuracy of the first position information is better than the second position information. The second determination subunit is used to determine the second part according to the third detection result, and the second part belongs to the first part. The second detection subunit is used to perform anomaly detection on each target audio in the second audio data collected by multiple audio collection devices in the second part based on the first detection model, and obtain a first detection result.

[0120] Based on the above-mentioned vehicle abnormality detection device, the embodiment of the present disclosure also discloses a vehicle.

[0121] The vehicle of this embodiment includes a plurality of audio collection devices and the above-mentioned vehicle abnormality detection device. The vehicle abnormality detection device can be as follows: Figure 7 The vehicle abnormality detection device shown.

[0122] Figure 8 A block diagram of an electronic device suitable for implementing a method for detecting abnormality in a vehicle according to an embodiment of the present disclosure is schematically shown.

[0123] like Figure 8As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0124] In RAM503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM502 and RAM503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM502 and / or RAM503. It should be noted that the program can also be stored in one or more memories other than ROM502 and RAM503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0125] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0126] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0127] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM502 and / or RAM503 described above and / or one or more memories other than ROM502 and RAM503.

[0128] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the vehicle abnormality detection method provided by the embodiment of the present disclosure.

[0129] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0130] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0131] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0132] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0133] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0135] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

[0136] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0137] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for detecting abnormality of a vehicle, comprising: Acquire audio data collected by a plurality of audio collection devices, wherein the plurality of audio collection devices are arranged at different positions of a vehicle, and the audio data represents target audio generated by at least one component of the vehicle; Based on the detection model, perform anomaly detection on each target audio in the audio data to obtain a detection result; The detection result includes abnormal information of at least one component corresponding to the target audio.

2. The method according to claim 1, wherein: The abnormal information includes spatial information; The method of performing abnormality detection on each target audio in the audio data based on the detection model to obtain a detection result includes: Processing the audio data based on a first detection model to obtain a first detection result, wherein the first detection result represents spatial information of a component corresponding to the target audio; The first detection model is used to determine first position information of a component that generates the target audio based on the audio data.

3. The method according to claim 1, wherein: The abnormal information includes attribute information; The method of performing abnormality detection on each target audio in the audio data based on the detection model to obtain a detection result includes: Processing the audio data based on a second detection model to obtain a second detection result, wherein the second detection result represents attribute information of a component corresponding to the target audio; Among them, the second detection model is used to determine the attribute information of the component that generates the target audio from the abnormal database based on the audio data, and the abnormal database includes multiple data pairs, each of which includes the attribute information of the component and the abnormal audio generated by the abnormal component.

4. The method according to claim 1, wherein: The abnormal information includes spatial information and attribute information; The step of performing abnormality detection on each target audio in the audio data based on the detection model to obtain a detection result further includes: Processing the audio data based on a first detection model to obtain a first detection result, wherein the first detection result represents spatial information of a component corresponding to the target audio, wherein the first detection model is used to determine first position information of a component generating the target audio according to the audio data; Processing the audio data based on a second detection model to obtain a second detection result, wherein the second detection result represents attribute information of a component corresponding to the target audio, wherein the second detection model is used to determine attribute information of a component that generates the target audio from an abnormality database based on the audio data, wherein the abnormality database includes a plurality of data pairs, each of which includes attribute information of a component and abnormal audio generated by the abnormality of the component; and The detection result is determined according to the first detection result and the second detection result.

5. The method according to claim 2, wherein: The step of processing the audio data based on the first detection model to obtain a first detection result includes: The first detection model determines, according to the audio data, association information of an audio collection device that collects the target audio, wherein the association information includes direction information of the audio collection device and audio signal strength of the target audio collected by the audio collection device; The first detection model obtains the first detection result according to the associated information.

6. The method according to claim 3, wherein: The second detection model is obtained by the following operations: Acquire a plurality of sample data; wherein the plurality of sample data includes a plurality of abnormal audios of the vehicle and a target component corresponding to each of the abnormal audios; Inputting the plurality of sample data into a neural network to obtain a prediction component for each of the abnormal audios; and According to the difference between the predicted component and the target component, the parameters of the neural network are adjusted until the neural network converges.

7. The method according to claim 1, wherein: Acquiring audio data collected by multiple audio collection devices, including: Acquire the target audio generated by the at least one component collected by the multiple audio collection devices; Acquire first spatial position information of the multiple audio collection devices relative to the vehicle, wherein the first spatial position information represents coordinate information of the audio collection devices in a unit coordinate system of the vehicle; Acquire second spatial position information of the at least one component relative to the vehicle, wherein the second spatial position information represents coordinate information of the component in a unit coordinate system of the vehicle; and Based on the first spatial position information and the second spatial position information, at least one of the target audios is encoded to generate the audio data.

8. The method according to claim 2, wherein: Processing the audio data based on the first detection model to obtain a first detection result includes: Based on the first detection model, an abnormality detection is performed on each of the target audio in the first audio data collected by the plurality of audio collection devices in the first local area to obtain a third detection result, wherein the third detection result represents the second position information of the component generating the target audio, and the accuracy of the first position information is better than that of the second position information; determining a second part according to the third detection result, wherein the second part belongs to the first part; Based on the first detection model, anomaly detection is performed on each of the target audio in the second audio data collected by the multiple audio collection devices in the second local area to obtain the first detection result.

9. A vehicle abnormality detection device, comprising: an acquisition module, configured to acquire audio data collected by a plurality of audio acquisition devices, wherein the plurality of audio acquisition devices are arranged at different positions of a vehicle, and the audio data represents a target audio generated by at least one component of the vehicle; and A detection module, configured to perform anomaly detection on each target audio in the audio data based on a detection model to obtain a detection result; The detection result includes abnormal information of at least one component corresponding to the target audio.

10. A means of transport, comprising: Multiple audio acquisition devices; as well as The vehicle abnormality detection device according to claim 9.

Images