Vehicle abnormal noise positioning method and system based on digital twinning

Abstract

The invention provides a vehicle abnormal noise positioning method and system based on digital twinning, and the method comprises the steps: collecting a multi-channel sound signal through a three-dimensional spherical microphone array, according to the arrival time delay inequality of vehicle abnormal noise to different microphone arrays, combining the geometric structure of the three-dimensional spherical microphone array, and carrying out the positioning of the vehicle abnormal noise. Estimating the azimuth angle of the abnormal noise of the vehicle relative to the corresponding microphone array, determining the position of the abnormal noise of the vehicle, and determining the absolute coordinate of the abnormal noise of the vehicle in a vehicle coordinate system according to the conversion relation between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system. And displaying the abnormal noise position of the vehicle in the digital twin of the vehicle. According to the scheme of the invention, the generation part of the abnormal noise of the vehicle can be quickly and conveniently judged, and the accuracy of positioning the abnormal noise of the vehicle is improved based on the three-dimensional spherical microphone array and time delay estimation.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle abnormal noise localization, and particularly relates to a method and system for vehicle abnormal noise localization based on digital twin. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Noise and abnormal noise are very common in daily life and industrial production. For example, abnormal whistling sounds during vehicle driving. To solve these noise problems, it is first necessary to identify the noise and locate where the noise is coming from and what equipment or component causes it, which is the problem of sound source localization.

[0004] During the current vehicle development process, the identification of abnormal noise in the whole vehicle mostly relies on engineers' tests on rough roads or characteristic roads, and depends on engineers' experience to judge the location of abnormal noise by ear, which has high requirements for engineers' skills and is prone to misjudgment, resulting in low efficiency in problem discovery and solution; moreover, during vehicle driving, engine operation, tire-ground friction, wind noise, etc. will all generate noise, and these noises will interfere with sound source localization. Summary of the Invention

[0005] To overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for vehicle abnormal noise localization based on digital twin, which can quickly and conveniently judge the occurrence location of vehicle abnormal noise and improve the accuracy of vehicle abnormal noise localization.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In the first aspect, the present invention provides a method for vehicle abnormal noise localization based on digital twin, including:

[0008] Based on the operating state of the current vehicle, using the constructed vehicle noise model to predict the vehicle self-noise estimate value, based on the vehicle self-noise estimate value, obtaining a multi-channel sound signal collected by a three-dimensional spherical microphone array after removing the vehicle self-noise, and preprocessing the multi-channel sound signal;

[0009] According to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, estimating the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, and determining the vehicle abnormal noise position according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays;

[0010] According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, determine the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system, and then display the position of the vehicle abnormal noise in the vehicle digital twin.

[0011] Preferably, according to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays and combining with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, specifically:

[0012] Estimate the time delay difference between each microphone array through generalized cross-correlation or phase difference;

[0013] Based on the geometric structure of the three-dimensional spherical microphone array, convert the time delay difference between each microphone array into the distance difference of the vehicle abnormal noise to different microphone arrays, and use geometric principles to determine the azimuth angle of the vehicle abnormal noise relative to the microphone array.

[0014] Preferably, according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays, determine the position of the vehicle abnormal noise, specifically:

[0015] Based on the fact that the time delay of the vehicle abnormal noise to three microphone arrays is a constant, construct two hyperbolic equations;

[0016] Solve the constructed hyperbolic equations to obtain the position of the vehicle abnormal noise.

[0017] Preferably, according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, determine the absolute coordinates of the vehicle abnormal noise position in the vehicle coordinate system, specifically:

[0018] According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, determine the transformation matrix;

[0019] Multiply the transformation matrix by the coordinates of the vehicle abnormal noise in the three-dimensional spherical microphone array coordinate system to obtain the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system.

[0020] Preferably, the vehicle noise model includes an engine noise model, a wind noise model, and a tire noise model.

[0021] Preferably, preprocess the multi-channel sound signals, specifically:

[0022] Convert the collected sound signals into the modal domain;

[0023] In the modal domain, decouple the array manifold vector and remove the frequency-related components to obtain a frequency-independent manifold vector;

[0024] Calculate the array manifold vector in the modal domain to obtain a beam pattern with invariant frequency.

[0025] In a second aspect, the present invention provides a vehicle abnormal noise localization system based on digital twin, comprising:

[0026] An acquisition module, which is configured to: based on the running state of the current vehicle, predict the vehicle's own noise estimate value by using the constructed vehicle noise model, based on the vehicle's own noise estimate value, obtain the multi-channel sound signals collected by the three-dimensional spherical microphone array after removing the vehicle's own noise, and preprocess the multi-channel sound signals;

[0027] A localization module, which is configured to: according to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, and determine the position of the vehicle abnormal noise according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays;

[0028] A display module, which is configured to: determine the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, and then display the position of the vehicle abnormal noise in the vehicle digital twin.

[0029] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the method described in the first aspect is completed.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first aspect is completed.

[0031] In a fifth aspect, the present invention provides a vehicle to which the method described in the first aspect is applied.

[0032] The above one or more technical solutions have the following beneficial effects:

[0033] The present invention removes the influence of the vehicle's own noise on the abnormal noise localization through the vehicle noise model, and improves the accuracy of the abnormal noise localization.

[0034] The multi-channel sound signals with the vehicle's own noise removed collected by the three-dimensional spherical microphone array in the present invention estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array according to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, and in combination with the geometric structure of the three-dimensional spherical microphone array, and then determine the position of the vehicle abnormal noise. According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system are determined, and then the position of the vehicle abnormal noise is displayed in the vehicle digital twin. The solution of the present invention can quickly and conveniently judge the occurrence part of the vehicle abnormal noise, and improve the accuracy of vehicle abnormal noise positioning based on the three-dimensional spherical microphone array and time delay estimation.

[0035] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which form a part of this specification, are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0037] Figure 1 It is a flowchart of a method for locating vehicle abnormal noise based on digital twin in an embodiment of the present invention;

[0038] Figure 2 It is a schematic diagram of a spherical microphone array in an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram of a far-field sound field model in an embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of sound source localization in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0042] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0043] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0044] This embodiment discloses a method for locating vehicle abnormal noise based on digital twin, including:

[0045] Obtain multi-channel sound signals collected by a three-dimensional spherical microphone array, and preprocess the collected sound signals;

[0046] According to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, and determine the vehicle abnormal noise position according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays;

[0047] According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, determine the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system, and then display the vehicle abnormal noise position in the vehicle digital twin.

[0048] As an implementation method, according to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, specifically:

[0049] Estimate the time delay difference between each microphone array through generalized cross-correlation or phase difference;

[0050] Based on the geometric structure of the three-dimensional spherical microphone array, convert the time delay difference between each microphone array into the distance difference of the vehicle abnormal noise to different microphone arrays, and use geometric principles to determine the azimuth angle of the vehicle abnormal noise relative to the microphone array.

[0051] As an implementation method, determine the vehicle abnormal noise position according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays, specifically:

[0052] Based on the fact that the time delay of the vehicle abnormal noise to three microphone arrays is a constant, construct two hyperbolic equations;

[0053] Solve the constructed hyperbolic equations to obtain the vehicle abnormal noise position.

[0054] As an implementation method, determine the absolute coordinates of the vehicle abnormal noise position in the vehicle coordinate system according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, specifically:

[0055] Determine the transformation matrix according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system;

[0056] Multiply the transformation matrix by the coordinates of the vehicle abnormal noise in the three-dimensional spherical microphone array coordinate system to obtain the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system.

[0057] As an implementation manner, the position of the abnormal vehicle noise is displayed in the vehicle digital twin, specifically: in the vehicle digital twin, a visual marker is used to display the position of the abnormal vehicle noise, and the vehicle components at the abnormal vehicle noise are highlighted to achieve a preliminary determination of the associated parts where the abnormal vehicle noise occurs.

[0058] As an implementation manner, preprocess the collected sound signal, specifically:

[0059] Convert the collected sound signal to the modal domain;

[0060] In the modal domain, by decoupling the array manifold vector and removing the frequency-related components, a manifold vector independent of frequency is obtained;

[0061] Calculate the array manifold vector in the modal domain to obtain a beam pattern with invariant frequency.

[0062] In the present invention, multi-channel sound signals are collected by a three-dimensional spherical microphone array. According to the arrival time delay difference of the abnormal vehicle noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, the azimuth angle of the abnormal vehicle noise relative to the corresponding microphone array is estimated, and then the position of the abnormal vehicle noise is determined. According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, the absolute coordinates of the abnormal vehicle noise in the vehicle coordinate system are determined, and then the position of the abnormal vehicle noise is displayed in the vehicle digital twin. The solution of the present invention can quickly and conveniently judge the occurrence location of the abnormal vehicle noise, and based on the three-dimensional spherical microphone array and time delay estimation, improve the accuracy of abnormal vehicle noise positioning.

[0063] The following combines Figure 1 A method for positioning abnormal vehicle noise based on digital twin proposed in this embodiment is described in detail, specifically including:

[0064] Step 101: Establish a vehicle noise model to eliminate the influence of the vehicle's own noise on abnormal noise positioning.

[0065] The vehicle noise model includes an engine noise model, a wind noise model, and a tire noise model.

[0066] Engine noise model: Under different operating conditions, such as different engine speeds and load conditions, high-precision microphones are used to collect engine noise signals at different positions in the engine compartment and inside the vehicle. At the same time, the operating parameters of the engine, such as speed and throttle opening, are recorded. The collected noise signals are analyzed to extract characteristic parameters, such as spectral characteristics, time-domain characteristics (mean, variance, peak, etc.), cepstrum characteristics, etc. These characteristic parameters can reflect the relationship between the characteristics of engine noise and the operating state of the engine. Using machine learning algorithms, such as neural networks and support vector machines, an engine noise model is established. Taking the engine operating parameters as input and the noise characteristic parameters as output, the mapping relationship between the two is learned by training the model. For example, a deep neural network can be used, with engine speed, throttle opening, etc. as input layer nodes, and the extracted noise characteristic parameters as output layer nodes. Multiple hidden layers are set in the middle, and the weights and thresholds of the network are adjusted through a large amount of data training to construct an engine noise model.

[0067] Wind noise model: The vehicle is placed in a wind tunnel. At different wind speeds, microphones are used to collect wind noise signals at different positions outside the vehicle, such as the side of the body, the front of the vehicle, the roof, etc., and at corresponding positions inside the vehicle. At the same time, parameters such as the wind speed and wind direction of the wind tunnel are measured. The collected wind noise signals are preprocessed, such as filtering and denoising, and then their characteristics, such as the characteristics of the power spectral density and sound pressure level changing with the wind speed, are extracted. Based on the experimental data, a regression analysis method is used to establish a wind noise model. For example, the mathematical relationship between the wind noise characteristic parameters and the wind speed is found by fitting a curve.

[0068] Tire noise model: Under different road conditions, such as dry roads, wet roads, rough roads, etc., and different vehicle speeds, microphones are used to collect tire noise signals near the tires and inside the vehicle. At the same time, parameters such as the vehicle's driving speed and tire pressure are recorded. The tire noise signals are subjected to spectral analysis, time-frequency analysis, etc., and characteristics related to road conditions and vehicle speed are extracted, such as the energy distribution in different frequency bands and characteristic frequencies. Using multiple linear regression or other suitable algorithms, a tire noise model is established. Taking vehicle speed, road condition type, tire pressure, etc. as independent variables and the extracted tire noise characteristic parameters as dependent variables, a tire noise model is obtained through the analysis and fitting of a large amount of test data.

[0069] Step 102: Based on the current operating state of the vehicle, the vehicle self-noise estimation value is predicted using the constructed vehicle noise model. Based on the vehicle self-noise estimation value, a multi-channel sound signal with vehicle self-noise removed collected by the three-dimensional spherical microphone array is obtained, and the multi-channel sound signal is preprocessed.

[0070] Based on the operating state of the current vehicle, such as engine speed, vehicle speed, external environment, etc., use the established vehicle noise model to predict the estimated values of engine noise, wind noise, and tire noise at the current moment. For example, according to the current engine speed and load, calculate the estimated signal of engine noise through the engine noise model; according to the vehicle speed and road conditions, calculate the estimated signal of tire noise through the tire noise model; according to the wind speed and vehicle shape, calculate the estimated signal of wind noise through the wind noise model.

[0071] Subtract the estimated noise signal from the collected total in-vehicle noise signal to obtain the remaining signal after removing vehicle noise. Perform sound source localization on the remaining signal after noise subtraction, which can more accurately identify and locate abnormal sounds in the vehicle and reduce the interference of vehicle noise on the localization of abnormal sounds.

[0072] Among them, a 3D spherical microphone array sensor is adopted. Since the spherical array has a symmetric structure, it can capture three-dimensional sound field information and can be analyzed under the framework of spherical harmonic decomposition. Therefore, it is suitable for omnidirectional sound source localization.

[0073] Exemplarily, as Figure 2 shown, six microphones are evenly distributed on the surface of the sphere, and a potential position sensor is arranged at the center of the sphere to confirm the position coordinates of the microphone array in the whole vehicle in real time. Collect 6-channel sound data through the microphone array, and use the position sensor to collect position coordinate information. The sound signal acquisition frequency is defined as 48KHz.

[0074] According to the collected multi-channel sound signals, use the microphone array to perform modal signal processing. Because modal array signal processing converts the sound field signals collected by the microphones to the modal domain. In the modal domain, by decoupling the array manifold vector and removing the frequency-related components, a frequency-independent manifold vector can be obtained. After transformation, calculate the array manifold vector in the modal domain, and it can provide a frequency-invariant beam pattern, decoupling the dependence of the array control matrix on the direction and frequency of the source signal. This means that regardless of how the frequency of the sound signal changes, the beam pattern always remains stable. This frequency invariance can stably focus on and process the sound in a specific direction when sound signals of different frequencies coexist, without causing the beam direction or shape to change due to frequency changes.

[0075] Step 103: According to the arrival time delay difference of the vehicle abnormal noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array. According to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays, determine the position of the vehicle abnormal noise.

[0076] The microphone array is divided into various forms such as linear array, planar array, and three-dimensional array. The form and number of the microphone array are usually determined according to the use of sound source localization and signal characteristics. There are also many theories and methods for the sound source localization technology of the microphone array. The localization methods can generally be divided into three categories: the method based on time delay estimation, the controllable beamforming method based on maximum output power, and the high-resolution spectrum estimation method. The time delay estimation method mainly calculates the position of the sound source relative to the hyperboloid where a group of microphones are located according to the time difference of the sound source signal arriving at two microphones at different positions, and determines the sound source position through the intersection of multiple hyperboloids; the controllable beamforming method based on maximum output power mainly uses beamforming technology to continuously adjust the receiving direction of the array signal, and at the same time scans the entire receiving space, and the direction with the maximum received energy is the sound source direction; the high-resolution spectrum estimation method mainly uses the correlation matrix between the signals of each array element to obtain the direction angle of the sound source by calculating the spatial spectrum of the correlation matrix. Among them, the localization algorithm based on time delay estimation is simple and has high localization accuracy. This embodiment adopts the localization method based on time delay estimation.

[0077] The sound source localization adopts the localization method based on the estimation of Time Difference of Arrival (TDOA), which is a method to determine the position of the source point by using the time difference of the signal arriving at different receiving points from the source point. In the three-dimensional sound array signal localization, the TDOA technology calculates the position of the sound source according to the time difference of the sound signal arriving at multiple microphone arrays and the speed of sound.

[0078] As an implementation method, as Figure 3 shown, under far-field conditions, assuming that the sound wave propagates in the form of a plane wave, there is a difference in the distances that the abnormal noise s(k) of the vehicle travels to two microphones, resulting in different propagation times and forming a time delay difference. Let the speed of sound be c. As can be seen from Figure 2 , the path difference of the abnormal noise of the vehicle arriving at two microphones is dcos(θ). According to time = path ÷ speed, the calculation formula for the time delay difference τ is τ = cdcos(θ). By first measuring the time difference of the signals received by two microphones, or calculating τ through signal processing methods such as the generalized cross-correlation method.

[0079] According to the time delay difference τ, and the known distance d between two microphones and the speed of sound c, according to the time delay difference formula τ = cdcos(θ), by deforming it, we can get θ = arccos c*τ / d, so as to calculate the incident angle θ of the abnormal noise of the vehicle.

[0080] In the definition of a hyperbola, the absolute value of the difference in distances from a point in a plane to two fixed points, i.e., two microphones, is a constant, and this constant is the path difference cτ of sound propagation. Taking the midpoint of the line connecting the two microphones as the origin and the line where the connection lies as the x-axis, a plane rectangular coordinate system is established. Let the coordinates of the two microphones be and Let the position of the vehicle abnormal noise be P(x,y). According to the distance formula between two points, the distances from the vehicle abnormal noise P to the two microphones M1 and M2, namely |PM1| and |PM2|, are constructed. From the definition of a hyperbola: ||PM1| - |PM2|| = cτ, the hyperbola equation with the two microphones as foci is obtained.

[0081] As Figure 4 shown, the red dot is the noise source, the black dots are the microphones. The time delay from the noise source to two microphones such as microphone 1 and microphone 3 is a constant. Through this constant, the green hyperbola can be drawn. The time delay from the noise source to microphone 3 and microphone 2 is another constant. Similarly, the black curve can be drawn. The intersection of the two curves is the position of the vehicle abnormal noise.

[0082] As an implementation method, a rectangular coordinate system is established with the center of the spherical microphone array as the origin. Taking the example of 6 microphones evenly distributed on the spherical surface, their position coordinates are determined according to the symmetry of the sphere. The microphones can be placed at the 6 points (±r, 0, 0), (0, ±r, 0), and (0, 0, ±r) respectively, where r is the radius of the sphere.

[0083] The time for the vehicle abnormal noise to propagate to each microphone array is different. The time delay is calculated by measuring the time difference of the sound signal arriving at different microphone arrays. Specifically, the generalized cross-correlation method or the phase difference method can be used to calculate the time delay between microphone pairs.

[0084] For microphone 1 and microphone 2, the time delay difference τ 12 of the signals they receive is calculated. According to the geometric characteristics of the three-dimensional spherical microphone array and the speed of sound c, the time delay difference is converted into the distance difference Δd 12 = cτ 12 between the vehicle abnormal noise and different microphones. Since the positions of the microphones on the spherical array are known, using trigonometric relationships and geometric principles, the azimuth angle of the vehicle abnormal noise relative to the microphone array can be calculated.

[0085] Calculate the time delay τ 13 from the vehicle abnormal noise to microphone 1 and microphone 3, and the time delay τ32 from the vehicle abnormal noise to microphone 3 and microphone 2. Using the speed of sound and the time delay, the time delay difference can be converted into a distance difference. That is, the distance difference Δd 13 = c×τ 13, the distance difference Δd between the abnormal noise of the vehicle and the microphones 3 and 2 32 = c × τ 32 .

[0086] According to the definition of a hyperbola, the locus of points in a plane where the absolute value of the difference in distances to two fixed points, namely the two microphones, is a constant, i.e., the calculated distance difference, is a hyperbola.

[0087] Let the coordinates of the noise source be (x, y), the coordinates of microphone 1 be (x1, y1), the coordinates of microphone 3 be (x3, y3), and the coordinates of microphone 2 be (x2, y2). Distance formula:

[0088]

[0089] Based on the distance formula and the previously obtained distance difference Δd 13 , Δd 32 , the hyperbola equations can be listed respectively:

[0090] For microphones 1 and 3:

[0091]

[0092] For microphones 3 and 2:

[0093]

[0094] Solve the equations composed of the above two hyperbola equations through numerical calculation methods such as iterative algorithms, Newton - Raphson methods, etc. to determine the coordinate position of the final abnormal noise of the vehicle.

[0095] Step 103: Determine the absolute coordinates of the abnormal noise of the vehicle in the vehicle coordinate system according to the conversion relationship between the three - dimensional spherical microphone array coordinate system and the vehicle coordinate system, and then display the position of the abnormal noise of the vehicle in the vehicle digital twin.

[0096] Determine the three - dimensional spherical microphone array coordinate system and the vehicle coordinate system. According to the previous calibration, obtain the conversion matrix T from the three - dimensional spherical microphone array coordinate system to the vehicle coordinate system. Multiply the coordinates of the abnormal noise of the vehicle in the three - dimensional spherical microphone array coordinate system by the conversion matrix T to obtain the absolute coordinates of the abnormal noise of the vehicle in the vehicle coordinate system.

[0097] Transfer the center point of the three-dimensional spherical microphone array and the vehicle abnormal noise coordinate point to the PC, and import them into 3D software. The 3D software combines the lightweight vehicle geometry data to automatically generate a 3D highlight of the vehicle abnormal noise point. Combining with the pre-opened 3D lightweight data of the corresponding physical vehicle, the specific location of the vehicle abnormal noise in the whole vehicle can be directly confirmed, and the components near the digital sound source can be highlighted, so as to preliminarily judge the related components where the abnormal noise occurs.

[0098] The present invention provides a vehicle abnormal noise positioning system based on digital twin, including:

[0099] An acquisition module, which is configured to: acquire multi-channel sound signals collected by a three-dimensional spherical microphone array, and preprocess the collected sound signals;

[0100] A positioning module, which is configured to: according to the arrival time delay difference between the vehicle abnormal noise and different microphone arrays, and combining with the geometric structure of the three-dimensional spherical microphone array, estimate the azimuth angle of the vehicle abnormal noise relative to the corresponding microphone array, and determine the vehicle abnormal noise position according to the azimuth angles of the vehicle abnormal noise estimated by different microphone arrays;

[0101] A display module, which is configured to: determine the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, and then display the vehicle abnormal noise position in the vehicle digital twin.

[0102] In more embodiments, a structure of a vehicle provided by the embodiments of the present invention is also provided. For example, the vehicle can be used to execute the vehicle abnormal noise positioning method based on digital twin provided in each of the above embodiments. The vehicle includes:

[0103] The vehicle may include an RF (Radio Frequency) circuit, a memory including one or more computer-readable storage media, an input unit, a display unit, sensors, an audio circuit, a WiFi (Wireless Fidelity) module, a processor including one or more processing cores, and a power supply and other components. Those skilled in the art can understand that the above components do not constitute a limitation to the vehicle, and may include more or fewer components than shown in the figure, or combine certain components, or arrange different components. Among them:

[0104] The RF circuit can be used for receiving and transmitting information or signals during communication. Specifically, after receiving the downlink information from the base station, it is handed over to one or more processors for processing. Additionally, the data related to the uplink is sent to the base station. Generally, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit can also communicate with the network and other devices through wireless communication. This wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0105] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.); the data storage area can store the data created according to the use of the vehicle (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. Correspondingly, the memory can also include a memory controller to provide access to the memory for the processor and the input unit.

[0106] The input unit can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, the input unit may include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display screen or a touchpad, can collect touch operations of the user on or near it (such as operations of the user using any suitable object or accessory such as a finger or a stylus on or near the touch-sensitive surface), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor, and can receive and execute the commands sent by the processor. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch-sensitive surface. In addition to the touch-sensitive surface, the input unit may also include other input devices. Specifically, the other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, etc.

[0107] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the vehicle, and these graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit may include a display panel. Optionally, the display panel can be configured in forms such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode). Further, the touch-sensitive surface can cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event. Subsequently, the processor provides a corresponding visual output on the display panel according to the type of touch event. The touch-sensitive surface and the display panel are implemented as two independent components to achieve input and input functions, but in some embodiments, the touch-sensitive surface and the display panel can be integrated to achieve input and output functions.

[0108] The vehicle may also include at least one sensor, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor can turn off the display panel and / or the backlight when the vehicle moves close to the ear. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of the acceleration in each direction (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used in applications for identifying the posture of the vehicle (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. As for other sensors that the vehicle may also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be elaborated here.

[0109] The audio circuit, speaker, and microphone can provide an audio interface between the user and the vehicle. The audio circuit can transmit the electrical signal converted from the received audio data to the speaker, and the speaker converts it into a sound signal for output. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit and then converted into audio data. After the audio data is output to the processor for processing, it is sent via the RF circuit to, for example, another vehicle, or the audio data is output to the memory for further processing. The audio circuit may also include an earphone jack to provide communication between the external earphone and the vehicle.

[0110] WiFi belongs to short - range wireless transmission technology. The vehicle can help users send and receive emails, browse the web, and access streaming media through the WiFi module, which provides users with wireless broadband Internet access. Although the WiFi module is shown, it can be understood that it does not belong to the essential components of the vehicle and can be completely omitted within the scope of not changing the essence of the invention according to needs.

[0111] The processor is the control center of the vehicle. It connects various parts of the entire vehicle using various interfaces and lines. By running or executing the software programs and / or modules stored in the memory, and by calling the data stored in the memory, it executes various functions of the vehicle and processes data, thereby monitoring the vehicle as a whole. Optionally, the processor may include one or more processing cores. Preferably, the processor may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor.

[0112] The vehicle further includes a power source (such as a battery) for supplying power to various components. Preferably, the power source can be logically connected to the processor through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. The power source can also include any components such as one or more DC or AC power sources, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0113] Although not shown, the vehicle may further include a camera, a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit of the vehicle is a touch screen display, and the vehicle further includes a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include those for executing the method shown in the above embodiment.

[0114] An embodiment of the present invention also provides a computer-readable storage medium, which is applied to a terminal. At least one instruction, at least one segment of program, a code set or an instruction set is stored in the computer-readable storage medium, and the instruction, the program, the code set or the instruction set is loaded and executed by a processor to implement the operations performed by the vehicle in the vehicle abnormal noise localization method based on digital twin in the above embodiment.

[0115] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiment can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc.

[0116] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle abnormal noise positioning method based on digital twin, characterized in that: include: Based on the current running state of the vehicle, the constructed vehicle noise model is used to predict the estimated value of the vehicle's own noise, and based on the estimated value of the vehicle's own noise, a multi-channel sound signal collected by a three-dimensional spherical microphone array with the vehicle's own noise removed is obtained, and the multi-channel sound signal is preprocessed; According to the arrival delay difference of abnormal vehicle noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, the azimuth of the abnormal vehicle noise relative to the corresponding microphone array is estimated, and the position of the abnormal vehicle noise is determined according to the azimuth of the abnormal vehicle noise estimated by different microphone arrays; According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system are determined, and then the position of the vehicle abnormal noise is displayed in the vehicle digital twin.

2. A vehicle abnormal noise location method based on digital twins as claimed in claim 1, characterized in that: According to the arrival delay difference of abnormal vehicle noise to different microphone arrays, combined with the geometric structure of the three-dimensional spherical microphone array, the azimuth angle of abnormal vehicle noise relative to the corresponding microphone array is estimated, specifically: The time delay difference between each microphone array is estimated by generalized cross-correlation or phase difference; Based on the geometric structure of the three-dimensional spherical microphone array, the time delay difference between each microphone array is converted into the distance difference from the vehicle abnormal noise to different microphone arrays, and the azimuth angle of the vehicle abnormal noise relative to the microphone array is determined using the geometric principle.

3. A vehicle abnormal noise location method based on digital twins as claimed in claim 1, characterized in that: According to the azimuth of abnormal vehicle noise estimated by different microphone arrays, the position of abnormal vehicle noise is determined, specifically: Based on the time delay from abnormal noise of the vehicle to the three microphone arrays as a constant, two hyperbolic equations are constructed; The constructed hyperbolic equation is solved to obtain the abnormal noise position of the vehicle.

4. A vehicle abnormal noise location method based on digital twins as claimed in claim 1, characterized in that: According to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, the absolute coordinates of the abnormal noise position of the vehicle in the vehicle coordinate system are determined, specifically: Determine a conversion matrix according to a conversion relationship between a three-dimensional spherical microphone array coordinate system and a vehicle coordinate system; The conversion matrix is ​​multiplied by the coordinates of the abnormal vehicle noise in the three-dimensional spherical microphone array coordinate system to obtain the absolute coordinates of the abnormal vehicle noise in the vehicle coordinate system.

5. The method for locating abnormal vehicle noise based on digital twins according to claim 1, characterized in that: The vehicle noise model includes an engine noise model, a wind noise model and a tire noise model.

6. A vehicle abnormal noise location method based on digital twins as claimed in claim 1, characterized in that: Preprocess the multi-channel sound signal, specifically: Convert the acquired sound signal into the modal domain; In the modal domain, the frequency-independent manifold vector is obtained by decoupling the array manifold vector and removing the frequency-dependent components; The array manifold vectors are calculated in the modal domain to obtain a frequency-invariant beam pattern.

7. A vehicle abnormal noise positioning system based on digital twins, characterized in that: include: An acquisition module is configured to: based on the current running state of the vehicle, use the constructed vehicle noise model to predict and obtain an estimated value of the vehicle's own noise; based on the estimated value of the vehicle's own noise, obtain a multi-channel sound signal collected by a three-dimensional spherical microphone array and with the vehicle's own noise removed; and pre-process the multi-channel sound signal; A positioning module is configured to: estimate the azimuth of the abnormal vehicle noise relative to the corresponding microphone array according to the arrival time delay difference of the abnormal vehicle noise to different microphone arrays in combination with the geometric structure of the three-dimensional spherical microphone array, and determine the position of the abnormal vehicle noise according to the azimuth of the abnormal vehicle noise estimated by different microphone arrays; The display module is configured to determine the absolute coordinates of the vehicle abnormal noise in the vehicle coordinate system according to the conversion relationship between the three-dimensional spherical microphone array coordinate system and the vehicle coordinate system, and then display the position of the vehicle abnormal noise in the vehicle digital twin.

8. An electronic device, characterized in that: It includes a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, a vehicle abnormal noise positioning method based on digital twins as described in any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the vehicle abnormal noise positioning method based on digital twins as described in any one of claims 1 to 6.

10. A vehicle, characterized in that: A vehicle abnormal noise positioning method based on digital twin is applied as described in claims 1-6.

Images