Abnormal sound detection method and related product

Abstract

The invention discloses an abnormal sound detection method and related products, and relates to the technical field of intelligent vehicles. In the application, the acquisition control device obtains the first audio data reflecting the running sound of the parts in the vehicle and provides the detection data for the abnormal sound detection device, and the abnormal sound detection device obtains the abnormal sound detection result of the vehicle based on the detection data. An audio acquisition and abnormal sound identification separation architecture (such as terminal acquisition-cloud detection) is adopted, automatic acquisition, reporting and analysis of audio data are realized, the efficiency, consistency and accuracy of abnormal sound detection are improved, and the quality risk and detection cost of a vehicle are reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle technology, and in particular to a method for detecting abnormal noises and related products. Background Technology

[0002] With the development of intelligent and connected vehicles, the number of components in vehicles is increasing, and the reliability of core components directly affects vehicle driving safety and user experience. Among these, abnormal noises from components are an important signal reflecting potential faults, making accurate detection of such noises a crucial aspect of vehicle maintenance. Especially during vehicle use, abnormal noises from moving parts are a prominent issue, directly impacting the driving and riding experience.

[0003] The detection of abnormal noises mainly relies on human experience, which is inefficient and inconsistent. Therefore, designing an efficient and accurate abnormal noise detection solution has become an urgent need in the industry. Summary of the Invention

[0004] This application provides a method and related products for detecting abnormal noises, which can improve the efficiency, consistency and accuracy of abnormal noise detection, and reduce vehicle quality risks and testing costs.

[0005] Firstly, this application provides a method for detecting abnormal noises, capable of detecting faults in a vehicle (referred to as the first vehicle for ease of distinction), the first vehicle including a first component. This method for detecting abnormal noises can be executed by a device with data processing capabilities; for ease of description, a data acquisition and control device will be used as an example below.

[0006] The abnormal noise detection method includes: a data acquisition and control device acquiring first audio data from an audio acquisition device, the first audio data including the sound of the first component operating. The data acquisition and control device sending detection data to the abnormal noise detection device, the detection data including the first audio data, the first audio data being used by the abnormal noise detection device to detect abnormal noises during the operation of the first component.

[0007] The abnormal noise detection device identifies abnormal noises based on audio data provided by the acquisition and control device. It is functionally separate from the acquisition and control device, meaning they are functionally independent. For example, the abnormal noise detection device can be a cloud server. Alternatively, it can be an independent detection terminal device. Furthermore, as the computing power of vehicles increases, the abnormal noise detection device can be installed inside the vehicle. For ease of description, some embodiments below will use the abnormal noise detection device as a server as an example.

[0008] In the above embodiments, the acquisition and control device can automatically acquire the first audio data and automatically send the detection data to the abnormal noise detection device, which then performs abnormal noise detection based on the detection data. Since the audio data is automatically collected and reported, there is no need to specifically arrange for testing personnel to perform sound identification and evaluation during the acquisition phase, which can shorten the acquisition and testing time and improve testing efficiency and automation level. Secondly, entrusting the audio data to the abnormal noise detection device means that multiple vehicles can use a unified algorithm or model for abnormal noise analysis, eliminating interference from human factors, ensuring the uniformity of testing standards and the repeatability of results, and reducing the risk of problematic parts leaking out. Moreover, separating abnormal noise identification from audio feedback and using an independent abnormal noise detection device makes the detection algorithm or model highly scalable and easy to upgrade, not limited to specific vehicle models or specific parts. For example, it can be applied to the abnormal noise detection of various vehicle interior components such as chassis, interior, motor, and transmission. Furthermore, designing the detection function in an independent abnormal noise detection device can also reduce the stringent requirements for the professional experience of the operators performing the testing operations.

[0009] In summary, the above solution adopts an architecture that separates audio acquisition, acquisition control, and abnormal noise recognition, providing an automated, standardized, data-driven, and remotely executable abnormal noise detection process. This improves the efficiency, consistency, and accuracy of abnormal noise detection, while reducing quality risks and detection costs.

[0010] In some cases, the data acquisition and control device is a component in the first vehicle, such as a controller or computing device, or a module within the controller or computing device, such as a software or hardware module. In other cases, the data acquisition and control device is a terminal, such as a portable terminal or intelligent device like a mobile phone, tablet, or diagnostic instrument.

[0011] For example, the first component includes at least one of the following components: vehicle cabin door, window door, air conditioning system, front trunk door, rear trunk door, fuel tank cap, charging port cover, toolbox door, seat, screen, or display device, etc.

[0012] For example, the first component includes at least one of the following components: vehicle cabin door, window door, air conditioning system, front trunk door, rear trunk door, fuel tank cap, charging port cover, toolbox door, seat, screen, or display device, etc.

[0013] In one possible implementation of the first aspect, the acquisition control device further controls the first component to operate in a test mode. The first audio data includes the sound of the first component operating in test mode.

[0014] For example, the acquisition and control device acquires a first control command, and based on the first control command, controls a first component in the first vehicle to operate in a test mode.

[0015] Traditional manual reproduction of operating conditions is cumbersome and time-consuming, and inconsistencies in operation between different people can easily lead to data discrepancies. In the above embodiment, the acquisition and control device can actively control the components to operate in test mode, solving the problems of difficult operating condition reproduction and low data quality in traditional testing, saving time and improving efficiency. Furthermore, because the test mode allows for precise setting of parameters such as travel distance, rotational speed, and load, the acquired audio data has a unified benchmark, facilitating subsequent analysis.

[0016] In one possible implementation of the first aspect, specific information about the test mode is recorded by the acquisition and control device and the abnormal noise detection device, for example, stored in a database. The test mode information corresponds to the audio data, allowing for accurate backtracking of operating conditions during subsequent analysis, which facilitates the location of the root cause of the fault.

[0017] Optionally, the test mode can be customized according to different components, adapting to the testing of multiple components and improving the versatility of the solution.

[0018] In one possible implementation of the first aspect, the data acquisition control method of the first aspect is applied to a first vehicle. The data acquisition control device, based on a first control command, controls a first component in the first vehicle to operate in a test mode, including the following operation: the data acquisition control device, based on the first control command, sends a control signal to the first component, the control signal being used to control the first component to operate in the test mode.

[0019] In another possible implementation of the first aspect, the data acquisition and control method of the first aspect is applied to a terminal, which has a communication connection with the first vehicle. The data acquisition and control device, based on a first control command, controls a first component in the first vehicle to operate in a test mode, including the following operation: the data acquisition and control device, based on the first control command, sends a second control command to the first vehicle, the second control command instructing the first vehicle to control the first component to operate in a test mode.

[0020] In the above embodiment, the acquisition and control device can be set up independently of the first vehicle, that is, the detection initiator is a device outside the vehicle. The acquisition and control device communicates with the vehicle to control the components in the vehicle to operate under test conditions. This architecture further reduces the impact of abnormal noise detection on the vehicle's own hardware and software, eliminating the need to add hardware and software adapted for abnormal noise detection inside the vehicle, decoupling the various modules involved in abnormal noise testing, facilitating the upgrading and optimization of the test logic, and improving the system's flexibility.

[0021] In yet another possible implementation of the first aspect, the test mode is used to control the number of actions and the action parameters (such as type and / or stroke) of the actions. Exemplarily, the test mode is used to indicate the number of times the first component performs an action and the action parameters for each execution of the action.

[0022] In some cases, motion parameters are used to indicate the type of motion. In other cases, the motion of the first component has a travel range, and motion parameters can be used to indicate the travel range of the motion, such as the opening angle, the opening position, the opening ratio, the distance moved, etc.

[0023] In the above embodiments, the test mode represents the action type and / or stroke. Executing the test mode can enter the abnormal noise detection condition, improving the accuracy of fault location. Moreover, if the sound of component operation is collected in a general way, it is difficult to improve the sensitivity during analysis. However, by defining or recording the action type and action parameters during operation, the correlation between abnormal noise characteristics and action parameters can be established, providing comparability for abnormal noise detection and improving the consistency of abnormal noise detection.

[0024] Optionally, the first control command may be a command pre-set inside the acquisition and control device, or a command from the automated execution process (i.e., from within).

[0025] Alternatively, the first control instruction may be input by the user. Obtaining the first control instruction includes: obtaining the first control instruction input by the user.

[0026] Alternatively, the first control command may originate from an external source; in other words, the acquisition and control device may receive a first control command from other devices (such as an abnormal noise detection device). Acquiring the first control command includes: acquiring test information from the abnormal noise detection device, wherein the test information includes the first control command.

[0027] In another possible implementation of the first aspect, the acquisition control device acquires test information from the abnormal noise detection device, the test information including at least one of test mode information, a first control command, and indication information of a recommended acquisition position, the recommended acquisition position being used to indicate the placement position of the audio acquisition device.

[0028] For example, the abnormal noise detection device is a server, and the acquisition and control device obtains indication information of the test mode from the server. The server here is an example of a possible entity of the abnormal noise detection device, which can be replaced by other independent devices or computing modules in the vehicle (such as controllers, data centers, etc.).

[0029] In the above implementation, the specific content of the test mode is dynamically distributed and managed by the server. This allows the detection strategy to be centrally optimized and continuously iterated, and can be flexibly configured for different vehicles or different types of abnormal noises. This facilitates the automation and standardization of the detection process, and improves detection efficiency and consistency.

[0030] For example, if vehicle A and vehicle B are different models, the test modes used when testing the audio data of the same type of component may be different.

[0031] In another possible implementation of the first aspect, the audio acquisition device is fixedly installed in the first vehicle. In the above solution, the abnormal noise detection directly reuses the audio acquisition device (such as a vehicle-mounted microphone) already fixedly installed in the first vehicle to obtain the first audio data, achieving efficient utilization of hardware resources. Moreover, this setup avoids the hardware cost and installation complexity of adding a dedicated microphone for specific testing, significantly reducing the overall deployment cost of the solution.

[0032] Meanwhile, since the original vehicle microphones with fixed positions are used, the sound signals they collect are inherently consistent with the actual noise, vibration, and harshness (NVH) of the vehicle, ensuring the stability and comparability of the data source and providing a basis for building a unified standard for judging abnormal noises.

[0033] In another possible implementation of the first aspect, the audio acquisition device is an additional device. Exemplarily, the audio acquisition device has a wider frequency response range than the microphone array inside the vehicle, and further has a higher signal-to-noise ratio and better directivity. In the above implementation, using an additional professional audio acquisition device can more completely and clearly capture the subtle sound characteristics emitted by components during operation, effectively reducing the impact of ambient noise and electronic interference inside the vehicle. Audio acquisition using an additional device is suitable for scenarios with extremely high requirements for detection accuracy, such as research and development, in-depth diagnostics, or standards development. This implementation allows for more sensitive identification and more precise localization of abnormal noise faults.

[0034] In another possible implementation of the first aspect, the acquisition control device may prompt the audio acquisition device to be placed in a recommended acquisition location via an interactive device.

[0035] In the above embodiments, using an interactive device to provide operators with guidance on the placement of the audio acquisition device can avoid differences in acoustic data caused by arbitrary placement of the audio acquisition device, and reduce the adverse effects of human operation on data consistency. By standardizing the acquisition points, audio data obtained from different vehicles and different testing processes can be compared, laying a reliable foundation for accurate and consistent abnormal noise analysis by the subsequent abnormal noise detection device, while also reducing reliance on the operator's personal experience.

[0036] In another possible implementation of the first aspect, the acquisition control device acquires indication information of the first component input by the user and sends the indication information of the first component to the abnormal noise detection device. The aforementioned recommended acquisition location is related to the first component.

[0037] The above-described embodiment provides a design for obtaining recommended acquisition positions. The acquisition control device can acquire user specifications for specific components, provide the components to be executed to the abnormal noise detection device, and the abnormal noise detection device determines the recommended acquisition position corresponding to the component and feeds it back to the acquisition control device. Based on the indication information of the first component, the abnormal noise detection device can guide the acquisition control device to the optimal acquisition point, thereby enabling the operator to place the acquisition device in the position that best captures the abnormal noise characteristics of the component. The above-described embodiment can improve the success rate and data quality of audio data acquisition, and improve the accuracy and reliability of analysis results.

[0038] In another possible implementation of the first aspect, the acquisition control device acquires detection requirement information input by the user and, in response to the detection requirement information, controls the audio acquisition device to acquire audio. For example, in response to the detection requirement information, the acquisition control device sends an acquisition command to the audio acquisition device, the acquisition command instructing the audio acquisition device to acquire first audio data.

[0039] The above implementation provides a timing mechanism for triggering data acquisition. The acquisition control device obtains the detection requirement information actively input by the user and triggers audio acquisition accordingly, realizing on-demand initiation of the detection operation. This interactive mode ensures that the acquired audio data highly corresponds to the actual issues of concern to the user, improving the targeting and purposefulness of the detection.

[0040] In another possible implementation of the first aspect, the detection requirement information is voice information. The acquisition control device responds to the detection requirement information by controlling the audio acquisition device to acquire audio, including the following operations: the acquisition control device analyzes the voice information to obtain the user's intent; if the user's intent is to provide feedback on the use of the first vehicle, it sends an acquisition command to the audio acquisition device.

[0041] The above-described implementation provides a method for users to input detection requirements. The acquisition and control device can recognize and analyze the user's voice input to understand their intention to report vehicle usage problems and automatically trigger the audio acquisition process accordingly. Voice interaction is a natural and intuitive voice interaction process, significantly reducing the user's operational threshold and improving the convenience of starting the detection and the user experience.

[0042] Optionally, in the above situations, the usage problem includes abnormal noise or non-abnormal noise malfunctions. That is, the above implementation includes two situations: ① the user reports a problem with the vehicle but the user cannot identify the faulty component and / or abnormal noise; ② the user reports the presence of abnormal noise.

[0043] Optionally, if the user's intention is to report an abnormal noise in the first vehicle, the audio acquisition device is controlled to acquire audio. In this case, the user has already perceived the abnormal noise. Furthermore, if the user can perceive the component causing the abnormal noise or the location of the abnormal noise, the user's intention includes information about the component causing the abnormal noise and its location.

[0044] In another possible implementation of the first aspect, the acquisition control device also acquires information about the scene in which the abnormal noise occurs, and controls the audio acquisition device to acquire audio in the scene. For example, in the scene of the abnormal noise, an acquisition command is sent to the audio acquisition device.

[0045] The above implementation provides another timing for data acquisition triggering. The acquisition control device can pre-acquire specific scenario information (such as vehicle speed, steering angle, road conditions, etc.) when the abnormal noise occurs, and automatically trigger audio acquisition under this specific scenario. In this way, the triggering conditions for data acquisition are associated with the actual environment in which the abnormal noise occurs, ensuring that the captured audio data can highly reproduce the original working conditions of the fault, improving the success rate of capturing abnormal noise, and thus enhancing the accuracy and reliability of fault reproduction and analysis.

[0046] In another possible implementation of the first aspect, the abnormal noise generation scenario includes situations where the operating status data of the first vehicle meets the vehicle operating status conditions. The acquisition and control device also acquires the operating status data of the first vehicle and, based on the operating status data of the first vehicle, determines whether the operating status data of the first vehicle meets the vehicle operating status conditions.

[0047] In the above embodiments, the acquisition and control device can acquire and analyze the operating status data of the first vehicle in real time, such as vehicle speed, gear position, engine speed, and steering angle, and automatically determine whether it meets the preset vehicle operating status conditions related to the occurrence of abnormal noise, thereby triggering audio acquisition. This achieves automated and scenario-based capture of abnormal noises in a real driving environment. This audio acquisition method can effectively capture intermittent abnormal noises that only occur under specific dynamic conditions and are difficult to reproduce manually, significantly improving the success rate of abnormal noise audio capture and the effectiveness of the data.

[0048] In another possible implementation of the first aspect, the abnormal noise generation scenario further includes the execution action of the first component. The acquisition device controls the audio acquisition device to acquire audio in the abnormal noise generation scenario, including the following operations: when the operating status data of the first vehicle meets the vehicle operating status conditions, the acquisition control device sends an acquisition command to the audio acquisition device, and the first audio data includes the sound when the first component performs its action.

[0049] In some cases, the execution of the first component does not require additional instruction control.

[0050] In some cases, the execution of the first component requires additional command control. In this case, if the operating status data of the first vehicle meets the vehicle operating status conditions, the acquisition and control device controls the first component to execute its actions and sends an acquisition command to the audio acquisition device.

[0051] In the above embodiments, the acquisition and control device can monitor vehicle operating status data in real time, and when preset conditions are met, it actively controls the target component to perform specific actions and synchronously triggers audio acquisition, enabling abnormal noise acquisition under real-world comprehensive operating conditions. This design can effectively reproduce abnormal noises that only appear under complex dynamic conditions (such as operating a window at a specific vehicle speed), and the use of programmed control ensures a high degree of consistency and repeatability of test conditions, thereby significantly improving the success rate of abnormal noise capture and the quality and comparability of test results.

[0052] In another possible implementation of the first aspect, the acquisition control device acquires information about the abnormal noise generation scenario, including the following operations: acquiring abnormal noise description information input by the user, and determining information about the abnormal noise generation scenario based on the abnormal noise description information.

[0053] In the above implementation, the acquisition and control device can acquire and parse the user's natural language description of the abnormal noise phenomenon, intelligently deduce the corresponding abnormal noise generation scenario information, and realize the transformation of the user's subjective and emotional fault description into objective and actionable detection parameters. This improves the system's usability and accessibility, ensures that the audio acquisition matches the problem perceived by the user, and enhances the user experience.

[0054] In another possible implementation of the first aspect, the acquisition control device acquires the abnormal noise detection results from the abnormal noise detection device, the abnormal noise detection results including the abnormal noise status, the abnormal noise status being used to indicate whether there is an abnormal noise in the first vehicle.

[0055] In the above embodiments, the acquisition and control device can obtain detection results containing clear abnormal noise conditions, enabling users at the acquisition end to quickly obtain authoritative fault judgments. Furthermore, the abnormal noise detection results can provide a direct and reliable basis for subsequent maintenance decisions, thereby enhancing the practicality of the entire detection system and realizing a service closed loop.

[0056] In another possible implementation of the first aspect, when the abnormal noise status indicates that there is an abnormal noise in the first vehicle, the abnormal noise detection result also includes the cause of the abnormal noise, which includes indication information of the abnormal noise component.

[0057] In another possible implementation of the first aspect, when the abnormal noise status indicates that there is an abnormal noise in the first vehicle, the abnormal noise detection result also includes a handling suggestion, which is used to suggest a way to handle the abnormal noise.

[0058] In another possible implementation of the first aspect, the acquisition and control device outputs abnormal noise detection prompt information through an interactive device, and the abnormal noise detection prompt information is used to indicate the abnormal noise detection result.

[0059] In another possible implementation of the first aspect, the detection data further includes test case information, which describes the acquisition environment of the first audio data. The test case information includes at least one of the following: operating status data of the first vehicle, identification information of the first vehicle, batch of the first vehicle, acquisition time of the first audio data, or acquisition location of the first audio data.

[0060] Secondly, this application provides a method for detecting abnormal noise, which is applied to an abnormal noise detection device.

[0061] The abnormal noise detection method includes: an abnormal noise detection device acquiring detection data from a data acquisition and control device, the detection data including first audio data, the first audio data including the sound of a first component in the first vehicle during operation. Based on the detection data, the abnormal noise detection device obtains an abnormal noise detection result for the first vehicle, the abnormal noise detection result including an abnormal noise status, the abnormal noise status being used to indicate whether there is an abnormal noise in the first vehicle.

[0062] In the above scheme, the abnormal noise detection device acquires detection data including audio data and generates detection results containing the abnormal noise status based on the detection data. This allows complex audio signal analysis tasks to be designed within the abnormal noise detection device, reducing the computational resource consumption on the acquisition and control device while enabling the use of more advanced algorithms and models that require significant computational resources, resulting in more accurate and reliable analysis results. Furthermore, centralized abnormal noise detection by the acquisition and control device ensures consistency in the analysis algorithms and judgment criteria used for abnormal noise detection across different vehicles and in different detection processes, guaranteeing the accuracy, consistency, and reproducibility of the detection results. Simultaneously, this architecture separating audio acquisition and control from abnormal noise identification facilitates data accumulation, data comparison, and logic optimization, giving the entire system the potential for continuous evolution in diagnostic capabilities.

[0063] In another possible implementation of the second aspect, the abnormal noise detection device obtains the abnormal noise detection result of the first vehicle based on the detection data, including the following operations: the abnormal noise detection device obtains sound feature information based on the first audio data, and the abnormal noise detection device obtains the abnormal noise detection result of the first vehicle based on the sound feature information.

[0064] In the above embodiments, the abnormal noise detection device transforms audio data into structured information representing sound characteristics, reducing the computational complexity and data interference of subsequent analysis modules and improving detection efficiency. Furthermore, feature extraction enables the use of more precise pattern recognition algorithms, such as machine learning models, to enhance the accuracy and interpretability of abnormal noise status judgment.

[0065] In another possible implementation of the second aspect, the first audio data includes audio data from multiple channels. The abnormal noise detection device obtains an abnormal noise detection result for the first vehicle based on the detection data, including the following operations: the abnormal noise detection device obtains audio data from at least one channel based on the first audio data, wherein the distance between the acquisition location of the at least one channel's audio data and the sound source location satisfies a preset condition. The abnormal noise detection device obtains sound feature information based on the audio data from at least one channel. The abnormal noise detection device obtains the abnormal noise detection result for the first vehicle based on the sound feature information.

[0066] In some cases, at least one channel is the channel closest to the sound source or M channels that are relatively close to the sound source, where M is an integer and M≥2.

[0067] In a multi-channel environment, the signal-to-noise ratios of signals collected by microphones at different distances from the sound source vary significantly. In the above implementation, the abnormal noise detection device prioritizes channel data that meets the distance requirements, which can effectively capture purer and stronger abnormal noise signals, suppress interference from irrelevant sound sources, and enhance the accuracy and anti-interference capability of the abnormal noise detection results.

[0068] In another possible implementation of the second aspect, the sound feature information includes one or more of the following: frequency information, amplitude information, and time-varying information of frequency and amplitude.

[0069] In another possible implementation of the second aspect, the sound feature information includes time-varying information of frequency and amplitude, which includes a color map. The color map includes an image region defined by a first coordinate axis and a second coordinate axis, where the first coordinate axis represents the time of the sound and the second coordinate axis represents the frequency of the sound. One unit length of the first coordinate axis and one unit length of the second coordinate axis form a pixel in the image region, and the pixel value in each pixel corresponds to the amplitude of the sound.

[0070] In the above embodiments, the abnormal noise detection device can convert audio data into a standardized color map defined by a time axis, a frequency axis, and pixel values ​​(representing amplitude), realizing the data structuring and visual representation of sound signals. After audio feature extraction processing, the audio data is converted into image data, and auditory features that are difficult to quantify precisely are converted into visual features that can be efficiently processed by machine vision algorithms.

[0071] Thus, when identifying abnormal noises, the noise detection device can apply image recognition algorithms or artificial intelligence (AI) models (such as convolutional neural networks and multimodal large language models) to identify abnormal noises, thereby significantly improving the accuracy and intelligence level of detection. Furthermore, the color mapping map enables effective data compression and standardization, allowing audio data to be compared and analyzed in a unified image space. This significantly enhances the comparability of feature information and the consistency of the analysis process, thereby improving the accuracy of abnormal noise identification.

[0072] In another possible implementation of the second aspect, the abnormal noise detection device obtains the abnormal noise detection result of the first vehicle based on sound feature information, including: the abnormal noise detection device obtains the abnormal noise detection result of the vehicle based on sound feature information and feature information of multiple abnormal noise audio, wherein the feature information of each audio is used to indicate the feature of at least one abnormal noise.

[0073] In the above embodiments, the abnormal noise detection device compares and analyzes the sound feature information of the vehicle to be tested with an information database containing a variety of known abnormal noise features to obtain abnormal noise detection results. This improves the discrimination ability, accuracy and reliability of the detection system, enabling it to more accurately identify known types of abnormal noises. It also provides the possibility of distinguishing different types of abnormal noises and assessing the severity of abnormal noises.

[0074] In another possible implementation of the second aspect, the abnormal noise detection device obtains the abnormal noise detection result of the vehicle based on sound feature information, including: the abnormal noise detection device inputs the sound feature information into an AI model, the output of the AI ​​model includes the abnormal noise detection result, and the AI ​​model is pre-trained.

[0075] In the above embodiments, the abnormal noise detection device can call upon an AI model to automate and intelligently determine abnormal noises. Utilizing the nonlinear mapping and feature learning capabilities of the AI ​​model, it can calculate deep-seated correlations between sound characteristics and abnormal noise states—corresponding patterns that are difficult for the human ear or traditional algorithms to define. This effectively addresses various complex abnormal noise patterns and significantly improves the accuracy and efficiency of the detection system.

[0076] In another possible implementation of the second aspect, the detection data further includes the operating status data of the first vehicle. The abnormal noise detection device obtains the abnormal noise detection result of the first vehicle based on the detection data, including: the abnormal noise detection device obtains the abnormal noise detection result of the first vehicle based on the first audio data and the operating status data of the first vehicle. Wherein, the operating status data of the first vehicle is the vehicle operating data within the sampling period corresponding to the first audio data.

[0077] In the above embodiment, the abnormal noise detection device performs collaborative analysis of the first audio data and the concurrent operating status data of the first vehicle, associating the sound signal with the specific vehicle operating condition at the time the sound was generated, thus providing contextual information for abnormal noise identification. When identifying abnormal noises, combining the vehicle's operating status data can eliminate noises during normal vehicle operation, distinguish between sounds generated by vehicle operation and genuine abnormal noises, reduce the false positive rate, and improve the accuracy of fault identification in complex scenarios.

[0078] In another possible implementation of the second aspect, the first audio data includes the sound of the first component operating in a test mode, the test mode indicating the number of times the first component performs an action and the action parameters for each action.

[0079] In another possible implementation of the second aspect, the abnormal noise detection device also sends test information to the acquisition control device, the test information including one or more of the following: test mode indication information, recommended acquisition location, or first control command.

[0080] In another possible implementation of the second aspect, the abnormal noise detection device acquires indication information from a first component of the acquisition control device and obtains a recommended acquisition location based on the indication information from the first component.

[0081] In another possible implementation of the second aspect, when the abnormal noise status indicates that the first vehicle has an abnormal noise, the abnormal noise detection result also includes the cause of the abnormal noise, which includes indication information of the abnormal noise component.

[0082] In another possible implementation of the second aspect, when the abnormal noise status indicates that there is an abnormal noise in the first vehicle, the abnormal noise detection result also includes a handling suggestion, which is used to suggest a way to handle the abnormal noise.

[0083] In another possible implementation of the second aspect, the detection data further includes information about test cases used to describe the acquisition environment of the first audio data. The abnormal noise detection results also include at least a portion of the test case information.

[0084] In another possible implementation of the second aspect, the abnormal noise detection device obtains a batch inspection report based on the abnormal noise detection results of the first vehicle and at least one second vehicle. Both the first vehicle and the at least one vehicle belong to the first batch of vehicles, and the batch inspection report is used to indicate whether the abnormal noise detection of the first batch of vehicles is qualified.

[0085] In the above embodiments, the abnormal noise detection device analyzes the abnormal noise detection results of vehicles belonging to the same batch and generates a comprehensive inspection report for that batch. This can effectively identify potential batch-related quality defects and provide a reliable basis for subsequent quality improvement decisions.

[0086] In another possible implementation of the second aspect, the abnormal noise detection device sends the abnormal noise detection result of the first vehicle to the first vehicle and / or the first terminal. The first terminal is a terminal pre-configured for receiving the abnormal noise detection result of the first vehicle.

[0087] In the above embodiments, the abnormal noise detection results obtained by the abnormal noise detection device are automatically sent to the first vehicle and / or a pre-set receiving terminal, completing a complete closed loop from data collection, cloud analysis to result feedback, thereby improving the efficiency and transparency of the detection process.

[0088] Thirdly, this application provides a method for detecting abnormal noise, applied to an audio acquisition device. The method includes: the audio acquisition device receiving an acquisition command from an acquisition control device, and in response to the acquisition command, acquiring sound from the operation of a first component to obtain first audio data. The audio acquisition device then sends the first audio data, which is used by the abnormal noise detection device to detect abnormal noise during the operation of the first component. The first component is a component in a first vehicle.

[0089] In one possible implementation of the third aspect, the audio acquisition device sends first audio data to the acquisition control device, which provides the first audio data to the abnormal noise detection device.

[0090] Fourthly, this application provides a data acquisition and control device, which includes an acquisition unit and a transmission unit. The acquisition unit is used to acquire information, such as receiving or reading information, and the transmission unit is used to transmit data. The data acquisition and control device is used to implement the method described in the first aspect or any possible embodiment of the first aspect.

[0091] Optionally, the data acquisition and control device further includes a control unit, which outputs one or more of control signals, acquisition commands, etc., to enable the corresponding actuator to perform its function. Optionally, the data acquisition and control device further includes a processing unit, which processes information (e.g., processes information acquired by the acquisition unit).

[0092] Regarding the acquisition unit, data reporting unit, control unit (if any), and processing unit (if any) in the fourth aspect, the steps they perform can be referred to the corresponding embodiments and implementation methods of the first aspect. Regarding the technical effects of the fourth aspect and any possible implementation method, refer to the description of the technical effects corresponding to the first aspect and implementation methods, and the same applies below.

[0093] Fifthly, this application provides a data acquisition and control device, including a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to invoke the computer instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0094] Sixthly, this application provides an abnormal noise detection device, including an information acquisition module and a processing module. The information acquisition module is used to acquire data, such as receiving data or reading data, and the processing module is used to process the data. The acquisition control device is used to implement the method described in the second aspect or any possible embodiment of the second aspect.

[0095] In a seventh aspect, this application provides an abnormal noise detection device, including a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to invoke the computer instructions to implement the method described in the second aspect or any possible implementation of the second aspect.

[0096] Eighthly, this application provides an audio acquisition device, including a receiving unit, a reporting unit, and at least one microphone. The receiving module is used to receive instructions, the at least one microphone is used to acquire sound in response to the instructions, and the reporting unit is used to transmit data. The audio acquisition device is used to implement the method described in the third aspect or any possible implementation of the third aspect.

[0097] Ninthly, this application provides an audio acquisition device, including a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to invoke the computer instructions to implement the method described in the third aspect or any possible implementation of the third aspect.

[0098] In a tenth aspect, this application provides an abnormal noise detection system, which includes an audio acquisition device, an acquisition control device, and an abnormal noise detection device. The acquisition control device is used to implement the method described in the first aspect or any possible implementation of the first aspect, the abnormal noise detection device is used to implement the method described in the second aspect or any possible implementation of the second aspect, and the audio acquisition device is used to implement the method described in the third aspect or any possible implementation of the third aspect.

[0099] Eleventhly, this application provides a computer-readable storage medium for storing computer program instructions that, when executed by a processor, cause an apparatus including a processor to implement the method described in the first aspect or any possible implementation of the first aspect, or the method described in the second aspect or any possible implementation of the second aspect, or the method described in the second aspect or any possible implementation of the second aspect.

[0100] In a twelfth aspect, this application provides a computer program product including computer program instructions, which, when executed by a processor, cause a device including a processor to implement the method described in the first aspect or any possible implementation of the first aspect, or to implement the method described in the second aspect or any possible implementation of the second aspect.

[0101] In a thirteenth aspect, this application provides a vehicle that includes a data acquisition and control device of the fourth aspect, or a vehicle that includes a data acquisition and control device of the fourth aspect and an audio data acquisition device of the eighth aspect, or a vehicle that includes a data acquisition and control device of the fourth aspect and an abnormal noise detection device of the sixth aspect, or a vehicle that includes an abnormal noise detection system of the tenth aspect, or a vehicle that includes a computer-readable storage medium of the eleventh aspect, or a vehicle that includes a computer program product of the twelfth aspect. Attached Figure Description

[0102] Figure 1 This is a schematic diagram of the architecture of an abnormal noise detection system provided in an embodiment of this application; Figure 2 This is a schematic diagram of the architecture of an abnormal noise detection device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the architecture of another abnormal noise detection system provided in the embodiments of this application; Figure 4 This is a schematic flowchart of an abnormal noise detection method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating another abnormal noise detection method provided in the embodiments of this application; Figure 6 This is a flowchart illustrating another abnormal noise detection method provided in the embodiments of this application; Figure 7 These are schematic diagrams illustrating three scenarios of abnormal noise generation provided in the embodiments of this application; Figure 8 This is a schematic diagram of a scenario for collecting abnormal noises provided in an embodiment of this application; Figure 9 This is a schematic diagram of the architecture of another abnormal noise detection system provided in the embodiments of this application; Figure 10 This is a flowchart illustrating another abnormal noise detection method provided in the embodiments of this application; Figure 11 This is a schematic diagram of a user interface provided in an embodiment of this application; Figure 12 This is a schematic diagram of yet another user interface provided in an embodiment of this application; Figure 13 This is a schematic diagram of the architecture of another abnormal noise detection system provided in the embodiments of this application; Figure 14 This is a flowchart illustrating another abnormal noise detection method provided in the embodiments of this application; Figure 15 This is a schematic diagram of the architecture of another abnormal noise detection system provided in the embodiments of this application; Figure 16 This is a schematic diagram of the architecture of another abnormal noise detection system provided in the embodiments of this application; Figure 17 This is a flowchart illustrating another abnormal noise detection method provided in the embodiments of this application; Figure 18 This is a schematic diagram illustrating how the amplitude of a sound changes over time. Figure 19 It is a color map; Figure 20 This is a schematic diagram showing the layout and location of microphones inside a vehicle; Figure 21 This is a schematic diagram illustrating the training process of an AI model provided in an embodiment of this application; Figure 22 This is a schematic diagram of the structure of a data acquisition and control device provided in an embodiment of this application; Figure 23 This is a schematic diagram of another data acquisition and control device provided in the embodiments of this application; Figure 24 This is a schematic diagram of another abnormal noise detection device provided in the embodiments of this application. Detailed Implementation

[0103] As consumers increasingly demand quietness and comfort in automobiles, the automotive industry is facing new challenges and opportunities for technological innovation. This is especially true in the field of new energy vehicles, where overall operating noise is significantly lower than that of traditional gasoline vehicles. This has greatly increased users' sensitivity to in-car noise and vibration, making these issues more noticeable and increasing the risk of complaints about them.

[0104] Among the abnormal noise issues and complaints in the market, those from movable parts such as car curtains, sunroofs, seats, and tailgates are particularly prominent, becoming high-frequency problems in user complaints and directly affecting the driving experience and brand image. The industry mainly offers the following two solutions for abnormal noise detection: In one approach, testers reproduce the conditions under which the abnormal noise occurs and rely on traditional experience to determine if the noise is present. This method is inefficient and inconsistent. For example, in pre-shipment vehicle inspections, current noise detection primarily involves random sampling of vehicles by personnel. The inspection process also relies on individual experience, resulting in low efficiency and high costs. Furthermore, the inconsistency of subjective judgment makes it difficult to guarantee accurate results. Moreover, this method cannot comprehensively inspect all vehicle components. Similarly, in after-sales noise detection solutions, repair personnel rely on manual experience to determine the noise, which may lead to inaccurate results and the potential for disassembling or repairing the wrong parts, causing customer dissatisfaction.

[0105] Another approach involves installing a detection module within the vehicle to detect abnormal noises based on collected audio data. This method requires additional hardware and software functionality for detection within the vehicle. However, abnormal noise detection is a component of vehicle condition monitoring and fault diagnosis testing, and it's not frequently used during daily vehicle operation. Furthermore, vehicles have numerous components, and abnormal noise detection involves many aspects, leading to significant computational resource consumption for the detection module. Integrating detection functionality into the vehicle would result in prolonged occupation of its computing resources, leading to low resource utilization.

[0106] In summary, current abnormal noise detection solutions all suffer from inherent drawbacks such as low efficiency, high cost, complex deployment, poor consistency, and inefficient use of vehicle-side resources, resulting in a persistently high risk of defective parts leaking out. Developing an efficient, accurate, and intelligent abnormal noise detection solution has become an urgent need for the industry.

[0107] In view of this, this application provides a method and related products for detecting abnormal noises. The acquisition and control device can automatically acquire first audio data and automatically send detection data to the server. The server then performs abnormal noise detection based on the detection data. Employing an architecture that separates acquisition, acquisition control, and abnormal noise identification, it provides an automated, standardized, data-driven, and remotely executable abnormal noise detection process, improving the efficiency, consistency, and accuracy of abnormal noise detection while reducing quality risks and detection costs. Furthermore, the audio data is provided to the server, which analyzes the abnormal noises for detection. Leveraging the server's powerful computing capabilities, sophisticated detection algorithms can be designed within the server, or intelligent AI detection models can be invoked, further enhancing detection accuracy.

[0108] Furthermore, some embodiments can support multiple acquisition methods such as static acquisition, dynamic acquisition (such as acquisition in response to user needs, acquisition based on conditions, etc.), in-vehicle acquisition, and external acquisition, to meet the abnormal noise detection needs in various scenarios, such as factory testing, fault diagnosis, and user-initiated testing.

[0109] Furthermore, some embodiments also support intelligent functions such as sampling location recommendation and sampling location prompts, which can lower the threshold for use and improve detection accuracy.

[0110] To facilitate understanding, the system architecture and business scenarios of this application are first introduced below. It should be noted that the system architecture and business scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application. As system architectures evolve and new business scenarios emerge, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0111] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of an abnormal noise detection system provided in an embodiment of this application. The abnormal noise detection system 100 includes an abnormal noise detection device 101, a data acquisition and control device 102, and an audio acquisition device 103. Wherein: Audio acquisition device 103 is a device for acquiring audio, including one or more microphones, such as... Figure 1 Microphones 31 to 3n are shown, where n is a positive integer greater than or equal to 1. The audio acquisition device is capable of acquiring the sound of components operating in the vehicle. As the input end of the system, the audio acquisition device 103 is located near the vehicle or directly in the vehicle; in the latter case, the audio acquisition device 103 includes a microphone or microphone array fixedly installed in the vehicle.

[0112] In some embodiments, the vehicle is in motion when the audio acquisition device 103 acquires audio data from the vehicle; in other words, the audio acquisition device 103 can acquire the sound of components operating in the vehicle while the vehicle is in motion. In still other embodiments, the first vehicle is not in motion when the audio acquisition device 103 acquires audio data.

[0113] The acquisition control device 102 is a device with control and communication capabilities (e.g., having input / output interfaces), capable of acquiring audio data from the audio acquisition device 103 and reporting detection data (including audio data) to the abnormal noise detection device 101. Furthermore, the acquisition control device 102 can control the completion of the audio acquisition process. For example, the acquisition control device 102 can trigger the start of audio data acquisition. Also, the acquisition control device 102 can trigger the end of acquisition (or the acquisition process can automatically end). In some embodiments, the acquisition control device 102 can also perform one or more of the following functions: controlling the movement of components in the vehicle, indicating the placement position of the audio acquisition device, or detecting whether the conditions for triggering audio acquisition are met.

[0114] The abnormal noise detection device 101 is a device with data processing capabilities, which can process the detection data to obtain the detection results and identify whether there is an abnormal noise in the first vehicle. The detection data includes audio data, and optionally also includes vehicle operating status data.

[0115] For example, the abnormal noise detection device 101 is located on the server side, or in the cloud. The abnormal noise detection device can be deployed on a server or in a data center, or the abnormal noise detection device 101 includes multiple modules, which can be located on different servers or in different data centers. As another example, the abnormal noise detection device is an independent detection terminal device. As yet another example, with the improvement of computing power in end-side devices such as vehicles and terminals, the abnormal noise detection device can be installed in the vehicle or terminal. For ease of description, some embodiments below will be described using a server as an example of the abnormal noise detection device.

[0116] Optionally, the abnormal noise detection device 101 and the acquisition and control device 102 are functionally isolated from each other and are physically configured as different hardware modules. Alternatively, the abnormal noise detection device 101 and the acquisition and control device 102 can be located on the same hardware entity, but are software functionally isolated from each other, for example, they can communicate through an application programming interface.

[0117] In some cases, the abnormal noise detection device 101 is a central server, an edge server, or a local server in a local data center.

[0118] In other cases, the abnormal noise detection device 101 is located in the cloud with computing resources. The cloud includes various models or tools (such as feature extraction tools, AI models, etc.), and the abnormal noise detection device 101 can call upon the models or tools in the cloud to complete its functions. The computing resources in the cloud include physical computing resources or virtual computing resources.

[0119] In the abnormal noise detection system 100, the audio acquisition device 103 can communicate with the acquisition control device 102, including direct communication or indirect communication (e.g., requiring intermediate device forwarding). The abnormal noise detection device 101 can communicate with the acquisition control device 102, including direct communication or indirect communication.

[0120] Optionally, the abnormal noise detection system 100 also includes components in the vehicle, with the first component 302 being used as an example for explanation. The first component 302 is a component in the vehicle. For example, the first component 302 can be a movable part that can be electrically controlled, such as a door, seat, window, sunroof, armrest box, suspension, or projection screen. Alternatively, the first component can be a fixed part that cannot be electrically controlled, such as a HUD, central control screen, or chassis. These components can be considered to be in an operating state during vehicle startup.

[0121] In the abnormal noise detection system 100, abnormal noise identification is achieved by the abnormal noise detection device 101. For ease of understanding, the structure of a possible abnormal noise detection device 101 is described below by way of example. Figure 2 , Figure 2 This is a schematic diagram of an abnormal noise detection device provided in an embodiment of this application. The abnormal noise detection device 101 includes an information acquisition module 1011, a feature extraction module 1012, and an abnormal noise recognition module 1013. Optionally, it may also include one or more modules selected from an information sending module 1014, a test mode decision module 1015, or a data collection location recommendation module 1016 (optional modules are indicated by dashed lines). The information acquisition module 1011 is capable of acquiring information. For example, the information acquisition module 1011 can receive audio data (or sound samples) from the acquisition control device 102, and optionally can also acquire one or more types of information such as test case information (not shown), vehicle indication information, or component indication information.

[0122] The feature extraction module 1012 can parse and extract features from audio data to obtain sound feature information of the audio data, such as spectral features, temporal features, and color mapping (described below).

[0123] The abnormal noise recognition module 1013 can identify abnormal noises based on the sound feature information provided by the feature extraction module 1012, diagnose whether there are abnormal noises in the audio data, the cause of the abnormal noises, and processing suggestions, and obtain the detection results.

[0124] In some cases, the abnormal noise recognition module 1013 may have deployed an AI model or be able to call the AI ​​model (e.g., by calling an external model through an application programming interface). The abnormal noise recognition module 1013 can then use the AI ​​model to identify abnormal noises based on the sound feature information of the audio data and obtain the detection results.

[0125] Optionally, the abnormal noise detection device 101 includes an information transmission module 1014. The information transmission module 1014 is used to provide information to devices other than the abnormal noise detection device. For example, the abnormal noise detection device 101 sends the detection results to other devices through the information transmission module 1014. Other devices include one or more of the following: a platform for storing detection results, a data acquisition and control device 102, a vehicle, or a terminal associated with the vehicle.

[0126] In some cases, the abnormal noise detection device 101 includes a test mode decision module 1015. When acquiring audio data, components in the vehicle need to operate in test mode. The test mode decision module 1015 can determine the specific parameters of the test mode (described below). Furthermore, the test mode indication information can be provided to other devices, such as the acquisition control device 102, via the information transmission module 1014.

[0127] Optionally, the abnormal noise detection device 101 can determine the specific parameters of the test mode for a component in the vehicle by using one or more of the vehicle's identification information, component indication information, etc. For example, when testing a specific component of a certain vehicle model, a specific test mode can be used. Similarly, when testing a specific problem with a specific component of a certain vehicle model, a specific test mode can be used, in which case the abnormal noise detection device 101 can obtain a description of the problem in advance.

[0128] In some cases, the abnormal noise detection device 101 includes a sampling location recommendation module 1016. The position of the audio sampling device 103 can be flexibly set. To ensure the sampling effect, the abnormal noise detection device can use the sampling location recommendation module 1016 to determine the recommended sampling location, and send the recommended sampling location to the audio sampling device 103, the sampling control device 102, or other equipment such as vehicles through the information sending module 1014.

[0129] Optionally, the abnormal noise detection device 101 can determine the recommended sampling location by one or more of the vehicle's identification information, component indication information, and description information of the abnormal noise problem.

[0130] The structure of the abnormal noise detection device 101 described above is only an example. In specific implementations, modules can be added or removed. For example, modules can be added or removed according to requirements such as load demand, interaction requirements, system flexibility design requirements, and security isolation requirements.

[0131] In this embodiment, the audio acquisition device 103 needs to be designed to be located near the vehicle (or installed inside the vehicle), while the acquisition control device 102 needs to communicate with both the audio acquisition device 103 and the abnormal noise detection device 101. In specific application scenarios, some devices can be integrated into one device, and there are various possible implementations. For ease of understanding, three application scenarios of this embodiment are described below as examples.

[0132] For application scenario one, please refer to [link / reference]. Figure 3 The first vehicle 300 completes the acquisition and uploading of audio data. The controller 301 (considered the acquisition control device 102) in the first vehicle 300 can acquire audio data from the audio acquisition device 103 and report the audio data. The microphone array in the first vehicle 300, such as... Figure 3As shown, microphones 31 to 34 are disposed at various locations in the first vehicle 300 and are capable of collecting the sound of components in the first vehicle 300 during operation. For example, the first component 302 includes a door of the first vehicle 300, and the microphone array is capable of collecting the sound of the door moving.

[0133] Optionally, the microphone array is a factory-installed microphone array for the first vehicle 300, capable of capturing sounds around the vehicle.

[0134] Alternatively, the microphone array may be added later. For example, microphones 31 to 34 may be detachably mounted in the first vehicle 300, installed during abnormal noise detection, and removed after the detection is completed.

[0135] Combination Figure 4 The first vehicle 300 collects and reports audio data, and the server 200 performs abnormal noise detection based on the audio data. Specifically, the audio acquisition device in the first vehicle 300 can collect first audio data and report it to the controller 301. Accordingly, the controller 301 acquires the first audio data (S12). The controller 301 provides the first audio data to the server 200, the server 200 acquires the first audio data (S13), and obtains the abnormal noise detection result based at least on the first audio data (S14).

[0136] Optionally, the controller 301 can control the audio acquisition device to acquire audio data (S11). For example, the controller 301 can send an acquisition command to the audio acquisition device 103, causing the audio acquisition device 103 to acquire audio. It should be understood that S11 is an optional step, and in some cases, the audio acquisition device is not controlled by the controller 301, for example, by additional equipment control or human control.

[0137] In some cases, the operating state of components in a vehicle can be electrically controlled, and abnormal noises occur when these components are in operation. Therefore, during audio acquisition, the acquisition control device 102 needs to keep the components in an operational state. (Combined with...) Figure 5 The first vehicle 300 can control the first component 302 to operate, and collect audio data when the first component 302 is active. Specifically, the controller 301 in the first vehicle 300 can control the first component to operate in a test mode (S10). In the test mode, the first component 302 can perform a specified action (including one or more actions). Accordingly, the first component 302 is controlled, for example, by receiving a control signal from the controller 301, to operate in the test mode. Further, the controller 301 controls the audio acquisition device to collect audio data (S11), so that the sound of the first component 302 operating in the test mode can be recorded. See below for other operations. Figure 4 Introduction.

[0138] As mentioned earlier, the audio acquisition process can be controlled. Below are three scenarios that trigger audio acquisition.

[0139] Timing 1: An abnormal noise occurs in the first vehicle 300 under specific circumstances, and audio recording can be performed during this specific scenario. Combined with... Figure 6 The controller 301 can acquire information about the abnormal noise generation scenario (S15). In the abnormal noise generation scenario, it controls the audio acquisition device to acquire audio data (S16). S16 can be regarded as... Figure 4 , Figure 5 S11 is shown. Figure 6 The remaining steps are described above.

[0140] In some cases, abnormal noises can be caused by environmental conditions and component operating conditions (optional). Environmental conditions include one or more factors such as vehicle operating conditions, weather conditions, and temperature conditions. (Combined...) Figure 7 Here, taking the operating conditions of a component as the operating conditions of a vehicle as an example, three possible examples are introduced: Example 1: When the vehicle is turning, there is an abnormal noise in the rear seat. At this time, the acquisition control device (such as controller 301) can control the acquisition of audio when the steering wheel angle reaches 15° (a turning condition).

[0141] Example 2: An abnormal noise occurs from the air conditioner when the vehicle brakes. In this case, the audio acquisition and control device controls the acquisition of audio when it detects that the vehicle deceleration has reached the first deceleration value (a braking condition).

[0142] Example 3: When the vehicle is traveling at high speed, an abnormal noise occurs if the sunroof is opened. In this case, the data acquisition and control device detects that the vehicle speed exceeds 80 km / h, controls the sunroof to open, and controls the audio acquisition.

[0143] Timing 2: The user actively triggers audio capture based on their needs. Combined with... Figure 8 The driver or passenger can trigger audio capture via voice interaction. The driver can say, "XX (wake-up word), there is an unusual noise in the car," and the controller, based on the user's voice command, will control the audio capture device to collect audio. Optionally, the voice interaction system can be set up separately from the controller, or the voice interaction system and the controller can be integrated into the same hardware, but regardless of the method... Figure 8 In the scenario shown, the audio acquisition and feedback functions of the voice interaction system and the controller can achieve functional collaboration.

[0144] Besides voice triggering, audio recording can also be triggered in other ways, such as interface control triggering, button triggering, and terminal control triggering. For example, pressing an interface control or a physical button will automatically record audio in the vehicle. After recording, the user can choose to upload feedback on any abnormal noises, and the controller will provide the recorded audio to the server.

[0145] Timing 3: Audio acquisition is performed in an automated process.

[0146] For application scenario two, please refer to [link / reference]. Figure 9 Terminal 400 acts as an audio acquisition device 103 to collect audio data. The first vehicle 300 acquires the audio data collected by terminal 400 and uploads it to server 200.

[0147] In some cases, the audio acquisition function of terminal 400 is controlled by the first vehicle 300. See also Figure 10 The controller 301 in the first vehicle 300 can control the audio acquisition device to acquire audio data (S22). For example, the controller 301 can send an acquisition command to the terminal 400, and the terminal 400 responds to the acquisition command to acquire audio, obtain the first audio data, and provide it to the controller 301. Other operations (such as S23, S24, and S25) can be found in the foregoing or the following description.

[0148] In some cases, the terminal 400 is portable, or can be detachably fixed in the first vehicle 300. The controller 301 can prompt the placement of the audio acquisition device 103 in a recommended acquisition position so that the acquired audio can more accurately reflect the actual situation of the abnormal noise.

[0149] Optionally, the controller 301 may prompt the user to place the audio acquisition device in a recommended acquisition location (S21). For example, the controller 301 outputs a prompt message through a user interaction device (such as a display screen, voice interaction system, etc.) in the first vehicle 300, the prompt message indicating that the audio acquisition device should be placed in the recommended acquisition location. For example, the user interaction device may be a display device capable of displaying a user interface that can prompt the user for the recommended acquisition location. Alternatively, the interaction device may be a voice interaction device capable of outputting a prompt voice that can prompt the user for the recommended acquisition location. Please refer to [link to relevant documentation]. Figure 11 The display screen in the vehicle can show the recommended collection location and can also prompt the current collection item and operation process. For example, it can prompt with text such as "Now we will perform a noise detection on the right rear door. Please place the terminal in the position shown in the figure and click [Start Detection] on the terminal to start audio recording."

[0150] In some scenarios, the controller 301 provides a recommended acquisition location to the terminal 400, which is equipped with a user interaction device. The user interaction device in the terminal 400 outputs a prompt message to instruct the audio acquisition device to be placed at the recommended acquisition location. Please refer to [link to relevant documentation]. Figure 12 The terminal includes a display screen, which shows the recommended data collection location and prompts the user with information about the current data collection item and operating procedure. For example, it may display a text message such as "Now performing a noise detection on the right rear door. Please place the terminal in the indicated location." After placing the terminal in the recommended location, the user can click "Start Detection" to begin audio recording or send a notification to the first vehicle (300) indicating that the terminal is ready. In the latter case, the terminal begins audio recording upon receiving the data collection command from the first vehicle.

[0151] Application scenario two can also include some of the possible implementations described in application scenario one. For example, the controller can control the first component 302 to operate in test mode, enabling the terminal 400 to collect the sound of the first component operating in test mode. Furthermore, the timing of the first vehicle 300 triggering the terminal 400 to perform audio collection can be found in the previous description of the collection timing.

[0152] Application Scenario 3, see Figure 13 Terminal 400 is an independent device, set up separately from the first vehicle 300. As a data acquisition and control device 102, terminal 400 can acquire audio data from audio acquisition device 103 and upload it to server 200.

[0153] Combination Figure 14 As one possible data acquisition process, terminal 400 controls audio acquisition device 103 to acquire audio and reports the audio data. Server 200 performs abnormal noise detection based on the reported first audio data. Specifically, terminal 400 can control audio acquisition device to acquire audio data (S31) and obtain the first audio data (S32). The remaining processes, such as S30, S32, S33, and S34, can be found in the corresponding descriptions above.

[0154] Optionally, the terminal 400 is communicatively connected to the first vehicle 300. The terminal 400 can control the first component 302 in the first vehicle 300 to operate in test mode. Figure 14 The terminal 400 sends a second control command to the first vehicle 300. The second control command is used to instruct the first vehicle 300 to control the first component to run in test mode, so that the first audio data can reflect the sound of the first component 302 running in test mode.

[0155] See Figure 15The terminal 400 can be connected to the controller 301 in the first vehicle 300, and the controller 301 can control the components in the first vehicle 300. For example, the controller 301 can respond to the aforementioned second control command and control the first component 302 to operate in test mode.

[0156] It should be understood that some possible implementations in the aforementioned application scenarios are also compatible with application scenario three. For example, the timing of the terminal 400 controlling the audio acquisition device 103 to perform audio acquisition can be found in the previous description of the acquisition timing. For instance, the terminal 400 can obtain the terminal's operating status data through the connection with the first vehicle 300, and can manually or automatically control the audio acquisition device to acquire audio data when the terminal's operating status data meets certain operating conditions.

[0157] exist Figure 14 In this configuration, the audio acquisition device 103 can be installed separately from the terminal 400. In some cases, the audio acquisition device 103 is a device independent of the first vehicle 300 and the terminal 400.

[0158] In some cases, the audio acquisition device 103 is a microphone array in the first vehicle 300, combined with Figure 16 Terminal 400 can connect to the microphone array in the first vehicle 300, including microphones 31 and 32 (for example only), via controller 301 in the first vehicle 300. Terminal 400 can send commands to controller 301 to indirectly control the operating state of the microphone array. Correspondingly, terminal 400 can also obtain data collected by the microphone array by forwarding data through controller 301. In addition to the above example, this application is also applicable to cases where the audio acquisition device 103 is located within terminal 400.

[0159] The architecture and application scenarios of the embodiments of this application have been described above. The method embodiments of this application are described below.

[0160] See Figure 17 , Figure 17 This is a schematic flowchart illustrating another abnormal noise detection method provided in an embodiment of this application. Optionally, this abnormal noise detection method can be applied in the aforementioned abnormal noise detection system 100. Figure 17 The data processing method shown may include steps S41 to S43. The order of steps here is merely an example, and the embodiments of this application are also applicable to other order of steps, multiple executions of a particular step, etc. S41 to S43 are detailed below: S41, the acquisition control device acquires the first audio data from the audio acquisition device.

[0161] The acquisition and control device is a device with control and communication capabilities, capable of acquiring first audio data and reporting the audio data to the abnormal noise detection device 101. The acquisition and control device can be a terminal or a vehicle, or a module within a terminal or a module within a vehicle; the module can include a software module or a hardware module.

[0162] For example, the data acquisition and control device can be a component in a vehicle, such as a domain controller (DC), mobile data center (MDC), electronic control unit (ECU), or vehicle intranet unit (VIU). The DC may include a cockpit domain controller (CDC) or a vehicle domain controller (VDC). Alternatively, the data acquisition and control device can be a terminal or a module within a terminal that is independent of the vehicle, such as a mobile phone, tablet, or handheld control device.

[0163] The first audio data includes the sound of the first component operating. The first component is a component in the vehicle (referred to as the first vehicle for ease of distinction). For ease of understanding, the following describes two possible states of the first component and explains the meaning of the first component being in an operating state.

[0164] Scenario 1: The first component is a movable component, or the first component includes movable parts. In this case, "the first component is in operation" means that the first component is in an active state, or that the parts included in the first component are in an active state. For example, the first component is a vehicle door, which can be opened and closed. Another example is an air conditioning system where the compressor and dampers are active when the system is turned on. Exemplarily, the first component includes one or more of the following: vehicle cabin door, window door, air conditioning system, air conditioning vent, front trunk door, rear trunk door, fuel tank cap, charging port cover, toolbox door (such as armrest door), seat, screen, display device, suspension system, drive shaft, or motor.

[0165] Furthermore, the operating state or movement of the first component can be electrically controlled. For example, the cabin door of the first vehicle is an electric door, which can be electrically controlled to open and close, and even control the opening movement. As another example, the first component is a sunroof, which can be controlled to open and close, and also control the opening movement.

[0166] Scenario 2: The first component is a non-movable component, i.e., a fixed part. The first component being in an operational state means that the first vehicle is in a running state or in motion. For example, the first component might be a fixed part such as the central control screen or the windshield, which are considered to be in an operational state during the vehicle's startup. Similarly, when the first vehicle is lifted or bumped by an external force, the vehicle moves, and some fixed parts may produce abnormal noises. Therefore, when the first vehicle is subjected to external forces, the components within the first vehicle are also considered to be in an operational state.

[0167] The first audio data is acquired by an audio acquisition device. This device is capable of recording sound and includes one or more microphones. The audio acquisition device is positioned near the vehicle or directly reuses an existing audio acquisition device within the vehicle. Two possible designs for the audio acquisition device are described below: Design 1: The audio acquisition device includes a microphone or microphone array permanently installed in the vehicle, such as... Figure 3 , Figure 8 Examples include the following embodiments. For instance, the audio acquisition device is a microphone array built into the vehicle. For example, a vehicle may include 16 microphones at the time of manufacture. These 16 microphones are arranged in various parts of the vehicle to form a microphone array, and the abnormal noise detection process can reuse the microphone array built into the vehicle.

[0168] Design 2: The audio acquisition device is located in a separate device from the vehicle, such as... Figure 9 , Figure 13 , Figure 15 Examples include [examples of other implementations]. In some cases, the audio acquisition device is a portable or detachable device. This type of audio acquisition device is positional relative to the vehicle, allowing for flexible changes in the acquisition location to accommodate various testing requirements.

[0169] As one implementation example, the audio acquisition device is a portable terminal, such as a mobile phone or tablet. As another implementation example, the audio acquisition device is a microphone or microphone array installed in a vehicle. For instance, to improve testing results, professional microphone equipment can be permanently installed in the vehicle for testing.

[0170] In Design 2, due to the large size of the vehicle, the sound acquisition effect varies depending on the location. In some possible implementations, where the location of the audio acquisition device can be flexibly designed, the acquisition control device can prompt the user to place the audio acquisition device in a recommended acquisition location via an interactive device.

[0171] For example, recommended sampling locations include inside the vehicle cabin, above the trunk lid, and outside the side doors. Interactive devices include displays, audio output devices (such as voice interaction systems), indicator lights, and vibration motors to prompt users to place the audio sampling device at the recommended location. For example, combining... Figure 11 or Figure 12 The acquisition control device can be connected to a display screen, such as a display screen in a vehicle or a display screen on an audio acquisition device. The display screen can present a user interface and thus suggest recommended acquisition locations.

[0172] In some possible implementations, the recommended sampling location varies depending on the component or location of the abnormal noise. In other words, the recommended sampling location is associated with the component (or the location of the abnormal noise). Referring to Table 1, abnormal noise from the door occurs at the connection between the door and the body; therefore, when testing for abnormal noise from the door, the recommended sampling location is near the connection between the door and the body. Similarly, in the V01 model, the air conditioning system is located at the front of the vehicle; when testing for abnormal noise from the air conditioning system, the recommended sampling location is near the footwell of the driver or passenger seat in the cabin.

[0173] Optionally, based on Table 1, the recommended data collection location may also be vehicle-specific. For example, for the V02 model where the air conditioning system is located behind the rear seats, the recommended data collection location when abnormal noises occur in the air conditioning system is the trunk.

[0174]

[0175] Of course, the recommended sampling locations listed here are for ease of explanation regarding the recommended sampling locations corresponding to different testing items. In specific implementations, the equipment or manufacturer may determine the recommended sampling locations based on one or more of the following: the layout of components inside the vehicle, the location of the abnormal noise, and safety during testing.

[0176] In some possible implementations, the recommended data acquisition location is provided by an external device. For example, a server provides indication information of the recommended data acquisition location to the data acquisition control device, and the data acquisition control device receives the indication information from the server to determine the recommended data acquisition location.

[0177] The recommended data collection location indication information can indicate the recommended data collection location. For example, the recommended data collection location indication information may be an image or text description. The data collection control device can display the image or text description through an interactive device, or read the text description aloud, thereby prompting the user with the recommended data collection location, such as... Figure 10 or Figure 11 As shown. For another example, the space inside the vehicle cabin and the surrounding exterior of the vehicle is divided into multiple zones, each zone is numbered (or distinguished by an identifier). The indication information for the recommended collection location can be the number (or identifier) ​​of a certain zone (such as the first zone), and the collection control device can index the first zone through the number of the first zone, and use the interactive device to display an image or text description of the first zone, or to read the text description of the first zone aloud.

[0178] In one scenario, the server provides the data acquisition control device with indications of recommended acquisition locations for one or more detection items. The data acquisition control device then prompts for the recommended acquisition locations corresponding to the detection items to be performed.

[0179] Optionally, the data acquisition control device obtains the indication information of the first component input by the user and sends the indication information of the first component to the server. Optionally, it also includes the identification information of the first vehicle, which indicates the vehicle model. The recommended acquisition location provided by the server is a recommended acquisition location for acquiring abnormal noises related to the first component. For example, during data acquisition, the user selects a test item on the data acquisition control device, such as testing for abnormal noises from the right rear door, and reports the test item to the server. Based on the test item, the server feeds back the recommended acquisition location corresponding to the test item to the data acquisition control device.

[0180] The above describes the audio acquisition device and its location. In some embodiments, the timing of audio acquisition is controlled. Three acquisition timing designs are described below: Acquisition Timing 1: The abnormal noise occurs in a specific scenario, and audio acquisition can be performed during this scenario. As a possible implementation example, the acquisition control device obtains information about the scenario in which the abnormal noise occurs. This information describes the scene at which the abnormal noise occurs, including environmental conditions and component operating conditions. Environmental conditions include one or more conditions such as vehicle operating conditions, weather conditions, and temperature conditions. Component operating conditions indicate how the component operates, such as whether the car windows need to be open or the air conditioning is in cooling mode.

[0181] As one possible implementation example, the abnormal noise generation scenario includes situations where the operating status data of the first vehicle meets the vehicle operating status conditions. In this case, the acquisition and control device can acquire the operating status data of the first vehicle, determine whether the operating status data of the first vehicle meets the vehicle operating status conditions based on the operating status data of the first vehicle, and thus perform audio acquisition in scenarios where the vehicle operating status conditions are met.

[0182] Combination Figure 7 Example 1 illustrates a scenario where the abnormal noise occurs during vehicle cornering. The data acquisition and control device can obtain vehicle operating status data, including information such as the vehicle's steering wheel angle, navigation route, or planned driving path that indicates whether the vehicle is in a cornering scenario. When the vehicle's operating status data indicates that the vehicle is in a cornering scenario, the device controls the audio acquisition device to collect audio data. For example, audio acquisition can be controlled when the steering wheel angle reaches 15°.

[0183] Combination Figure 7Example 2 illustrates a scenario where the abnormal noise occurs during vehicle braking. The data acquisition and control device can obtain vehicle operating status data, including information such as vehicle deceleration, brake pedal opening, and speed changes, indicating whether braking is in progress. When the vehicle operating status data indicates that the vehicle is in a braking scenario, the audio acquisition device is controlled to collect audio data. For example, audio can be collected when the vehicle deceleration reaches a first deceleration value. This first deceleration value can be predefined, included in the abnormal noise scenario information, or obtained through intelligent analysis of the abnormal noise scenario input by the user.

[0184] In some possible implementations, the abnormal noise generation scenario includes situations where the operating status data of the first vehicle meets the vehicle operating status conditions, and situations where the abnormal noise generation scenario also includes the execution action of the first component, where the execution action includes single action execution and / or multiple action execution. When the operating status data of the first vehicle meets the vehicle operating status conditions, the acquisition control device sends an acquisition command to the audio acquisition device, and the first audio data includes the sound when the first component performs the aforementioned execution action. In some cases, the execution action of the first component does not require additional command control. In other cases, the execution action of the first component requires additional command control. In this case, when the operating status data of the first vehicle meets the vehicle operating status conditions, the acquisition control device controls the first component to perform its execution action and sends an acquisition command to the audio acquisition device.

[0185] Combination Figure 7 Example 3 illustrates a scenario where the abnormal noise occurs when the vehicle is traveling at high speed with the sunroof open. In this case, the data acquisition and control device can obtain the vehicle's operating status data, including its speed. When the vehicle's speed reaches a certain speed threshold, such as 80 km / h, the data acquisition and control device controls the sunroof to open at least once and controls the audio acquisition device to collect audio data. The aforementioned speed threshold can be predefined, included in the information of the abnormal noise occurrence scenario, or obtained through intelligent analysis of the abnormal noise scenario input by the user.

[0186] In some possible implementations, the acquisition and control device can receive abnormal noise descriptions input by the user and determine information about the scenario in which the abnormal noise occurs based on these descriptions. For example, the acquisition and control device can obtain the user's description of the abnormal noise through one or more rounds of dialogue and automatically extract information about the scenario in which the abnormal noise occurs based on the user's description.

[0187] For example, the data acquisition and control device obtains abnormal noise description information through one or more rounds of dialogue with the user. The multi-round dialogue may include prompting the user to provide details about the abnormal noise. For instance, if the abnormal noise description is "there is an abnormal noise when the vehicle turns," the data acquisition and control device can transform this description into a scenario where the abnormal noise occurs, such as "the vehicle's turning angle is ≥15°." This converts the user's natural language description of the abnormal noise into quantifiable information, facilitating subsequent testing.

[0188] As another example, the acquisition and control device can obtain abnormal noise description information through interface input, such as by selecting or inputting vehicle speed, location, time, sound volume, etc.

[0189] Acquisition timing 2: Audio acquisition is performed in response to user requests. For example, the acquisition control device obtains detection request information input by the user, and in response to the detection request information, controls the audio acquisition device to acquire audio (e.g., sends an acquisition command to the audio acquisition device to enable it to acquire audio), thus obtaining first audio data.

[0190] The detection demand information indicates the user's current need for audio acquisition, reflecting the user's intent to detect abnormal noises. For example, if the detection demand information is voice information, the acquisition control device analyzes the user's input voice information to determine the user's intent. If the user's intent is to report a problem with the first vehicle, the device controls the audio acquisition device to acquire audio. Figure 8 The user activates the voice system by uttering a wake-up word. The user can then use voice commands such as "There's an unusual noise in the car," "I want to report a problem," "There's an unusual noise in the cabin," or "Detect what this sound is," to indicate a need to report a problem with the vehicle. Upon receiving these voice commands, if the user's intention is to report a problem with the vehicle (or specifically, an unusual noise), the vehicle can record audio as the first audio data.

[0191] The above-described scenario regarding voice input detection requirements is merely an example. In specific implementations, user requests can be input in other modalities, such as interface input, button input, or terminal input.

[0192] For example, when a user presses an interface control on a user interface or a physical button, audio recording automatically begins in the vehicle. After recording, the user can choose to upload feedback on any abnormal noise issues, and the acquisition and control device will provide the recorded audio to the abnormal noise detection device.

[0193] Acquisition Timing 3: Audio acquisition occurs within an automated process. For example, after vehicle factory testing or after-sales testing, audio acquisition is automatically triggered at a certain step in a predefined process. Another example is when the controller controls the audio acquisition device to acquire audio while or before controlling the first component to move in test mode.

[0194] In some possible implementations, the acquisition control device can control the audio acquisition device to acquire first audio data. As one possible implementation, the acquisition control device sends an acquisition command to the audio acquisition device, which receives the acquisition command from the acquisition control device and, in response to the acquisition command, acquires the sound of the first component operating, obtains the first audio data, and sends it to a device outside the audio acquisition device. Exemplarily, the audio acquisition device sends the first audio data to the acquisition control device, causing the acquisition control device to receive the first audio data from the audio acquisition device.

[0195] In some scenarios, the operating status of vehicle components can be electrically controlled, and abnormal noises occur when these components are in operation. Accordingly, the acquisition and control device can control the components to be in operation, ensuring that the audio collected by the audio acquisition device reflects the sound of the vehicle when the components are in operation.

[0196] In one possible implementation, the acquisition and control device can control the first component to operate in a test mode, and the first audio data includes the sound of the first component operating in the test mode. The test mode refers to the operating mode used to test for abnormal noises. For example, the test mode can indicate the number of times the first component performs an action and the action parameters for each action. The action parameters are used to indicate the type of action, and / or, the action of the first component has a travel distance, and the action parameters are used to indicate the travel distance of the action. For example, the actions that a car door can perform include door opening, door closing, locking, unlocking, etc., where opening and closing have a travel distance, such as the opening angle or opening ratio. Similarly, the actions that a seat can perform include backrest angle, seat angle, leg rest angle, one-touch zero-gravity opening, one-touch zero-gravity closing, seat heating (optionally including multiple levels), seat ventilation (optionally including multiple levels), seat massage, seat audio on, seat audio off, seat audio volume adjustment, etc., one or more of these actions, some of which have an adjustment travel distance.

[0197] Optionally, the testing mode adopts a standardized procedure, such as using a fixed combination of actions for each door test. Referring to Table 1, which lists the testing modes for the three components, the acquisition control device can control the door to execute actions 1 to 7 sequentially, thereby obtaining the sound inside the vehicle under this fixed combination of actions.

[0198]

[0199] Optionally, the test mode can be flexibly defined. For example, the actions of the test mode can be customized based on the description of the abnormal noise, the situation of other vehicles in the batch, user needs, etc.

[0200] In some possible implementations, the acquisition control device obtains a first control command, and based on the first control command, controls a first component in the first vehicle to operate in a test mode. The first audio data includes the sound of the first component operating in test mode. Exemplarily, the acquisition control device is a component in the first vehicle. Based on the first control command, the acquisition control device sends a control signal to the first component, the control signal being used to control the first component to operate in test mode. Further exemplarily, the acquisition control device is a terminal or a module within a terminal, the terminal having a communication connection with the first vehicle. Based on the first control command, the acquisition control device sends a second control command to the first vehicle, the second control command being used to instruct the first vehicle to control the first component to operate in test mode.

[0201] Optionally, the first control command may be a pre-set command within the data acquisition and control device, or a command from an automated execution process (i.e., originating internally). Alternatively, the first control command may be input by the user. Alternatively, the first control command may originate externally; in other words, the data acquisition and control device may receive a first control command from other devices (such as an abnormal noise detection device).

[0202] In some possible implementations, the test mode is indicated by the abnormal noise detection device or a server. For example, the server sends test mode indication information to the acquisition control device, which then controls the first component to operate in the test mode based on this indication information. Optionally, the test mode indication information may directly include the action combination and corresponding action parameters corresponding to the test mode, or it may include a number (or identifier) ​​corresponding to the test mode, enabling the acquisition control device to index the corresponding test mode based on the corresponding number or identifier.

[0203] In some possible implementations, the abnormal noise detection device or server sends test information to the acquisition and control device. The test information includes at least one of the following information items: test mode information, first control command, and indication of recommended acquisition location. When the test information includes multiple information items, the multiple information items can be sent in one message, or each information item can be sent as an independent message.

[0204] In some other possible implementations, the test modes are pre-stored in the acquisition control device and do not need to be obtained from external sources.

[0205] Optionally, the acquisition control device prompts the user when audio acquisition begins. Optionally, the user is prompted to save and / or upload the audio after audio acquisition is completed.

[0206] S42, the acquisition and control device sends detection data to the abnormal noise detection device. Correspondingly, the abnormal noise detection device acquires the first audio data from the acquisition and control device.

[0207] In this embodiment, the abnormal noise detection device is a device with data processing capabilities, capable of processing audio data to obtain detection results. In some cases, the abnormal noise detection device is located on the server side, or in the cloud, specifically as a server or a module within a server. See the foregoing architecture section for details.

[0208] The detection data includes first audio data. For example, the acquisition control device is a controller in a vehicle, and the vehicle is equipped with a communication device. The controller can provide detection data to the abnormal noise detection device via the communication device, and correspondingly, the abnormal noise detection device receives the data. As another example, the acquisition control device is a terminal, which is communicatively connected to the abnormal noise detection device, and the terminal uploads detection data through the communication connection.

[0209] Optionally, the detection data also includes test case information, which describes the acquisition environment of the first audio data. For example, the test case information includes at least one of the following: the operating status data of the first vehicle, the identification information of the first vehicle, the batch of the first vehicle, the acquisition time of the first audio data, or the acquisition location of the first audio data, or more of the following.

[0210] S43, the abnormal noise detection device obtains the abnormal noise detection result based on the detection data.

[0211] The abnormal noise detection device can process and analyze audio data collected during vehicle operation, identify abnormal sounds, and obtain abnormal noise detection results. Abnormal noise detection results refer to the conclusions obtained after identifying abnormal noises based on audio data; they can also be called a detection report, diagnostic report, etc., or included within a detection report or diagnostic report.

[0212] The abnormal noise detection results include the abnormal noise status, which indicates whether an abnormal noise exists in the first vehicle. Referring to Table 3, for example, the abnormal noise status is represented by status values ​​1 and 0, where 1 indicates the presence of an abnormal noise and 0 indicates the absence of an abnormal noise. Another example is the abnormal noise severity, indicated by multiple levels, from highest to lowest: A, B, C, and D. Level D indicates no abnormal noise, while levels A, B, and C all indicate the presence of an abnormal noise. Of course, these values ​​and level representations are merely examples and are not intended to limit the number of levels or their specific values.

[0213]

[0214] Optionally, the abnormal noise detection results also include the cause of the abnormal noise, which includes indication information of the noisy component. For example, the noisy component may be a rotating shaft, rubber strip, or other parts. It should be noted that the noisy component may or may not belong to the first component. For example, it may be a component that interferes with the movement of the first component, or a component related to the electrification control of the first component, etc.

[0215] Optionally, the abnormal noise detection results may also include handling suggestions, which are used to indicate how to handle the abnormal noise.

[0216] Optionally, the detection data also includes test case information, which describes the acquisition environment of the first audio data. The abnormal noise detection results also include at least a portion of the test case information. For example, referring to Table 4, the abnormal noise detection results may include one or more information such as the audio data acquisition environment, vehicle information, abnormal noise status, severity, and cause of the abnormal noise. Vehicle information includes the vehicle identification number (VIN), and the audio data acquisition environment includes one or more information such as the temperature conditions during the test, the test site, and the audio data upload time.

[0217]

[0218] In some possible implementations, the noise detection device can provide a diagnostic result viewing service. Devices or applications connected to the noise detection device can request detection results, and the noise detection device can provide the detection results to the device or application with valid permissions. It should be noted that the formats of the detection results listed in Tables 3 and 4 above are only examples, and the content and display format of the detection results in specific implementations may have other designs.

[0219] The above describes the results of abnormal noise detection. Below, we introduce some designs for obtaining abnormal noise detection results based on audio data.

[0220] In one possible design, the abnormal noise detection device can obtain sound feature information based on the first audio data, and obtain the abnormal noise detection result of the first vehicle based on the sound feature information. The audio data processing flow is as follows: Figure 2 As shown. Since sound is a physical signal, the first audio data is an analog signal or a discretized digital signal that can characterize the sound. Sound feature information, on the other hand, is quantified information extracted from the first audio data that characterizes the physical properties or patterns of the sound. This information can distinguish different sounds and provide a basis for subsequent detection results.

[0221] Sound feature information includes one or more of the following: frequency information, amplitude information, and time-varying information of frequency and amplitude. Different types of features correspond to different dimensions of sound, such as time, frequency, human auditory perception, or transient changes, etc. Several possible features are described below: Feature 1: Time-domain features. The time domain is the most intuitive dimension of a sound signal, describing the fluctuation of a certain dimension (usually amplitude) of the signal over time, such as... Figure 18 This paper introduces the variation of sound amplitude over time.

[0222] For example, amplitude information includes peak value and peak-to-peak value. Peak value is the maximum amplitude of a signal over a period of time, reflecting the maximum instantaneous energy. Peak-to-peak value is the difference between the maximum and minimum amplitude, reflecting the range of signal fluctuations. For instance, when the shock absorbers in a vehicle's suspension system are functioning normally, the peak sound when going over a speed bump is the first amplitude. If the shock absorbers malfunction, the direct impact between the spring and metal will produce a stronger transient impact, increasing the peak value to the second amplitude, which is greater than the first amplitude, and the range of peak-to-peak values ​​will also expand.

[0223] For another example, amplitude information includes the root mean square (RMS), which is the square root of the average of the squares of the amplitude of the sound signal over a period of time, corresponding to the average energy of the sound. As an example, when a vehicle engine is idling normally, its sound RMS will be stable at the third amplitude value. If there is piston ring wear, the irregular impact caused by the wear will cause the sound energy to fluctuate, and the RMS will abnormally rise to above the fourth amplitude value, with a significant increase in the fluctuation frequency.

[0224] Feature 2: Frequency Domain Features. Sound is essentially the propagation of mechanical vibrations; different faults correspond to vibration sources that produce sound at specific frequencies. Frequency domain features, through Fourier transform, convert the time-domain signal to the frequency dimension, facilitating the location of abnormal frequency components. For example, frequency information includes spectral peaks, which are the maximum energy values ​​at a specific frequency point in the frequency domain diagram, corresponding to the most prominent frequency component in the sound, thus facilitating the identification of abnormal sound. As another example, frequency information includes the centroid frequency, which is the frequency point where sound energy is concentrated, reflecting whether there is a frequency shift in the sound. Identifying this shift can help determine if there are problems such as aging or insufficient installation strength.

[0225] Feature 3, time-frequency domain features, such as time-varying information of frequency and amplitude. Time-frequency domain features combine the temporal and frequency characteristics of sound. For example, a color map is a tool that maps the numerical characteristics of audio data (such as energy, amplitude, spectral intensity, etc.) to visual colors, primarily used for visualizing audio signals to help intuitively understand the characteristics of sound. Figure 19 In a color map, the horizontal axis of pixels represents the time domain, the vertical axis represents frequency, and the pixel value represents the sound pressure level amplitude (in dB). Colors can include multiple color values. Figure 19 For ease of illustration, only 5 color values ​​are shown, each representing the amplitude of the corresponding level.

[0226] As can be seen, the audio data is transformed into a standardized image defined by a time axis, a frequency axis, and pixel values ​​(representing amplitude), realizing the data structuring and visual representation of the sound signal. Because the model and algorithm have a very high processing capability for visual features, using color mapping for abnormal sound recognition can significantly improve the accuracy and intelligence level of detection.

[0227] Feature 4: Mel-frequency cepstrum (MFCC). Since human sound perception is not linear, MFCC is a feature designed based on this characteristic, compressing redundant information and retaining the most critical discriminative features. For example, abnormal noise caused by poor gear meshing during seat back angle adjustment may exhibit a change in the overall distribution of multiple frequency components compared to normal sound. In the case of gear wear, gear movement leads to an increase in some frequency components, causing changes in the MFCC coefficients. Accordingly, the abnormal noise detection device can identify that the abnormal noise during seat adjustment is caused by gear wear by comparing the differences in the MFCC coefficients.

[0228] The above four features are merely examples. In specific implementations, the fault detection device can obtain one or more of the above features through feature extraction for subsequent abnormal noise detection.

[0229] In some possible implementations, the first audio data includes audio data from multiple channels. When detecting abnormal noises, only the sound feature information from a subset of the channels can be selected for detection. Specifically, the abnormal noise detection device obtains audio data from at least one channel based on the first audio data, and the distance between the acquisition location of the at least one channel's audio data and the sound source location meets a preset condition. The abnormal noise detection device obtains sound feature information based on the audio data from at least one channel, and based on the sound feature information, obtains the abnormal noise detection result for the first vehicle.

[0230] Combination Figure 20The audio acquisition device includes eight microphones, numbered 31 to 38, and the first audio data includes data from eight channels. When detecting abnormal noise, M channels closest to or relatively close to the sound source can be selected from the eight channels for detection, where M is an integer and M≥2.

[0231] In one scenario, before extracting sound feature information, the abnormal noise detection device can analyze the distance between the audio data of each channel and the sound source based on the first audio data, and select at least one channel's audio data based on the distance between the audio data of each channel and the sound source. The abnormal noise detection device inputs the audio data of at least one channel into the feature extraction module for feature extraction to obtain sound feature information, and performs abnormal noise detection based on the sound feature information.

[0232] In another scenario, during the audio data processing stage, the abnormal noise detection device simultaneously analyzes the distance between each channel and the sound source, as well as the acoustic characteristics of each channel's audio data. The output of the feature extraction stage is the acoustic feature information corresponding to the audio data of the channels that meet preset conditions. In other words, during the acoustic feature information extraction stage, channels are filtered, and the final output acoustic feature information is the acoustic feature information corresponding to the filtered channels.

[0233] The extraction of sound feature information from the first audio file has been described above. Below, we introduce two possible scenarios for obtaining abnormal noise detection results based on sound feature information: In scenario 1, the abnormal noise detection device obtains the abnormal noise detection result of the vehicle based on the sound feature information of the first audio data and the feature information of multiple abnormal noise audios. The feature information of each audio is used to indicate the characteristics of at least one abnormal noise.

[0234] In other words, the abnormal noise detection device compares and analyzes the sound characteristic information of the vehicle under test with an information database containing various known abnormal noise characteristics to obtain the abnormal noise detection result. Furthermore, by matching the characteristic information of multiple abnormal noise audio, it can not only determine whether an abnormal noise exists, but also determine the specific type of abnormal noise, and even determine the severity of the abnormal noise by energy intensity, frequency of occurrence of the abnormal noise, and pitch of the abnormal noise sound.

[0235] Scenario 2: The abnormal noise detection device can utilize an AI model for detection. The device inputs sound feature information into the AI ​​model, which outputs the abnormal noise detection results. Some AI models possess non-linear mapping and feature learning capabilities, enabling them to calculate deep, complex relationships between sound features and abnormal noise states—relationships that are difficult for the human ear or traditional algorithms to define—effectively handling various complex abnormal noise patterns. Furthermore, AI models have generalization capabilities, improving recognition accuracy in challenging scenarios.

[0236] AI is used to describe computer systems that perform tasks approaching human intelligence (such as classification, recognition, and visual perception); that is, intelligence generated by software. Models are mathematical functions or rules learned from experience. For example, AI models include models trained using machine learning, deep learning, reinforcement learning, etc., such as neural network models, decision trees, classification trees, and Transformer models. Neural network models include convolutional neural networks (CNNs) and recurrent neural networks (RNNs). In some possible implementations, the AI ​​model is a pre-trained model. That is, the AI ​​model can be trained and is obtained through pre-training.

[0237] In the example above, abnormal noise detection is divided into two stages: feature extraction and abnormal noise recognition. In actual implementation, feature extraction may only serve as an internal input; the abnormal noise detection device can directly identify abnormal noises based on audio data to obtain the recognition result, and feature extraction may only be an internal step. Optionally, the features extracted include one or more hidden features expressed mathematically, such as multi-dimensional vectors or embedded features, where sound features are embedded.

[0238] In some possible implementations, the detection data also includes the operating status data of the first vehicle. This operating status data can be combined with the analysis during abnormal noise identification. Examples of the first vehicle's operating data include vehicle speed, gear position, engine speed, and air conditioning setting. On one hand, the operating status data of the first vehicle supplements the audio data with information about the acquisition scene, aiding in accurate analysis. On the other hand, the operating status data of the first vehicle can also eliminate noise during normal vehicle operation, distinguishing sounds generated by vehicle operation from genuine abnormal noises, thus reducing the false positive rate. This implementation method can be combined with the aforementioned feature extraction; for example, features extracted after the audio data enters the feature extraction stage can be combined with the vehicle's operating data for abnormal noise identification.

[0239] In some possible implementations, the abnormal noise detection device analyzes the abnormal noise detection results of vehicles belonging to the same batch and generates a comprehensive inspection report for that batch. Based on the abnormal noise detection results of a first vehicle and the abnormal noise detection results of at least one second vehicle in the same batch as the first vehicle, the abnormal noise detection device obtains a batch inspection report, which indicates whether the abnormal noise detection of the first batch of vehicles is qualified.

[0240] For example, the abnormal noise detection device can also receive second audio data from the acquisition and control device. The second audio data reflects the sound of a component (optionally the same type as the first component) operating in the second vehicle. The abnormal noise detection device can obtain the abnormal noise detection result of the second vehicle based on the second audio data. The number of second vehicles can be designed to be multiple, so as to collectively reflect whether there are quality defects in the first batch, which helps to provide a fault detection view of the batch and provides a reliable basis for decision-making for subsequent quality improvement.

[0241] In some possible implementations, the abnormal noise detection device sends the abnormal noise detection results of the first vehicle to the first vehicle and / or a first terminal, whereby the first terminal is a terminal pre-configured for receiving the abnormal noise detection results of the first vehicle. For example, the first terminal is the diagnostic equipment of the inspector of the first vehicle, or the first terminal is the device logged into by the owner of the first vehicle (or a user with access rights).

[0242] For example, the abnormal noise detection device also sends the abnormal noise detection results of the first vehicle to the acquisition and control device. Correspondingly, the acquisition and control device receives the abnormal noise detection results from the abnormal noise detection device. The abnormal noise detection results include the abnormal noise status, which indicates whether the first vehicle has an abnormal noise. In this way, the acquisition and control device can obtain detection results containing a clear abnormal noise status, enabling the user at the acquisition end to quickly obtain an authoritative fault diagnosis, thereby enhancing the practicality of the entire detection system and realizing a service closed loop.

[0243] Furthermore, the acquisition and control device outputs abnormal noise detection prompt information through the interactive device, which is used to indicate the abnormal noise detection results.

[0244] exist Figure 17 In the illustrated embodiment, the acquisition and control device acquires first audio data reflecting the operating sounds of components in the vehicle and submits it to the abnormal noise detection device, which then obtains the abnormal noise detection results based on the first audio data. By adopting an architecture that separates audio acquisition, acquisition control, and abnormal noise identification, the automated collection, reporting, and analysis of audio data are achieved, improving the efficiency, consistency, and accuracy of abnormal noise detection, while reducing quality risks and detection costs.

[0245] Furthermore, the above embodiments can also support multiple acquisition methods such as static acquisition and dynamic acquisition, meeting the abnormal noise detection needs in various scenarios, such as factory testing, fault diagnosis, and user-initiated testing.

[0246] Furthermore, the above embodiments also support intelligent functions such as sampling location recommendation and sampling location prompts, which can lower the threshold for use and improve detection accuracy.

[0247] The basic architecture of the solution has been introduced above, and several possible implementation methods have been provided. These various implementation methods can be combined without mutual exclusion. For ease of understanding, several possible implementation methods are illustrated below, and the methods in the following examples can also be combined without mutual exclusion.

[0248] Example Scheme 1 utilizes the vehicle's ECU and microphone for coordinated control, enabling automatic audio data acquisition and submission. Specifically, the ECU sends motion control commands to movable components within the vehicle, such as the number of door lock / unlock cycles and the travel parameters when the sunroof opens. The ECU then uses these control signals to drive the actuators of these movable components to execute these commands. Simultaneously, the ECU triggers the vehicle's built-in microphone to collect audio data, which is then uploaded by the ECU to a server for abnormal noise detection. Example Scheme 1 achieves a fully automated abnormal noise detection process, including automatic triggering of vehicle-mounted systems, automatic data acquisition, and automatic data transmission to the cloud.

[0249] Example Scheme 2: During after-sales service, the vehicle's driver or maintenance personnel can report an abnormal noise problem in a first component of the first vehicle to the cloud. A diagnostic application in the vehicle (e.g., installed in the vehicle's infotainment system) can provide the server with indication information for the first component (optionally including the vehicle's identification information), and the cloud can provide the first vehicle with indication information for a test mode. Based on the test mode indication information, the diagnostic application controls the first component to operate in test mode, such as opening and closing the door a specific number of times, raising and lowering the cargo door a specific distance, or cycling a specific seat's fore-and-aft adjustment. While controlling the operation of the first component, the diagnostic application can trigger an audio acquisition device to collect audio data. Optionally, the audio acquisition device includes a microphone array deployed inside the vehicle (e.g., in the driver's cab or passenger compartment), and / or a multi-channel microphone array deployed outside the vehicle (e.g., for components such as the cargo box or chassis). The diagnostic application acquires the multi-channel audio data collected by the audio acquisition device and uploads the multi-channel audio data and vehicle status data at the time of audio acquisition to the server via the vehicle networking module for abnormal noise detection.

[0250] Example 3 uses multiple microphones to collect audio data, such as four microphones: driver's seat, front passenger seat, second-row left, and second-row right. When the server performs abnormal noise detection, it can automatically extract data from the microphone closest to the sound source from the four microphones for feature extraction (i.e., audio data processing) and abnormal noise diagnosis, depending on the specific vehicle components being tested, such as the driver's seat or sunroof. Of course, the four microphones mentioned here are for illustrative purposes only; the actual channel settings are merely illustrative.

[0251] Some of the solutions mentioned above involve using AI models for abnormal noise identification. Below, we will combine this with... Figure 21This paper introduces the training process of an AI model for abnormal noise recognition. The AI ​​model construction process includes data processing, model design, construction and training, evaluation and optimization, deployment and inference.

[0252] The data processing stage includes one or more of the following operations: abnormal sound dataset creation, abnormal sound data annotation, abnormal sound data preprocessing, abnormal sound dataset partitioning, and color map preprocessing. Abnormal sound dataset creation refers to forming a dataset from audio data of various abnormal sounds, with one or more audio data points forming a training sample within the dataset. Abnormal sound data annotation adds labels to the dataset and / or training samples, such as distinguishing between normal and abnormal types. Abnormal sound data preprocessing refers to preprocessing the abnormal sound data, such as cleaning, denoising, enhancement, extracting partial channels, and extracting sound features, with the aim of improving data quality. Abnormal sound dataset partitioning refers to dividing the dataset into different sets according to certain rules, such as training, validation, and test sets, to meet the needs of model training and evaluation. Color map preprocessing refers to extracting a color map from the audio data; optional preprocessing of the color map may also be included.

[0253] The goal of the model design phase is to select or design a model suitable for the abnormal sound recognition task. For example, ResNet series networks and / or Inception networks can be used as AI models for abnormal sound recognition. When designing ResNet series networks, it is necessary to base the design on the ResNet network architecture and compare the differences between ResNet models of different depths (layers). Inception networks also include various types. Taking InceptionV3 as an example, if InceptionV3 is selected, its network characteristics need to be analyzed, and its network architecture needs to be designed.

[0254] The goal of the build and training phase is to utilize tools such as Python and PyTorch to set up the environment and train the model. Taking deep learning models (such as CNNs) as an example, the build and training phase includes two sub-phases: deep learning model building and deep learning model training. In the deep learning model building phase, the development environment and model framework are set up manually or automatically, the model is created or loaded, and training hyperparameters (such as learning rate, batch size, etc.) are set. In the deep learning model training phase, the training loop of the neural network (such as forward propagation, back propagation, parameter updates) is implemented manually or automatically, and the CNN training mechanism is optimized for abnormal sound detection scenarios.

[0255] The goal of evaluation and optimization is to validate model performance and iteratively improve it. Taking deep learning models (such as CNNs) as an example, in the evaluation phase, predefined evaluation metrics can be used to validate the model, such as accuracy and recall. Cross-validation is a validation method that can verify the model's generalization ability. In the optimization phase, one or more optimization methods, such as hyperparameter optimization (e.g., adjusting the learning rate, number of network layers), regularization (to prevent overfitting), and optimization algorithms (e.g., Adam, SGD), are used to optimize the model and improve its performance.

[0256] Deployment and inference refer to the practical application of the trained model. For example, the best-performing model can be manually or automatically saved and deployed to the cloud (server-side) or edge devices (such as industrial equipment, mobile phones, etc.). For example, during the model inference phase, the deployed model can be used for abnormal noise detection. When the model is deployed in the cloud, a communication process between the edge and cloud sides can be manually or automatically established, allowing the model to detect abnormal noises in real-time in real-world scenarios.

[0257] In some scenarios, adopting a terminal-based data acquisition and cloud-based detection architecture, and using AI models in the cloud for automated abnormal noise detection, can significantly improve detection accuracy and efficiency. In particular, it can improve detection accuracy under external noise interference; under noise interference, the accuracy of abnormal noise identification and classification can reach over 85%, and the detection accuracy under no external noise interference can also reach over 85%.

[0258] Furthermore, in some solutions, the input to the AI ​​model is the extracted sound feature information. In this case, the feature extraction module can be performed using cloud services or digital platforms, which can significantly increase the accuracy and efficiency of the extracted features.

[0259] The methods of the embodiments of this application have been described in detail above. The apparatus of the embodiments of this application is provided below.

[0260] It should be understood that the division of units in the apparatus provided in the embodiments of this application is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Moreover, the names of the units below are only examples, and in actual implementation, the names of the apparatus, modules, systems, and functions may exist in other designs.

[0261] In some cases, the units in the device are implemented by the processor calling software. For example, the device includes a processor connected to a memory that stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the modules of the device. The processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is either internal or external to the device.

[0262] In other cases, the units in the device are implemented as hardware circuits. The functionality of some or all of the units is achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the above units is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit is implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it includes a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby achieving the functionality of some or all of the above units.

[0263] In other cases, the units in the device are configured as one or more processors (or processing circuits) that implement the above methods, such as: CPU, graphics processing unit (GPU), neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), MPU, digital signal processor (DSP), ASIC, FPGA, or a combination of at least two of these processor forms.

[0264] In other cases, all or part of the units in the device are integrated together or implemented independently. In one implementation, these units are integrated together as a system-on-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as including a CPU and an FPGA, or including a CPU and an MCU, or including a CPU and a GPU, etc.

[0265] Several possible devices are listed below.

[0266] Please see Figure 22 , Figure 22 This is a schematic diagram of a data acquisition and control device provided in an embodiment of this application. Optionally, the data acquisition and control device 102 can be a standalone device, such as a computing device, controller, or terminal. Alternatively, the data acquisition and control device 102 can be a component within a standalone device, such as a chip or integrated circuit. The data acquisition and control device 102 is used to implement the aforementioned method, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 , Figure 17 The operation performed by the acquisition and control device in the abnormal noise detection method shown.

[0267] The data acquisition and control device 102 includes an acquisition unit 1021 and a transmission unit 1022. The acquisition unit 1021 is used to acquire information, such as receiving or reading information, while the transmission unit 1022 is used to transmit data, such as reporting data. Optionally, it can also be used to implement other operations involved in the foregoing embodiments. It should be understood that the unit division and naming here are only illustrative; in specific implementations, some units may be combined together, or one unit may be split into multiple units.

[0268] In one possible implementation, the acquisition unit 1021 is used to acquire first audio data. The sending unit 1022 is used to send detection data to the server.

[0269] In another possible implementation, the acquisition control device 102 further includes a control unit 1023. The control unit 1023 is used to control other components to perform operations. For example, the control unit 1023 is used to control the first component to operate in a test mode.

[0270] In another possible implementation, the acquisition unit 1021 is further configured to acquire a first control command, and the control unit 1023 is configured to control a first component in the first vehicle to operate in a test mode based on the first control command.

[0271] In another possible implementation, the control unit 1023 is used to send a control signal to the first component based on a first control command, the control signal being used to control the first component to operate in a test mode.

[0272] In another possible implementation, the sending unit 1022 is further configured to send a second control command to the first vehicle based on the first control command.

[0273] In another possible implementation, the acquisition unit 1021 is also used to acquire test information from the abnormal noise detection device.

[0274] In another possible implementation, the acquisition control device 102 further includes an interaction unit 1024, which is used to output prompts to the user via the interaction device. For example, the interaction unit 1024 is used to prompt the user to place the audio acquisition device at a recommended acquisition location via the interaction device.

[0275] In another possible implementation, the acquisition unit 1021 is further configured to acquire indication information of the first component input by the user, and the sending unit 1022 is further configured to send an acquisition command to the audio acquisition device.

[0276] In another possible implementation, the acquisition unit 1021 is further configured to acquire detection requirement information input by the user, and the sending unit 1022 is further configured to send a collection command to the audio acquisition device in response to the detection requirement information.

[0277] In another possible implementation, the acquisition control device 102 further includes a processing unit 1025 for processing data. Exemplarily, the processing unit 1025 is used to analyze voice information to obtain user intent. The control unit 1023 is also used to control the audio acquisition device to acquire audio if the user intent is to provide feedback on the use of the first vehicle.

[0278] In another possible implementation, the acquisition unit 1021 is further configured to acquire information about the abnormal noise generation scenario acquired by the control device. The control unit 1023 is further configured to control the audio acquisition device to acquire audio in the abnormal noise generation scenario.

[0279] In another possible implementation, the abnormal noise occurs when the operating status data of the first vehicle meets the vehicle operating status conditions. The acquisition unit 1021 is further configured to acquire the operating status data of the first vehicle. The processing unit 1025 is further configured to determine, based on the operating status data of the first vehicle, whether the operating status data of the first vehicle meets the vehicle operating status conditions.

[0280] In another possible implementation, the abnormal noise generation scenario also includes the execution action of the first component. The sending unit 1022 is further configured to send a collection command to the audio acquisition device when the operating status data of the first vehicle meets the vehicle operating status conditions, wherein the first audio data includes the sound when the first component performs the aforementioned execution action.

[0281] In another possible implementation, the acquisition unit 1021 is further configured to receive abnormal noise description information input by the user, and the processing unit 1025 is further configured to determine information about the abnormal noise generation scenario based on the abnormal noise description information.

[0282] In another possible implementation, the acquisition unit 1021 is also used to acquire the abnormal noise detection results received by the control device from the server.

[0283] In another possible implementation, the interaction unit 1024 is also used to output abnormal noise detection prompt information through the interaction device, the abnormal noise detection prompt information being used to indicate the abnormal noise detection result.

[0284] Combination Figure 2 , Figure 2 This is a schematic diagram of the structure of an abnormal noise detection device provided in an embodiment of this application. The abnormal noise detection device 101 includes an information acquisition module 1011 (or simply acquisition module), a feature extraction module 1012, and an abnormal noise recognition module 1013. Optionally, it may also include one or more modules selected from an information sending module 1014, a test mode decision module 1015, or a collection location recommendation module 1016 (optional modules are shown in dashed lines). The feature extraction module 1012 and the abnormal noise recognition module 1013 can be considered as a processing module 1017.

[0285] The abnormal noise detection device 101 can implement the aforementioned method, for example... Figure 4 , Figure 10 , Figure 14 , Figure 17 The operation performed by the abnormal noise detection device in the abnormal noise detection method shown.

[0286] In one possible implementation, the information acquisition module 1011 is used to acquire detection data from the acquisition control device. The processing module 1017 is used to obtain the abnormal noise detection result of the first vehicle based on the detection data.

[0287] In another possible implementation, the feature extraction module 1012 is used to obtain sound feature information based on the first audio data, and the abnormal noise recognition module 1013 is used to obtain the abnormal noise detection result of the first vehicle based on the sound feature information.

[0288] In another possible implementation, the first audio data includes audio data from multiple channels. The feature extraction module 1012 is used to obtain audio data from at least one channel based on the first audio data, and to obtain sound feature information based on the audio data from at least one channel. The abnormal noise recognition module 1013 is used to obtain an abnormal noise detection result for the first vehicle based on the sound feature information.

[0289] In another possible implementation, the abnormal noise recognition module 1013 is used to obtain the abnormal noise detection result of the vehicle based on the sound feature information and the feature information of multiple abnormal noise audios, wherein the feature information of each audio is used to indicate the feature of at least one abnormal noise.

[0290] In another possible implementation, the abnormal noise recognition module 1013 is used to input sound feature information into the AI ​​model, and the output of the AI ​​model includes the abnormal noise detection result.

[0291] In another possible implementation, the detection data also includes the operating status data of the first vehicle. The processing module 1017 is used by the abnormal noise detection device to obtain the abnormal noise detection result of the first vehicle based on the first audio data and the operating status data of the first vehicle.

[0292] In another possible implementation, the information sending module 1014 is used to send test mode indication information to the acquisition control device.

[0293] In another possible implementation, the information acquisition module 1011 receives mode association information from the acquisition control device, and the test mode decision module 1015 is used to determine the test mode based on the mode management information. The mode association information includes one or more of the following: the identification information of the first vehicle or the indication information of the first component.

[0294] In another possible implementation, the information sending module 1014 is also used to send a recommended acquisition location to the acquisition control device.

[0295] In another possible implementation, the information acquisition module 1011 is further configured to receive indication information from the first component of the acquisition control device. The acquisition location recommendation module is configured to obtain a recommended acquisition location based on the indication information from the first component.

[0296] In another possible implementation, the information acquisition module 1011 is further configured to send the abnormal noise detection result of the first vehicle to the first vehicle and / or the first terminal. The first terminal is a terminal pre-configured for receiving the abnormal noise detection result of the first vehicle.

[0297] This application also provides an audio acquisition device, including a receiving unit, a reporting unit, and at least one microphone. The receiving unit is used to receive instructions, the at least one microphone is used to acquire sound in response to the instructions, and the transmitting unit is used to transmit data. The audio acquisition device is used to implement the method described in the third aspect or any possible implementation of the third aspect.

[0298] In one possible implementation, the receiving unit is used to receive a collection command from the acquisition control device. At least one microphone is used to collect sound from the operation of the first component in response to the collection command, thereby obtaining first audio data. The reporting unit is used to transmit the first audio data.

[0299] In another possible implementation, the reporting unit is used to send the first audio data to the acquisition control device, and the acquisition control device is used to provide the first audio data to the abnormal noise detection device.

[0300] Please see Figure 23 , Figure 23 This is a schematic diagram of another data acquisition and control device provided in this application embodiment. The data acquisition and control device 102 may include at least one processor, at least one memory, and a communication interface. Further optionally, the data acquisition and control device 102 also includes connection lines, wherein the processor, communication interface, and / or memory are connected via the connection lines, and / or communicate with each other via the connection lines to transmit control signals and / or data signals. Wherein: A processor is a module that performs arithmetic and / or logical operations, and may include one or more of the following modules: CPU, application processor (AP), MCU, ECU, GPU, MPU, ASIC, image signal processor (ISP), DSP, FPGA, complex programmable logic device (CPLD), or coprocessor, etc.

[0301] A communication interface can be used to provide information input or output to at least one processor, or to receive and / or send signals to externally transmitted signals. For example, a communication interface includes interface circuitry. Furthermore, a communication interface may include data transmission interfaces such as Ethernet interfaces, serial data interfaces, and parallel data interfaces, or it may include wireless link interfaces (Wi-Fi, Bluetooth, general wireless transmission, vehicular short-range communication technology, and other short-range wireless communication technologies).

[0302] As one possible design, the acquisition and control device 102 can be a chip or circuit, and the communication interface includes an input interface and an output interface, which can be the same interface or different interfaces. Optionally, the functionality of the communication interface can be implemented through a transceiver circuit or a dedicated transceiver chip.

[0303] The memory provides storage space, which can store data such as the operating system and computer programs. The memory can be one or a combination of several of the following: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM). The functions and operations of the modules or units in the acquisition and control device 102 listed above are merely illustrative examples.

[0304] Each functional unit in the acquisition and control device 102 can be used to implement the aforementioned method, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 , Figure 17 The operation performed by the acquisition and control device in the abnormal noise detection method shown.

[0305] Optionally, the processor in the acquisition control device 102 can be a processor specifically designed to execute the aforementioned methods (for ease of distinction, it can be referred to as a dedicated processor), or a processor that executes the aforementioned methods by calling a computer program (for ease of distinction, it can be referred to as a dedicated processor). Optionally, at least one processor may include both dedicated processors and general-purpose processors.

[0306] Optionally, if the computing device includes at least one memory, and the processor implements the aforementioned abnormal noise detection method by calling a computer program, the computer program can be stored in the memory.

[0307] Please see Figure 24 , Figure 24 This is a schematic diagram of another abnormal noise detection device provided in this application embodiment. The abnormal noise detection device 101 may include at least one processor, at least one memory, and a communication interface. Optionally, the abnormal noise detection device 101 may further include connection lines. For a description of the relevant components, please refer to the foregoing. Exemplarily, the abnormal noise detection device 101 is a server or a module within a server.

[0308] The functional units in the abnormal noise detection device 101 can be used to implement the aforementioned method, for example... Figure 4 , Figure 10 , Figure 14 , Figure 17 The abnormal noise detection method shown includes operations performed by the abnormal noise detection device.

[0309] Optionally, the processor in the abnormal noise detection device 101 can be a processor specifically designed to perform the aforementioned methods (for ease of distinction, it can be referred to as a dedicated processor), or a processor that performs the aforementioned methods by calling a computer program (for ease of distinction, it can be referred to as a dedicated processor). Optionally, at least one processor may include both dedicated processors and general-purpose processors.

[0310] Optionally, if the computing device includes at least one memory, and the processor implements the aforementioned abnormal noise detection method by calling a computer program, the computer program can be stored in the memory.

[0311] This application also provides an audio acquisition device, including a processor and a memory. The memory stores computer instructions, and the processor invokes the computer instructions to implement the aforementioned method, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 The operation performed by the acquisition and control device in the abnormal noise detection method shown.

[0312] This application also provides a chip including logic circuitry and a communication interface. The communication interface is used to receive and / or send information, or to input and / or output information, while the logic circuitry processes the information. This chip is used to implement the aforementioned methods, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 , Figure 17 The methods for detecting abnormal noises are shown.

[0313] This application also provides a computer-readable storage medium storing instructions that, when executed on at least one processor, implement the aforementioned method, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 , Figure 17 The methods for detecting abnormal noises are shown.

[0314] This application also provides a computer program product, which includes computer instructions for implementing the aforementioned method, for example... Figure 4 , Figure 5 , Figure 6 , Figure 10 , Figure 14 , Figure 17 The methods for detecting abnormal noises are shown.

[0315] This application also provides a terminal, which includes one or more of the following: a data acquisition and control device, an abnormal noise detection device, an audio acquisition device, a chip, an abnormal noise detection system, a computer-readable storage medium, or a computer program product, etc. Related products can be found in the foregoing description.

[0316] In addition, a few additional points need to be made regarding this application: I. The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the protection scope of the technical solutions of the embodiments of this application.

[0317] 2. Unless otherwise stated, “multiple” means two or more.

[0318] 3. Unless otherwise specified or in case of logical conflict, the terms and / or descriptions in different embodiments of this application are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0319] IV. The various numerical designations used in this application are merely for descriptive convenience and are not intended to limit the scope of protection of this application. Unless otherwise specified, the order of the serial numbers used in this application does not imply the sequence of execution; the execution order of each process should be determined by its function and internal logic. For example, the terms "first," "second," and other various terminology (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0320] Furthermore, any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0321] V. The terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such process, method, product, or apparatus. Furthermore, the modules indicated by dashed lines in this application are optional modules, and the data flows indicated by dashed lines represent optional data flows.

[0322] VI. Unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can mean A or B. In this application, "and / or" is merely a description of the relationship between the related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.

[0323] VII. Unless otherwise stated, the names of devices, equipment, modules, units, and other information in the embodiments of this application are merely examples, and devices, equipment, modules, and units are used to represent possible entities that implement a certain function, and the meanings of the four can be used interchangeably. For example, a data acquisition and control device, an abnormal noise detection device, and an audio acquisition device all have certain computing capabilities and can be called a computing device, a control device, or a device.

Claims

1. A rattling sound detection method characterized by comprising: The method comprises: obtaining first audio data from an audio acquisition device, the first audio data comprising sound of a first component in operation, the first component being a component in a first vehicle; sending detection data to an abnormal sound detection device, the detection data comprising the first audio data, the first audio data being used by the abnormal sound detection device to detect abnormal sound of the first component in operation.

2. The method of claim 1, wherein, The method further comprises: obtaining a first control instruction; based on the first control instruction, controlling the first component in the first vehicle to operate in a test mode, the first audio data comprising sound of the first component operating in the test mode.

3. The method of claim 2, wherein, The method is applied to the first vehicle, and the controlling the first component in the first vehicle to operate in the test mode based on the first control instruction comprises: based on the first control instruction, sending a control signal to the first component, the control signal being used to control the first component to operate in the test mode.

4. The method of claim 2, wherein, The method is applied to a terminal, the terminal being in communication connection with the first vehicle; The controlling the first component in the first vehicle to operate in the test mode based on the first control instruction comprises: based on the first control instruction, sending a second control instruction to the first vehicle, the second control instruction being used to instruct the first vehicle to control the first component to operate in the test mode.

5. The method according to any one of claims 2-4, characterized in that, The test mode indicates a number of times of performing an action by the first component and an action parameter at each time of performing the action; wherein the action parameter is used to indicate a type of the action, and / or the action of the first component has a stroke, and the action parameter is used to indicate the stroke of the action.

6. The method according to claims 2-5, characterized in that, The method further comprises: obtaining test information from the abnormal sound detection device, the test information comprising at least one of information of the test mode, the first control instruction, and indication information of a recommended acquisition position, the recommended acquisition position being used to indicate a placement position of the audio acquisition device.

7. The method of claim 6, wherein, The method further comprises: obtaining indication information of the first component input by a user; sending the indication information of the first component to the abnormal sound detection device, the recommended acquisition position being related to the first component.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: prompting, by an interaction device, to place the audio acquisition device at the recommended acquisition position.

9. The method of claim 1, wherein, Before the obtaining the first audio data, the method further comprises: obtaining detection requirement information input by a user; in response to the detection requirement information, sending an acquisition instruction to the audio acquisition device, the acquisition instruction being used to instruct the audio acquisition device to acquire the first audio data.

10. The method of claim 9, wherein, The detection requirement information is voice information, and the sending the acquisition instruction to the audio acquisition device in response to the detection requirement information comprises: analyzing the voice information to obtain a user intention; in a case where the user intention is to feed back a use problem of the first vehicle, sending the acquisition instruction to the audio acquisition device.

11. The method of claim 1, wherein, Before the obtaining the first audio data, the method further comprises: obtaining information of an abnormal sound generation scenario; in the abnormal sound generation scenario, sending an acquisition instruction to the audio acquisition device, the acquisition instruction being used to instruct the audio acquisition device to acquire the first audio data.

12. The method of claim 11, wherein, The abnormal sound generation scenario includes that the running state data of the first vehicle meets a vehicle running state condition, and the method further includes: obtaining running state data of the first vehicle; determining whether the running state data of the first vehicle meets a vehicle running state condition based on the running state data of the first vehicle.

13. The method of claim 12, wherein, The abnormal sound generation scenario further includes an execution action of the first component, The sending of the collection instruction to the audio collection device in the abnormal sound generation scenario includes: sending a collection instruction to the audio collection device in a case where the running state data of the first vehicle meets a vehicle running state condition, and the first audio data includes sound when the first component executes the execution action.

14. The method according to any one of claims 1 to 13, characterized in that, The method further includes: obtaining an abnormal sound detection result from the abnormal sound detection device, the abnormal sound detection result including an abnormal sound state, the abnormal sound state being used to indicate whether the first vehicle has an abnormal sound.

15. The method of claim 14, wherein, In a case where the abnormal sound state indicates that the first vehicle has an abnormal sound, the abnormal sound detection result further includes an abnormal sound cause, the abnormal sound cause including indication information of an abnormal sound component; and / or, In a case where the abnormal sound state indicates that the first vehicle has an abnormal sound, the abnormal sound detection result further includes a processing suggestion, the processing suggestion being used to prompt a manner of processing the abnormal sound.

16. The method according to any one of claims 1 to 15, characterized in that, The first component includes at least one of the following components: a vehicle cabin door, a window door, an air conditioning system, a front trunk door, a rear trunk door, a fuel tank cover, a charging port cover, a tool box door, a seat, a curtain, or a display device.

17. The method according to any one of claims 1 to 16, characterized in that, The detection data further includes information of a test case, the information of the test case being used to describe a collection environment of the first audio data; wherein the information of the test case includes at least one of the following: running state data of the first vehicle, identification information of the first vehicle, a batch of the first vehicle, a collection time of the first audio data, or a collection position of the first audio data.

18. An abnormal sound detection method characterized by comprising: The method includes: obtaining detection data from a collection control device, the detection data including first audio data, the first audio data including sound when a first component in a first vehicle is running; obtaining an abnormal sound detection result of the first vehicle based on the detection data, the abnormal sound detection result including an abnormal sound state, the abnormal sound state being used to indicate whether the first vehicle has an abnormal sound.

19. The method of claim 18, wherein, The obtaining of the abnormal sound detection result of the first vehicle based on the detection data includes: obtaining sound feature information based on the first audio data; obtaining the abnormal sound detection result of the first vehicle based on the sound feature information.

20. The method of claim 19, wherein, The first audio data includes audio data of multiple channels; The obtaining of the abnormal sound detection result of the first vehicle based on the detection data includes: obtaining audio data of at least one channel based on the first audio data, a collection position of the audio data of the at least one channel meeting a preset condition with respect to a distance to a sound source position; obtaining sound feature information based on the audio data of the at least one channel; obtaining the abnormal sound detection result of the first vehicle based on the sound feature information.

21. The method according to claim 19 or 20, characterized in that, The sound feature information comprises frequency information, amplitude information, and time-varying information of the frequency and amplitude.

22. The method of claim 21, wherein, The sound feature information comprises time-varying information of the frequency and amplitude, and the time-varying information of the frequency and amplitude comprises a color mapping diagram. The color mapping diagram comprises an image region formed by a first coordinate axis and a second coordinate axis. The first coordinate axis is time of sound, the second coordinate axis is frequency of sound, one unit length of the first coordinate axis and one unit length of the second coordinate axis form one pixel, and a pixel value in each pixel corresponds to an amplitude of sound.

23. The method according to any one of claims 19-22, characterized by, The sound feature information is used to obtain the abnormal sound detection result of the first vehicle. The sound feature information and feature information of a plurality of abnormal sound audios are used to obtain the abnormal sound detection result of the vehicle, and each feature information of the audio is used to indicate features of at least one abnormal sound.

24. The method according to any one of claims 19-23, characterized by, The sound feature information is used to obtain the abnormal sound detection result of the vehicle. The sound feature information is input into an artificial intelligence (AI) model, and an output of the AI model comprises the abnormal sound detection result, and the AI model is pre-trained.

25. The method according to any one of claims 18-24, characterized by, The detection data further comprises running state data of the first vehicle.

26. The method of any one of claims 18-25, wherein, The method further comprises: sending test information to the collection control device, wherein the test information comprises one or more of indication information of a test mode, a recommended collection position, or a first control instruction; wherein the recommended collection position is used to indicate a placement position of an audio collection device, and the first audio data is collected by the audio collection device; the first control instruction is used to instruct the collection control device to control a first component in the first vehicle to operate in a test mode.

27. The method of claim 26, wherein, The test information comprises a recommended collection position, and the method further comprises: obtaining indication information of the first component from the collection control device; based on the indication information of the first component, obtaining the recommended collection position.

28. The method of any one of claims 18-27, wherein, In a case where the abnormal sound state indicates that the first vehicle has an abnormal sound, the abnormal sound detection result further comprises an abnormal sound cause, and the abnormal sound cause comprises indication information of an abnormal sound component. and / or In a case where the abnormal sound state indicates that the first vehicle has an abnormal sound, the abnormal sound detection result further comprises a processing suggestion, and the processing suggestion is used to prompt a processing manner of the abnormal sound.

29. The method of any one of claims 18-26, wherein, The detection data further comprises information of a test case, and the information of the test case is used to describe a collection environment of the first audio data. The abnormal sound detection result further comprises at least part of the test case information.

30. The method of any one of claims 18-29, wherein, The method further comprises: based on the abnormal sound detection result of the first vehicle and an abnormal sound detection result of at least one second vehicle, obtaining a first batch of detection reports, wherein the first vehicle and the at least one vehicle belong to the first batch of vehicles, and the batch detection report is used to indicate whether the abnormal sound detection of the first batch of vehicles is qualified.

31. The method of any one of claims 18-30, wherein, The method further comprises: sending the abnormal sound detection result of the first vehicle to the first vehicle and / or a first terminal, and the first terminal is a terminal pre-set to receive the abnormal sound detection result of the first vehicle.

32. A rattling sound detection method characterized by comprising: The method is applied to an audio collection device, and comprises: receiving a collection instruction from a collection control device; in response to the collection instruction, collecting sound when a first component is running to obtain first audio data, the first component being a component in a first vehicle; sending the first audio data, the first audio data being used for an abnormal sound detection device to detect an abnormal sound when the first component is running.

33. The method of claim 32, wherein, The sending of the first audio data comprises: sending the first audio data to the collection control device, the collection control device being used to provide the first audio data to the abnormal sound detection device.

34. A harvesting control device, characterized by The collection control device comprises an acquisition unit and a sending unit, and is used to implement the method according to any one of claims 1-17.

35. An abnormal sound detecting apparatus characterized by comprising: The abnormal sound detection device comprises an information acquisition module and a processing module, and is used to implement the method according to any one of claims 18-31.

36. An audio acquisition device, comprising: The audio collection device comprises a receiving unit, a reporting unit and at least one microphone, and is used to implement the method according to claim 32 or 33.

37. A computing device comprising: The collection control device comprises a processor and a memory, the memory being used to store computer instructions, and the processor being used to invoke the computer instructions stored in the memory to enable the collection control device to implement the method according to any one of claims 1-17, or to implement the method according to any one of claims 18-31, or to implement the method according to claim 32 or 33.

38. A rattling sound detection system characterized by comprising: The abnormal sound detection system comprises an audio collection device, a collection control device and an abnormal sound detection device, The collection control device is used to implement the method according to any one of claims 1-17, The abnormal sound detection device is used to implement the method according to any one of claims 18-31, The audio collection device is used to implement the method according to claim 32 or 33.

39. A vehicle characterized by The vehicle comprises the collection control device according to claim 34, Alternatively, the vehicle comprises the collection control device according to claim 34 and the audio collection device according to claim 36, Alternatively, the vehicle comprises the collection control device according to claim 34 and the abnormal sound detection device according to claim 35, Alternatively, the vehicle comprises the abnormal sound detection system according to claim 38.

40. A computer program product comprising computer program instructions, wherein, The computer program instructions, when executed by a processor, enable a device comprising the processor to implement the method according to any one of claims 1-33.

Images