Vehicle-mounted refrigerator multi-surface automatic detection method, device and equipment and storage medium

Abstract

The invention relates to a multi-surface automatic detection method, device and equipment for a vehicle-mounted refrigerator and a storage medium, and the method comprises the steps: extracting image features of the vehicle-mounted refrigerator, and generating an image static detection result; applying vibration excitation for simulating a vehicle driving condition to the vehicle-mounted refrigerator, and synchronously collecting abnormal sound audio data and vibration feedback parameters; abnormal sound energy features corresponding to the abnormal sound audio data are extracted and input into the voiceprint classification model, an abnormal sound type recognition result is obtained, and the current vehicle working condition type is determined according to the vibration feedback parameters; inputting the abnormal sound type identification result, the vehicle working condition type and the image static detection result into a defect classification model to obtain a defect probability list of the vehicle-mounted refrigerator; and if any defect probability exceeds a preset defect threshold value, a detection abnormity report is generated, and the vehicle-mounted refrigerator is transferred to the abnormity to-be-processed area. According to the invention, comprehensive and accurate detection of the vehicle-mounted refrigerator under the static and dynamic working conditions is realized, the detection precision and efficiency are effectively improved, and the high reliability of the product quality is ensured.

Description

Technical Field

[0001] The present application relates to vehicle refrigerator technology, and in particular to a method, device, equipment and storage medium for automatic multi-faceted detection of a vehicle refrigerator. Background Art

[0002] In modern car refrigerator production, the correct installation of components is crucial for ensuring product quality and functional reliability. Due to the complex structure of car refrigerators, comprehensive inspection of multiple exterior surfaces is required to prevent defects such as missing or incorrect installation from being introduced into subsequent processes. Currently, the industry generally relies on manual visual inspection or automated equipment based on basic computer vision technology. However, these traditional methods have significant limitations in practical application.

[0003] First, manual visual inspection is highly dependent on the operator's experience and concentration. Operators work for long hours, especially in multi-faceted inspection scenarios, requiring repeated adjustments to product angles or multiple positioning. This further prolongs the time it takes to inspect a single piece, making it impossible to meet the efficiency requirements of high-rate production lines. Operators also easily become fatigued under high-intensity work, making it difficult to ensure inspection efficiency and consistency. Traditional rule-based visual inspection technology is extremely sensitive to environmental interference such as changes in lighting, metal reflections, or slight deformations of components. False detections and missed inspections are frequent, requiring frequent manual re-inspections and severely impacting production continuity. Existing technologies also lack a systematic mechanism for recording and tracing inspection data. Defect analysis relies on manual statistics, making it difficult to quickly locate the root cause of the problem and optimize the process.

[0004] Therefore, traditional inspection methods have bottlenecks in many aspects. As the demand for car refrigerator production changes, there is an urgent need for a method that can efficiently perform multi-faceted automatic inspection of car refrigerators to meet the needs of modern intelligent manufacturing. Summary of the Invention

[0005] The purpose of this application is to solve the above problems and provide a method for automatic multi-faceted detection of a vehicle-mounted refrigerator and its corresponding devices, equipment, non-volatile readable storage medium, and computer program product.

[0006] According to one aspect of the present application, a method for automatic multi-surface detection of a vehicle-mounted refrigerator is provided, comprising:

[0007] Acquire an outer surface image of the vehicle refrigerator, and extract image features of each outer surface image to generate an image static detection result;

[0008] Applying a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collecting corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation;

[0009] Extracting an abnormal sound energy feature corresponding to the abnormal sound audio data, and inputting the abnormal sound energy feature into a voiceprint classification model that has been pre-trained to a convergence state to obtain an abnormal sound type recognition result;

[0010] determining a current vehicle operating condition type based on the vibration feedback parameter, inputting the abnormal noise type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtaining a defect probability list for the vehicle refrigerator;

[0011] If any defect probability in the defect probability list exceeds a preset defect threshold, a detection abnormality report is generated according to the defect probability list, and the vehicle refrigerator is moved to an abnormality waiting area.

[0012] According to another aspect of the present application, there is provided a multi-surface automatic detection device for a vehicle-mounted refrigerator, comprising:

[0013] an image acquisition module configured to acquire an image of an outer surface of the vehicle refrigerator and extract image features of each outer surface image to generate an image static detection result;

[0014] an excitation applying module, configured to apply a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collect corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation;

[0015] an abnormal sound recognition module, configured to extract abnormal sound energy features corresponding to the abnormal sound audio data, and input the abnormal sound energy features into a voiceprint classification model pre-trained to a convergence state to obtain an abnormal sound type recognition result;

[0016] a defect detection module configured to determine a current vehicle operating condition type based on the vibration feedback parameter, input the abnormal noise type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtain a defect probability list of the vehicle refrigerator;

[0017] The abnormality handling module is configured to generate a detection abnormality report according to the defect probability list if any defect probability in the defect probability list exceeds a preset defect threshold, and move the vehicle refrigerator to an abnormality waiting area.

[0018] According to another aspect of the present application, a multi-faceted automatic detection device for a vehicle-mounted refrigerator is provided, comprising a six-degree-of-freedom vibration table, a visual camera, a multimodal sensor, an audio acquisition device, and a control unit. The six-degree-of-freedom vibration table is used to apply vibration excitation simulating a vehicle driving condition to the vehicle-mounted refrigerator. The visual camera and the multimodal sensor are used to acquire an image of the outer surface of the vehicle-mounted refrigerator and corresponding vibration feedback parameters of the vehicle-mounted refrigerator during the application of the vibration excitation. The audio acquisition device is used to acquire corresponding abnormal sound audio data of the vehicle-mounted refrigerator during the application of the vibration excitation. The control unit comprises a central processing unit and a memory. The central processing unit is used to call and run a computer program stored in the memory to execute the steps of the method described in the present application.

[0019] According to another aspect of the present application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the multi-faceted automatic detection method of the vehicle-mounted refrigerator in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the method are executed.

[0020] According to another aspect of the present application, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the method when executed by a processor.

[0021] The multi-faceted automatic inspection method for vehicle refrigerators proposed in this application has achieved significant technological progress and efficiency improvement compared to traditional manual visual inspection and basic visual inspection technologies for vehicle refrigerators. By combining high-precision visual inspection with abnormal noise analysis under vibration excitation, this method can simultaneously capture static defects caused by incorrect or missing assembly of parts and abnormal noise problems under dynamic operating conditions, significantly improving the detection coverage and accuracy. The pre-trained voiceprint classification model and defect classification model are used to analyze the abnormal noise energy characteristics, abnormal noise type recognition results, vehicle operating condition types and image static detection results from multiple sources to generate a corresponding defect probability list that can characterize the most likely defects in the vehicle refrigerator. This can effectively reduce the false alarm rate to an extremely low level while efficiently and comprehensively completing the multi-faceted automatic inspection of the vehicle refrigerator. Moreover, through the data recording of abnormality reports and the transfer mechanism of abnormality to be processed areas, the full-link traceability and closed-loop management of vehicle refrigerator defect problems can be achieved, significantly shortening the manual re-inspection time and optimizing the production process. The method can also be used to continuously improve the design and manufacturing of vehicle refrigerators, further improving the production efficiency and production quality of vehicle refrigerators. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of an embodiment of the method for automatic multi-surface detection of a vehicle refrigerator of the present application;

[0023] Figure 2This is a functional block diagram of the multi-surface automatic detection device for a vehicle refrigerator according to the present application;

[0024] Figure 3 This is a structural diagram of a multi-surface automatic detection device for a vehicle refrigerator used in this application. DETAILED DESCRIPTION

[0025] This application describes a vehicle refrigerator multi-side automatic inspection device that can be used to automatically inspect multiple surfaces of a vehicle refrigerator. The device primarily includes key components such as a six-degree-of-freedom vibration platform, a visual camera, a multimodal sensor, an audio acquisition device, and a control unit. These components collaborate through electrical connections and data transmission to complete the multi-side automatic inspection of the vehicle refrigerator.

[0026] The control unit serves as the brains of the entire device, coordinating the operation of various components through electrical connections. It determines the current vehicle operating condition based on vibration feedback parameters and inputs the abnormal noise type identification, operating condition type, and image static detection results into a pre-trained defect classification model to generate a defect probability list. If the detected defect probability exceeds a preset threshold, the control unit triggers the exception handling process, generates a detection anomaly report, and controls the slide transfer mechanism through electrical connections to move the vehicle refrigerator to the abnormal pending area. The anomaly report is also sent to the regional detection terminal.

[0027] The control unit serves as the brains of the entire inspection device, coordinating and controlling the operation of its various components. The control unit includes a central processing unit (CPU) and memory, which stores a computer program product containing code instructions for implementing the steps of the method for automatically inspecting multiple surfaces of a vehicle refrigerator. When the CPU calls and executes this computer program product, the control unit controls the vehicle refrigerator's automatic inspection device to begin capturing images of the refrigerator's exterior surface, thereby precisely controlling the device's subsequent inspection operations.

[0028] The six-degree-of-freedom vibration table is the source of vibration excitation. After receiving the vibration control instructions issued by the control unit, it applies multi-directional dynamic vibration excitation to the car refrigerator by simulating the complex movements of the vehicle during driving, such as acceleration, braking, steering and bumps, etc., accurately reproducing the mechanical shock and continuous vibration that may occur in the car refrigerator under real driving conditions. Its function is to trigger potential defects inside the refrigerator, such as loose screws and cracked brackets, which are exposed as detectable abnormal noise or vibration under dynamic working conditions. By simultaneously collecting vibration feedback parameters such as acceleration and frequency for analysis, the reliability of the car refrigerator in real usage scenarios is comprehensively evaluated, which makes up for the deficiency of traditional static detection in capturing dynamic defects.

[0029] The visual camera is responsible for capturing images of the outer surface of the car refrigerator and transmitting the image data to the control unit through an electrical connection for use in detecting static image detection results; the multimodal sensor is used to capture the vibration feedback parameters of the car refrigerator. By real-time monitoring of the dynamic parameters during the vibration excitation process of the six-degree-of-freedom vibration table, it can obtain vibration feedback parameters for analyzing the vehicle operating condition type and transmit the vibration feedback parameters to the control unit; the audio acquisition device is used to capture the abnormal sound audio of the car refrigerator in real time during the vibration excitation process, and also transmit the abnormal sound audio data to the control unit through an electrical connection. The central processing unit inside the control unit pre-processes the audio data to extract the corresponding abnormal sound energy characteristics, and then uses the pre-trained voiceprint classification model to identify the abnormal sound type of the captured abnormal sound audio of the car refrigerator.

[0030] When the multi-surface automatic inspection system for vehicle refrigerators is in operation, the control unit first controls the visual camera to capture an image of the vehicle refrigerator's exterior surface. Simultaneously, it transmits stored vibration excitation parameters to a six-degree-of-freedom vibration table. The six-degree-of-freedom vibration table applies vibration excitation to the vehicle refrigerator based on the received parameters and controls the multimodal sensor and audio acquisition device to collect corresponding vibration feedback parameters and abnormal sound audio data during the vibration excitation process. The control unit analyzes the acquired abnormal sound audio data using a voiceprint classification model to identify the abnormal sound type. Based on the vibration feedback parameters, the control unit determines whether the current vehicle operating condition is acceleration, braking, steering, or jolting. The corresponding vehicle operating condition type and abnormal sound type identification results, along with the image static inspection results, are then input into a defect classification model. The defect classification model then outputs the probability of each defect type in the vehicle refrigerator, obtains a corresponding defect probability list, and generates a detection anomaly report. Once the refrigerator's abnormality is accurately determined, the refrigerator is moved to the abnormality handling area. In this way, the multi-faceted automatic detection equipment for vehicle-mounted refrigerators of the present application can efficiently and accurately complete the multi-faceted automatic detection tasks for vehicle-mounted refrigerators, thereby ensuring the production quality and efficiency of vehicle-mounted refrigerators.

[0031] See also Figure 1 According to the present application, a method for automatically detecting multiple surfaces of a vehicle refrigerator can be implemented as a computer program product installed in a device for automatically detecting multiple surfaces of a vehicle refrigerator. In some embodiments, the method includes the following steps:

[0032] Step S1100: Acquire an outer surface image of the vehicle refrigerator, and extract image features of each outer surface image to generate an image static detection result.

[0033] During the multi-faceted automatic inspection process for vehicle refrigerators, comprehensive image capture of the refrigerator's exterior surface is first required. To achieve this, a visual camera, such as a CCD camera, mounted on a manipulator executes a preset multi-faceted capture program to capture images of the refrigerator's exterior surface. Using a freely movable manipulator mounted on the inspection station, the visual camera can capture images of the refrigerator's exterior surface from multiple angles. These images encompass the front, back, left, right, and top surfaces of the refrigerator, ensuring that all critical exterior surfaces of key components, such as screws, buckles, and door hinges, are covered. The visual camera can be triggered by a command from the control unit. For example, after the refrigerator is positioned at the inspection location, the control unit issues a detection command, and the visual camera then takes images based on the preset shooting angle and distance.

[0034] After acquiring the outer surface image of the vehicle refrigerator through a visual camera, the key components on the vehicle refrigerator can be located using a preset template matching algorithm, including visual information of key areas such as the shape outline of screws and the distribution of buckle textures. In the process of extracting image features from each outer surface image through visual information, the geometric features and brightness differences of key components are identified through grayscale contrast and edge detection technology, and a feature data set containing their position coordinates and matching values is generated. The feature data set corresponding to the key component is then compared with the pre-stored standard component template database for similarity. If the matching degree is lower than the preset threshold, such as the screw matching degree is less than 85% or the buckle brightness difference is greater than 15%, the vehicle refrigerator is judged to have a missing or mis-installed defect, and the defect type, defect location and defect confidence information are output accordingly to generate the image static detection result.

[0035] The static image inspection results generated by this automated visual inspection method can be used to quickly identify problems with the outer surface assembly of car refrigerators. This not only ensures that static defects are accurately intercepted at the early stage of the production line, preventing defective car refrigerators from flowing into subsequent processes, but also provides basic data support for subsequent dynamic inspections, further reducing random errors and efficiency bottlenecks in manual visual inspections.

[0036] Step S1200: applying a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collecting corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation.

[0037] To comprehensively evaluate the product's reliability under dynamic operating conditions, a six-degree-of-freedom vibration platform is used to simulate various driving conditions. Driven by a servo motor, the six-degree-of-freedom vibration platform generates multi-directional mechanical vibrations according to preset vibration control commands, accurately replicating the shock and sustained vibrations experienced in real-world driving environments, such as acceleration, braking, bumps, and cornering. This simulation ensures that the dynamic conditions experienced by the refrigerator during testing are highly consistent with those likely to be encountered in actual use.

[0038] While vibration excitation is being applied, a high-sensitivity acoustic sensor array can be used as an audio acquisition device to collect real-time audio data of abnormal noises generated by the car refrigerator. The high-sensitivity acoustic sensor can capture subtle sounds such as high-frequency clicking sounds and friction noises emitted by the car refrigerator, providing a rich data basis for subsequent acoustic analysis. At the same time, the three-axis acceleration sensor measures instantaneous acceleration in different directions in real time. The direction measured by the three-axis acceleration sensor can be set to the X / Y / Z three-axis direction, and combined with the preset spectrum analysis module to generate a vibration frequency distribution diagram. The vibration frequency distribution diagram is used to verify the consistency of the vibration excitation with the target working condition. For example, the vibration frequency under bumpy working conditions needs to be concentrated in 10Hz-15Hz. Vibration feedback parameters including instantaneous acceleration parameters, vibration frequency parameters and spectrum energy distribution parameters are thus obtained to verify whether the vibration excitation meets the preset vehicle working conditions. For example, under braking conditions, the peak acceleration of the Z axis needs to reach 8m / s. 2 , providing corresponding information for analyzing defect triggering conditions.

[0039] In one embodiment, in order to ensure the accurate correlation between the abnormal sound generated by the car refrigerator and the vibration working condition, all collected abnormal sound audio data and vibration feedback parameters can be aligned through a unified timestamp. This time synchronization mechanism ensures the timing matching of the abnormal sound audio data and vibration feedback parameters, and further accurately identifies and locates the defects of the car refrigerator.

[0040] Therefore, by collecting the corresponding abnormal sound audio data and vibration feedback parameters of the car refrigerator during the application of vibration excitation, hidden defects that cannot be captured by static detection can be exposed through dynamic vibration excitation, such as loose screws or micro-cracks in the bracket. These defects may not be easy to be found in static detection, but will cause abnormal noise or vibration under dynamic conditions. Therefore, by quantifying the correlation between abnormal sound audio data and vibration feedback parameters, not only can the misjudgment rate be reduced, but the detection efficiency can also be significantly improved, providing solid data support for the reliability verification of car refrigerators in real usage scenarios, ensuring that only products that perform reliably under various working conditions can pass the inspection, and at the same time meeting the dual requirements of vehicle refrigerator detection efficiency and detection accuracy.

[0041] Step S1300: extract the abnormal sound energy feature corresponding to the abnormal sound audio data, and input the abnormal sound energy feature into a voiceprint classification model that has been pre-trained to a convergence state to obtain an abnormal sound type recognition result.

[0042] In this embodiment, after obtaining the abnormal sound audio data of the car refrigerator, in order to further obtain the abnormal sound situation of the car refrigerator contained in the abnormal sound audio data, the abnormal sound audio data needs to be further processed. First, the environmental noise in the abnormal sound audio data is removed by bandpass filtering and frame processing, and the corresponding abnormal sound audio signal after removing the environmental noise is divided into equal time windows, so as to pre-process the abnormal sound audio data to ensure the accuracy of the subsequent abnormal sound analysis and avoid the environmental noise from covering up the real abnormal sound characteristics of the car refrigerator. Subsequently, the abnormal sound audio signal of each frame after the segmentation is subjected to wavelet packet decomposition, and the energy proportion of the preset frequency band is extracted as the abnormal sound energy feature. Since the typical defective sounds emitted by the car refrigerator, such as the high-frequency abnormal sound or friction noise similar to the "click" sound, have a more significant energy distribution in the 5kHz-8kHz frequency band, the extraction frequency band of the abnormal sound energy feature is usually set to 5kHz-8kHz.

[0043] After acquiring the abnormal noise energy signature, it is input into a pre-trained voiceprint classification model that has been converged. This voiceprint classification model is constructed based on a convolutional neural network and a Mel spectrum feature extraction layer. It undergoes supervised training until the model converges using a large number of historical abnormal noise samples, which are labeled with the defect type. This allows the voiceprint classification model to learn the characteristic representations of different abnormal noise types and accurately classify the input abnormal noise energy signature. The voiceprint classification model outputs a corresponding abnormal noise type identification result, which includes the identified car refrigerator abnormal noise type and the corresponding confidence level, such as "high-frequency abnormal noise, probability 92%" or "friction noise, probability 85%." This voiceprint classification model not only provides the abnormal noise type but also the corresponding confidence level, thereby improving the objectivity and interpretability of car refrigerator defect determination.

[0044] In this way, the correlation between the frequency band of the abnormal sound energy characteristics and the acoustic mode can be quantified, and the audio signal during the dynamic vibration process of the vehicle refrigerator can be converted into a classifiable defect feature. Therefore, through the linkage analysis with the static image detection results and vibration feedback parameters, the hidden defects caused by loose screws and micro-cracks in the bracket can be further accurately identified. The comprehensive analysis of this multimodal data fills the blind spots of traditional visual inspection and provides key data support for the comprehensive quality assessment of vehicle refrigerators. At the same time, the probabilistic output of the voiceprint classification model not only improves the accuracy of vehicle refrigerator detection, but also enhances the interpretability of the detection results, making the detection and control process of vehicle refrigerators more transparent and reliable.

[0045] Step S1400: Determine the current vehicle operating condition type based on the vibration feedback parameters, input the abnormal sound type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtain a defect probability list of the vehicle refrigerator.

[0046] In this embodiment, when determining the vehicle operating condition type, the vibration feedback parameters, such as the instantaneous acceleration of the X / Y / Z axes, the vibration frequency and the spectrum energy distribution, are matched with the preset vehicle operating condition mapping table. For example, when the peak acceleration of the Z axis reaches 8m / s 2 , and when the spectrum energy is concentrated in 10Hz-15Hz, it is judged as a bumpy operating condition; or when the X-axis acceleration suddenly increases and the frequency energy distribution conforms to the braking characteristics, it is classified as a braking operating condition to ensure accurate identification of the vehicle operating condition type and provide a reliable operating condition background for subsequent defect analysis.

[0047] After obtaining the abnormal noise type identification results, vehicle operating condition type, and static image detection results (e.g., 92% probability for high-frequency abnormal noise and 70% confidence for bumpy operating conditions and screw leaks), this data is encoded into a joint feature vector. This joint feature vector integrates static visual defects, dynamic abnormal noise characteristics, and vibration operating conditions to comprehensively reflect the status of the vehicle refrigerator under different detection dimensions. The joint feature vector is then input into a defect classification model that has been pre-trained to convergence. This defect classification model can be constructed based on a random forest or deep neural network. It is supervised and trained with a large number of historical multimodal datasets containing defect type labels until the model is trained to convergence. This allows the defect classification model to learn the complex correlations between different feature combinations and defect types, thereby accurately analyzing the input joint feature vector.

[0048] After inputting the joint feature vector into the defect classification model, the model outputs a list of defect probabilities corresponding to the defect types included in the vehicle refrigerator, such as "loose door hinge screw, probability 85%; cracked compressor bracket, probability 12%." This defect classification model not only provides specific defect types but also assigns corresponding confidence levels, clarifying the priority of defect handling. This probabilistic list output not only improves vehicle refrigerator inspection accuracy but also intuitively enhances the interpretability of inspection results. This approach enables the multifaceted automated inspection process for vehicle refrigerators to quantify the causal relationship between multimodal data—combining abnormal noise type identification, vehicle operating conditions, and static image inspection results—to accurately pinpoint the root causes of both explicit and implicit defects in the vehicle refrigerator, effectively avoiding the limitations of a single inspection method. Furthermore, the probabilistic output guides maintenance personnel to quickly locate problems, improving inspection efficiency and interpretability, providing a solid foundation for production line quality control.

[0049] Step S1500: If any defect probability in the defect probability list exceeds a preset defect threshold, a detection abnormality report is generated according to the defect probability list, and the vehicle refrigerator is moved to an abnormality waiting area.

[0050] In this embodiment, when the probability of any defect in the defect probability list exceeds a preset threshold, such as the probability of the door hinge screw loosening is greater than 85%, or the probability of the compressor bracket crack is greater than 75%, the detection abnormality report generation process is triggered accordingly. The detection abnormality report integrates the defect type, confidence probability, abnormal sound energy characteristics (such as 72% of the energy in the 5kHz frequency band), vibration feedback parameters (such as the peak value of the Z-axis acceleration under bumpy conditions of 8m / s) corresponding to the vehicle refrigerator. 2 ) and static image detection results (e.g., screw leak detection with 70% confidence), then generates a structured data packet using an encryption algorithm. This data is then bound to the refrigerator's RFID tag or barcode, ensuring data traceability and security. Simultaneously, the control unit sends a sorting instruction to a pre-set slide transfer mechanism, which drives it to precisely move the refrigerator to the abnormality handling area.

[0051] In one embodiment, the transfer process can monitor the transfer path in real time through photoelectric sensors and positioning modules to ensure that the product reaches the target location accurately and avoids collisions with other products. When the transfer is completed, the control unit pushes the detection abnormality report to the maintenance terminal and MES system through the industrial bus protocol Profinet, and updates the status of the corresponding vehicle refrigerator to "pending re-inspection". As a result, the defect type and processing priority of the vehicle refrigerator are highlighted in the MES dashboard, such as "Loose door hinge screws, urgency: high", to provide intuitive guidance for maintenance personnel. Maintenance personnel can retrieve complete detection data by scanning the product barcode. Since the detection data includes abnormal sound spectrum, vibration waveform and defect location coordinates, maintenance personnel can quickly locate the problem and perform targeted abnormality processing.

[0052] Through the above-mentioned processing method, the multi-faceted automatic inspection equipment for vehicle refrigerators can realize the immediate interception and accurate traceability of defective products, preventing defective products from flowing into subsequent processes. The structured detection anomaly report integrates multimodal detection data, providing comprehensive information support for the abnormal handling of vehicle refrigerators, significantly reducing the manual investigation time; at the same time, through the deep integration of the control unit and the MES system, the transparency of the entire process of vehicle refrigerator quality control can be further ensured, and the security and auditability of data are guaranteed through encryption and status identification, thereby achieving the goal of high-precision and high-efficiency vehicle refrigerator inspection, providing stable protection for production line quality control.

[0053] It is not difficult to understand from the above embodiments that the present application has achieved significant beneficial effects compared to the traditional vehicle refrigerator outer surface detection method, including but not limited to:

[0054] Compared with traditional manual visual inspection or single visual inspection technology, the multi-faceted automatic inspection method for vehicle refrigerators proposed in this application achieves a comprehensive improvement in multi-dimensional defect detection and intelligent decision-making by integrating multimodal data including abnormal sound type recognition results, vehicle operating condition types, and image static detection results with intelligent model analysis. By taking images of the outer surface of the vehicle refrigerator with a visual camera, the corresponding static assembly defect problems of the outer surface can be obtained; by accurately simulating the vehicle driving conditions through a six-degree-of-freedom vibration table, and combining a high-sensitivity acoustic sensor array with a three-axis acceleration sensor to synchronously collect abnormal sound audio data and vibration feedback parameters, it can dynamically expose hidden defects such as loose screws and micro-cracks in brackets that are difficult to capture through static inspection; and by using pre-trained voiceprint classification models and defect classification models, the abnormal sound energy characteristics, vibration operating condition types, and image static detection results are integrated and analyzed to generate a probabilistic defect probability list. This can not only accurately identify explicit defects such as incorrect or missing assembly of parts, but also combine static visual inspection with dynamic vibration excitation analysis to reduce the false alarm rate to an extremely low level. At the same time, this application further quantifies the corresponding defect priority. When the probability of any defect exceeds the preset threshold, the abnormal report generation is automatically triggered, and the vehicle refrigerator with defects is transferred to the abnormal waiting area to prevent defective products from flowing into subsequent processes. This significantly improves the automation level and reliability of multi-faceted inspection of vehicle refrigerators, so as to further ensure the reliability of vehicle refrigerators in real usage scenarios, reduce the cost of manual re-inspection, and meet the needs of modern vehicle refrigerator intelligent manufacturing for high-precision and high-efficiency production.

[0055] Based on any embodiment of the method of the present application, before the step of extracting the image features of each of the outer surface images, the method includes:

[0056] Step S1001: perform illumination normalization processing on the outer surface image, separate the illumination component image and the reflection component image of the outer surface image by a preset component analysis algorithm, and retain the reflection component image as the basic image.

[0057] Since the outer surface images of the vehicle refrigerator acquired by the visual camera may contain factors that are not defects due to reasons such as label reflection and process design, the outer surface images contain factors that are not defects. Therefore, before the step of extracting the image features of each outer surface image of the vehicle refrigerator, in order to improve the accuracy and robustness of the image static detection results of the outer surface images, the acquired outer surface images are first subjected to illumination normalization processing. Based on the illumination normalization processing, the image deviation of the outer surface images of the vehicle refrigerator caused by changes in ambient lighting conditions is eliminated to ensure the accuracy of subsequent image feature extraction.

[0058] In this embodiment, the Retinex algorithm is used as a component analysis algorithm to decompose each pixel value of the external surface image into an illumination component image and a reflection component image, wherein the illumination component image mainly contains slowly changing brightness information, while the reflection component image reflects the real texture and structural information of the surface of the vehicle refrigerator. Specifically, the external surface image is first subjected to multi-scale Gaussian filtering to extract the illumination component image. The kernel size of the Gaussian filter is adjusted according to the resolution of the external surface image and the characteristics of the illumination change to ensure that the filtered external surface image can accurately reflect the illumination information. Furthermore, a subtraction operation is performed on the filtered illumination component image and the original external surface image, that is, the reflection component image can be extracted from the original external surface image to separate it. The reflection component image thus obtained is used as the basic image, which can better reflect the real texture of the external surface of the vehicle refrigerator and reduce the impact of illumination changes on the detection results.

[0059] Step S1002: performing adaptive histogram equalization processing on the basic image to generate an outer surface image after illumination normalization.

[0060] After obtaining the reflective component image through the component resolution algorithm, adaptive histogram equalization (AHE) is performed on it. This process adjusts the image's grayscale distribution to enhance detail clarity and improve the accuracy of subsequent feature extraction. The AHE image is first divided into multiple local regions, each of which is sized based on the resolution and feature detail of the AHE image. Furthermore, a grayscale histogram is calculated for each local region, and a cumulative distribution function (CDF) is calculated based on this histogram. The CDF is then used to remap the pixel values within the local region, enhancing contrast within that region. During this process, smooth transitions between adjacent regions are applied to avoid noticeable boundary artifacts. Furthermore, a clipping constraint parameter is introduced to constrain the grayscale distribution to suppress noise amplification. Consequently, adaptive histogram equalization further optimizes the illumination normalization of the AHE image. The resulting illumination-normalized exterior surface image not only preserves the structural information of the original image but also significantly enhances detail visibility, enabling accurate identification of potential defects on the exterior surface of vehicle refrigerators even under complex lighting conditions.

[0061] Through the above-mentioned illumination normalization processing, the outer surface image of the car refrigerator can maintain consistent feature expression under different lighting conditions, ensuring the accuracy and reliability of static image detection. This not only improves the robustness of outer surface image detection, but also provides a high-quality basic image for subsequent feature extraction and defect recognition, so that potential defects on the outer surface of the car refrigerator can be accurately identified even in complex lighting environments.

[0062] Based on any embodiment of the method of the present application, extracting image features of each of the outer surface images includes:

[0063] Step 2100: Perform image block processing on each of the illumination-normalized outer surface images, and calculate the first eigenvector and the second eigenvector of each image block respectively.

[0064] In this embodiment, the exterior surface image is divided into multiple equal-sized image blocks. The size of each image block can be adjusted based on the resolution and feature detail of the exterior surface image, typically set to 16×16 or 32×32 pixels. This block processing helps capture local features of the image and improves feature extraction accuracy. The first eigenvector and second eigenvector are calculated for each image block. The first eigenvector is a histogram of oriented gradients eigenvector, and the second eigenvector is a local binary pattern texture eigenvector.

[0065] The directional gradient histogram feature vector captures the edge and contour information of the image by calculating the direction and amplitude of the pixel intensity change in the image block. The specific steps are: calculate the gradient amplitude and direction of the image block, divide the direction interval into multiple bins, for example, divide 0°-180° into 9 bins, count the sum of the gradient amplitudes in each bin to form a histogram, and normalize the histogram to finally obtain the directional gradient histogram feature vector.

[0066] The local binary pattern texture feature vector captures the texture information of the image by analyzing the grayscale relationship between the pixels in the image block and their neighboring pixels. The specific steps are: taking each pixel as the center, take the pixels in its neighborhood, such as a 3×3 neighborhood, compare the grayscale value of the central pixel with the grayscale value of the neighboring pixels, generate a binary pattern, count the frequency of occurrence of different binary patterns to form a histogram, and normalize the histogram to finally obtain the local binary pattern texture feature vector.

[0067] By calculating the directional gradient histogram feature vector and local binary pattern texture feature vector of each image block separately, the edge, contour and texture information of the image can be fully captured, providing rich feature descriptions for subsequent feature fusion and defect recognition. This multi-feature combination method can improve the expressiveness of features and enhance the accuracy and robustness of detection.

[0068] Step S2200: Concatenate the first eigenvector and the second eigenvector into a joint eigenvector in a preset order, and perform normalization processing on the joint eigenvector to obtain a feature descriptor representing illumination robustness.

[0069] After extracting the first eigenvector and the second eigenvector of each image block, in order to generate a feature descriptor that can fully characterize the illumination robustness of the image features, the two eigenvectors need to be concatenated and normalized.

[0070] In this embodiment, the directional gradient histogram feature vector and the local binary pattern texture feature vector are first spliced in a preset order. Specifically, the directional gradient histogram feature vector is placed in the front part of the joint feature vector, and the local binary pattern texture feature vector is placed in the back part. By setting this order, the advantages of the directional gradient histogram feature vector in capturing image edge and contour information and the local binary pattern texture feature vector in describing texture details can be fully utilized. Although the joint feature vector obtained after splicing contains rich image feature information, the scales of different features therein may be inconsistent. In order to eliminate the scale differences between different features and improve the stability and comparability of the feature descriptor, it is necessary to normalize the joint feature vector. The normalization method usually adopts L2 norm normalization, that is, each element in the joint feature vector is divided by the L2 norm of the vector, so that the normalized vector has unit length, ensuring the robustness of the feature descriptor under different lighting conditions.

[0071] Therefore, after further normalization processing, the obtained feature descriptor can accurately characterize the illumination robustness characteristics of the outer surface image of the car refrigerator. This feature descriptor not only integrates edge, contour and texture information, but also enhances the adaptability to illumination changes through normalization, providing a high-quality feature foundation for subsequent feature matching and defect recognition.

[0072] Step S2300: Match the feature descriptor with the feature descriptor corresponding to the preset standard vehicle refrigerator template. If the obtained feature matching degree is lower than the set threshold, it is determined that the corresponding outer surface of the vehicle refrigerator has an abnormal condition.

[0073] When detecting abnormalities on the exterior surface of a vehicle refrigerator, feature descriptors generated from the exterior surface image are matched with feature descriptors corresponding to a preset standard vehicle refrigerator template. This can further assist in determining whether the exterior surface of the vehicle refrigerator has any abnormalities. The feature descriptors for the standard vehicle refrigerator template are obtained by applying the same feature extraction process to images of a normal, defect-free vehicle refrigerator exterior surface, including illumination normalization, image segmentation, feature vector calculation, and concatenation and normalization of the joint feature vectors. The matching process can use similarity metrics such as dynamic time warping or Euclidean distance to calculate the similarity between the feature descriptors of the vehicle refrigerator under inspection and those of the standard template. By comparing the similarities between the feature descriptors, the degree of difference between the two can be quantified. If the resulting feature matching degree falls below a set threshold, for example, a matching degree below 85%, the corresponding exterior surface of the vehicle refrigerator is determined to have an abnormality. Such abnormalities may include defects such as scratches, dents, loose screws, or missing parts.

[0074] To ensure accurate and robust matching, the matching process typically combines feature descriptors from multiple image patches for comprehensive evaluation. This allows for a comprehensive analysis of every area of the vehicle refrigerator's exterior surface, ensuring that no potential defects are missed. This feature matching approach not only improves detection accuracy but also enhances the interpretability of the results, providing a reliable basis for quality control of vehicle refrigerators.

[0075] According to this embodiment, it can be seen that through block processing and feature extraction, the local features of the outer surface image of the vehicle refrigerator can be captured, thereby improving the level of detection detail; by splicing and normalizing different feature vectors, a feature descriptor with lighting robustness can be generated, thereby enhancing the stability and adaptability of the features; by matching with the standard template feature descriptor, it is possible to accurately determine whether there are abnormal conditions on the outer surface, thereby improving the accuracy and reliability of detection, and not only improving the expression ability of the image features of the outer surface image, but also enhancing the interpretability of the detection results, thereby providing strong technical support for the quality control of vehicle refrigerators.

[0076] Based on any embodiment of the method of the present application, applying a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collecting corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation, including:

[0077] Step S3100: Generate corresponding vibration control instructions based on the vehicle operating condition type corresponding to the vibration excitation by using a preset six-degree-of-freedom vibration table. The vibration control instructions include acceleration value instructions, vibration frequency instructions, and action time instructions for the six-degree-of-freedom vibration table in different directions. The vehicle operating condition types include acceleration conditions, braking conditions, steering conditions, and bumpy motion conditions.

[0078] The control unit generates corresponding vibration control instructions based on the preset vehicle operating condition type, which define the acceleration value, vibration frequency, and action time of the six-degree-of-freedom vibration table in different directions through the vibration control instructions. The vibration control instructions include the acceleration value instructions, vibration frequency instructions, and action time instructions implemented by the six-degree-of-freedom vibration table in different directions. Among them, the acceleration value instruction can be set to control the target acceleration of the six-degree-of-freedom vibration table in the X / Y / Z axis direction. For example, under the bumpy working condition, the peak acceleration of the Z axis is set to 8m / s 2 , used to simulate the mechanical impact intensity of different actions during vehicle driving; the vibration frequency instruction can be set to set the target frequency of the six-degree-of-freedom vibration table's periodic vibration to correspond to the vehicle's motion rhythm under different working conditions; the action time instruction can be set to specify the duration for the six-degree-of-freedom vibration table to maintain a specific acceleration and frequency, ensuring that the dynamic excitation is consistent with the time characteristics of the actual driving scenario, and verifying the stability of the product under continuous vibration.

[0079] Vibration control instructions can be adjusted accordingly to suit different working conditions. For example, for acceleration conditions, vibration control instructions may include applying a gradually increasing acceleration in the X-axis direction to simulate the dynamic changes when the vehicle accelerates; for braking conditions, a gradually decreasing acceleration can be applied in the X-axis direction to simulate the dynamic changes when the vehicle decelerates or brakes; for steering conditions, a certain acceleration can be applied in the Y-axis or Z-axis direction to simulate the centrifugal force when the vehicle turns; and for bumpy motion conditions, random accelerations and vibration frequencies may be applied in multiple directions to simulate the bumps of the vehicle driving on uneven roads.

[0080] Therefore, the vibration control instructions are sent to the six-degree-of-freedom vibration table through the control unit. The six-degree-of-freedom vibration table adjusts the corresponding parameters of the vibration excitation it applies according to the received vibration control instructions to accurately simulate the corresponding vehicle operating conditions. In this way, the vehicle refrigerator can experience dynamic conditions similar to those in actual use during the detection process, thereby effectively exposing potential dynamic defects and ensuring the accuracy and reliability of the vehicle refrigerator detection results.

[0081] Step S3200: Execute acceleration, braking, steering, and bumping actions respectively according to the vibration control instruction, and collect the abnormal sound audio data and the vibration feedback parameters during the execution of the vibration control instruction.

[0082] The types of vehicle operating conditions corresponding to the vibration excitation applied by the six-degree-of-freedom console to the car refrigerator include but are not limited to acceleration conditions, braking conditions, steering conditions and bumpy conditions. The corresponding operations can be adjusted according to actual needs. When vibration excitation needs to be applied, the control unit generates the corresponding vibration control instruction and sends it to the six-degree-of-freedom vibration table, which then applies the corresponding vibration excitation to the car refrigerator.

[0083] The six-degree-of-freedom vibration table receives vibration control instructions from the control unit. These instructions define in detail the motion parameters under different vehicle operating conditions, including but not limited to acceleration values, vibration frequency, and action time. It adjusts its motion parameters according to the corresponding vibration control instructions to accurately simulate the corresponding vehicle dynamics. During execution, a high-sensitivity acoustic sensor array can collect abnormal noise audio data generated by the vehicle refrigerator in real time to capture subtle sounds, such as high-frequency abnormal noise or friction noise. At the same time, the three-axis acceleration sensor measures the instantaneous acceleration in different directions and combines it with the spectrum analysis module to generate a vibration frequency distribution map to obtain vibration feedback parameters, including instantaneous acceleration, vibration frequency, and spectral energy distribution. Among them, the spectrum analysis module is a data processing module integrated into the control unit. After collecting the time-domain vibration signal of the three-axis acceleration sensor, such as the X / Y / Z-axis acceleration waveform, it uses fast Fourier transform to convert the time-domain vibration signal into a frequency-domain vibration signal, thereby generating a vibration frequency distribution map. Therefore, when the sensor outputs an analog vibration signal in real time, the analog vibration signal is filtered and amplified by the signal conditioning circuit and converted into a digital signal by the data acquisition card. The spectrum analysis module then performs a fast Fourier transform operation on the digital signal to calculate the energy proportion of each frequency component. For example, the energy proportion of the 10Hz-15Hz frequency band is 70%; and the correspondence between frequency and amplitude is displayed in the form of a spectrum diagram to verify whether the vibration excitation matches the preset working conditions. At the same time, abnormal frequency bands are identified. For example, the frequency band of the bumpy working condition needs to be concentrated in the 10Hz-15Hz range. If a 50Hz frequency band appears, it is considered that there is power frequency interference, and it is removed accordingly to optimize the detection parameters.

[0084] After acquiring the abnormal noise audio data and vibration feedback parameters, they are aligned using a unified timestamp to ensure they accurately correspond to the specific vibration excitation phase. This synchronization mechanism is crucial for accurately identifying and locating potential defects, allowing for more precise determination of hidden issues in vehicle refrigerators under dynamic operating conditions, providing more rigorous and reliable data support for subsequent defect analysis.

[0085] By simulating various vehicle operating conditions in this embodiment, the performance of the vehicle refrigerator in a dynamic environment can be comprehensively evaluated, and potential defects can be effectively discovered. Precise vibration control instructions ensure the consistency and repeatability of test conditions, improving the accuracy and efficiency of detection. The simultaneous collection of abnormal sound audio data and vibration feedback parameters provides rich information for analysis, which helps to accurately locate the root cause of the problem, thereby enhancing the quality control of the vehicle refrigerator and ensuring the reliability and stability of the vehicle refrigerator in actual use.

[0086] Based on any embodiment of the method of the present application, extracting the abnormal sound energy feature corresponding to the abnormal sound audio data, and inputting the abnormal sound energy feature into a voiceprint classification model pre-trained to a convergent state to obtain an abnormal sound type recognition result, including:

[0087] Step S4100: performing wavelet packet decomposition on the abnormal sound audio data, extracting the energy proportion of a preset frequency band, and obtaining the abnormal sound energy feature based on the energy proportion;

[0088] First, the collected abnormal sound audio data is decomposed through wavelet packets. The appropriate wavelet function and decomposition level are selected to decompose the audio signal into multiple frequency sub-bands. By calculating the energy percentage within each sub-band, the energy distribution of different frequency bands can be quantified. Because the 5kHz-8kHz frequency band usually contains the acoustic characteristics of typical defects such as high-frequency abnormal noise and friction noise, when processing the abnormal noise audio data, a bandpass filter is used to remove low-frequency environmental noise and high-frequency interference signals, retaining the effective audio components of 1kHz-10kHz. The filtered signal is then framed according to a fixed time window of 50ms to ensure that each frame of audio contains complete abnormal noise event characteristics. Next, wavelet packet decomposition is performed on each frame signal. The Daubechies wavelet basis function and 5 decomposition levels are selected to decompose the signal step by step into sub-signals of different frequency bands. Finally, the sub-node corresponding to the preset 5kHz-8kHz frequency band is located. The proportion of the signal energy in this frequency band to the total energy of the entire frequency band is calculated to generate a normalized abnormal noise energy ratio. For example, the energy of a certain frame signal in the 5kHz-8kHz frequency band accounts for 72%. The abnormal noise energy characteristics are obtained based on the eigenvector corresponding to this abnormal noise energy ratio.

[0089] The abnormal noise energy feature characterizes the energy concentration of the abnormal noise audio data in the key frequency band. For example, high-frequency abnormal noises are effectively distinguished because their energy is significantly concentrated in this frequency band. Finally, the feature vectors of each frame are integrated in time series as the input of the voiceprint classification model, thereby achieving accurate correlation between the abnormal noise type represented by the abnormal noise audio data and the defect type of the car refrigerator.

[0090] Step S4200: Input the abnormal sound energy feature into the voiceprint classification model to obtain the abnormal sound type identification result, wherein the voiceprint classification model is constructed based on Mel spectrum and convolutional neural network, and the abnormal sound type output by the voiceprint classification model is any one or more of the preset high-frequency abnormal sound, low-frequency abnormal sound, and friction noise;

[0091] To input the abnormal sound energy features into the voiceprint classification model, the energy percentage data of the 5kHz-8kHz frequency band extracted by wavelet packet decomposition must first be integrated with the Mel-spectrogram features in the voiceprint classification model. First, the abnormal sound energy features are subjected to Mel-spectrogram analysis. After pre-emphasis, framing, and windowing, the short-time Fourier transform of each frame's feature signal is calculated to obtain a spectrogram. The linear frequency scale is then converted to a Mel-scale using 40 Mel-filter banks to generate a Mel-spectrogram, which reflects the acoustic characteristics perceived by the human ear. This Mel-spectrogram is then input into a pre-trained convolutional neural network model. The convolutional neural network model extracts spatial features from the Mel-spectrogram and learns the characteristic representations of different abnormal sound types. The Softmax function then outputs a preset probability distribution of abnormal sound types, which can be set to high-frequency abnormal sound, low-frequency abnormal sound, and friction noise. The voiceprint classification model uses a cross-entropy loss function and the Adam optimizer during training. End-to-end training is performed until convergence based on a historical audio dataset labeled with defect types, such as loose screw samples labeled "high-frequency abnormal noise." This ensures that the model can learn the mapping between abnormal noise types and vehicle refrigerator defect types from the fused features. Ultimately, the model outputs the corresponding abnormal noise type identification result, such as "high-frequency abnormal noise, probability 92%; friction noise, probability 7%." The specific abnormal noise type can be determined by setting a probability threshold, such as a single-type probability >80% or a cumulative probability of multiple types >90%, providing a quantitative basis for subsequent defect correlation analysis.

[0092] This embodiment uses wavelet packet decomposition to extract the energy proportion of preset frequency bands in abnormal sound audio data, which can accurately capture key abnormal sound characteristics such as high-frequency abnormal sounds, low-frequency abnormal sounds and friction noise, and can effectively filter out environmental noise interference; combined with the voiceprint classification model based on Mel spectrum and convolutional neural network, it can further realize the efficient identification of abnormal sound types, and the corresponding output of abnormal sound type identification results containing probability can also intuitively improve the accuracy and interpretability of detection.

[0093] Based on any embodiment of the method of the present application, the current vehicle operating condition type is determined according to the vibration feedback parameter, the abnormal sound type recognition result, the vehicle operating condition type, and the image static detection result are input into a defect classification model pre-trained to a convergence state, and a defect probability list of the vehicle refrigerator is obtained, including:

[0094] Step S5100: Matching a preset vehicle operating condition type mapping table according to the vibration feedback parameters to determine whether the current vehicle operating condition type is any one of an acceleration condition, a braking condition, a steering condition, and a bumping motion condition.

[0095] In this embodiment, the vibration feedback parameters include the instantaneous acceleration, vibration frequency and spectral energy distribution of the X / Y / Z axes. These vibration feedback parameters are compared with the characteristic ranges in the preset mapping table containing different vehicle operating conditions. For example, when the peak acceleration of the Z axis reaches 8m / s 2 If the spectrum energy is concentrated between 10Hz and 15Hz, the current vehicle operating condition is matched to a bumpy one. If the X-axis acceleration suddenly increases and the frequency distribution matches braking characteristics, it is classified as a braking condition. This matching mechanism accurately identifies the current operating condition, ensuring that subsequent abnormal noise analysis and defect detection are based on a more accurate vehicle operating condition context, thereby improving the accuracy and reliability of test results.

[0096] Step S5200: Input the vehicle operating condition type, the abnormal sound type identification result, and the image static detection result of the corresponding vehicle refrigerator into the defect classification model constructed based on the decision tree algorithm, and the defect classification model obtains the probability of the defect type contained in the vehicle refrigerator through the decision tree rule to obtain the defect probability list according to the defect type probability.

[0097] In this embodiment, the vehicle operating condition types including acceleration, braking, steering and bumps are encoded as one-hot vectors, the abnormal sound type recognition results including high-frequency abnormal sounds, low-frequency abnormal sounds and friction noises are converted into corresponding probability vectors, and the image static detection results including missing screws and missing buckles are also encoded as probability vectors. These vectors together constitute the input feature vector.

[0098] The defect classification model is based on a decision tree algorithm and is trained through supervised learning using a historical multimodal dataset containing defect type labels. During the defect classification model learning process, in order to reduce the computational complexity of the model, a decision tree is used to branch according to the importance of the features. For example, if the abnormal noise type is high-frequency and the vehicle operating condition is bumpy, the branch points to the possible defect type, and each leaf node stores the probability distribution of the corresponding defect type. The complexity of the defect classification model obtained by directly outputting the defect type through training the decision tree can be significantly lower than that of the model obtained based on GNN training, avoiding the efficiency reduction caused by excessive calculation time.

[0099] During the actual detection calculation process, after the input feature vector is input into the defect classification model, the input feature vector reaches the corresponding leaf node through the branching rules of the defect classification model's decision tree, thereby obtaining the defect type probability and ultimately outputting a corresponding defect probability list. The defect probability list contains probability values for various preset defect types, such as "loose door hinge screws, probability 85%" or "compressor bracket crack, probability 12%." This decision tree-based classification method can effectively associate multi-source data with defect types, provide probabilistic defect assessment, enhance the interpretability of detection results, and provide a reliable basis for quality control of car refrigerators.

[0100] By matching vibration feedback parameters with a preset vehicle operating condition mapping table, the implementation method in this example can quickly and accurately identify the current vehicle operating condition, providing a precise operating context for subsequent abnormal noise analysis. By integrating the vehicle operating condition type, abnormal noise type identification results, and image static detection results into a defect classification model based on a decision tree algorithm, the correlation between multi-source data can be systematically analyzed. By learning the mapping relationship between features and defect types in historical data, a detailed list of defect type probabilities can be accurately output. This comprehensive analysis method not only improves the accuracy of detection results, but also enhances the interpretability of the test results, reduces the need for manual re-inspection, and improves the automation and reliability of vehicle refrigerator detection.

[0101] Based on any embodiment of the method of the present application, if any defect probability in the defect probability list exceeds a preset defect threshold, a detection abnormality report is generated according to the defect probability list, and the vehicle refrigerator is moved to an abnormality waiting area, including:

[0102] Step S6100: If the probability of any defect type in the defect type list exceeds the defect threshold, the defect type that exceeds the defect threshold is marked accordingly, and the detection anomaly report is generated according to the defect type, wherein the detection anomaly report includes the defect type, the defect type probability, the abnormal sound audio data, the vibration feedback parameter and the preset detection identification code of the vehicle refrigerator, and the detection identification code is a unique identifier to distinguish each independent vehicle refrigerator.

[0103] During the multi-faceted automatic inspection process of vehicle-mounted refrigerators, when the probability of any defect type in the defect type list exceeds the preset defect threshold, the defect type will be marked accordingly and a corresponding detection anomaly report will be generated. Specifically, the system identifies the defect types that exceed the threshold, such as the probability of loose door hinge screws exceeding 85% or the probability of cracks in the compressor bracket exceeding 75%, and generates structured detection anomaly reports for these defect types that exceed the threshold.

[0104] The detection anomaly report integrates multi-source data, including the specific defect type, the corresponding confidence probability, the collected abnormal sound audio data, vibration feedback parameters, and the preset detection identification code of the car refrigerator. Among them, the defect type and its probability value are extracted from the defect classification model, such as "the door hinge screw is loose, with a probability of 87%", and the corresponding abnormal sound audio data, such as the audio waveform with 75% of the energy in the 5kHz-8kHz frequency band, and vibration feedback parameters, such as the peak Z-axis acceleration of 8m / s under bumpy conditions. 2 The system displays a spectrum diagram and static image inspection results, such as the coordinate information of a screw leak device with a 70% confidence level. The preset inspection identification code uniquely identifies each individual vehicle refrigerator and is typically bound to the refrigerator's RFID tag, barcode, or QR code to ensure data traceability. During the binding process, this data is combined with the refrigerator's unique inspection identification code, and a structured inspection anomaly report is generated using the AES-256 encryption algorithm to ensure data integrity and security. The inspection identification code is then written to the refrigerator's physical tag using a scanner or RFID reader / writer, uniquely linking the product and data.

[0105] Step S6200, control the preset slide transfer mechanism to transfer the vehicle refrigerator to the abnormal waiting area, and send the detection abnormality report to the area detection terminal corresponding to the abnormal waiting area, so that the area detection terminal marks the vehicle refrigerator as an abnormal waiting state based on the abnormal detection report.

[0106] When the control unit receives an abnormality detection report, it sends a sorting instruction to the sliding transfer mechanism based on the information in the abnormality detection report. The sliding transfer mechanism is usually composed of an automated conveyor device. It can move the vehicle refrigerator along a predetermined path to the abnormality processing area according to the sorting instruction. During the transfer process, the transfer path is monitored in real time by photoelectric sensors and positioning modules to ensure that the vehicle refrigerator reaches the target location accurately and avoid collision or confusion with other products.

[0107] After the vehicle refrigerator is transferred, the control unit sends the detection anomaly report to the regional detection terminal corresponding to the abnormal pending area through the Profinet industrial bus protocol. After receiving the detection anomaly report, the regional detection terminal will mark the vehicle refrigerator as abnormal pending state according to the content of the detection anomaly report and update its status information in the MES system. In the MES dashboard, the defect type and processing priority of the vehicle refrigerator will be highlighted to provide intuitive guidance for maintenance personnel. Maintenance personnel can scan the barcode or RFID tag of the vehicle refrigerator to retrieve complete detection data, including abnormal sound spectrum, vibration waveform and defect location coordinates, so as to quickly locate the problem and perform targeted repairs, ensuring the immediate interception and accurate tracing of defective products, preventing defective products from flowing into subsequent processes, and at the same time, through real-time status synchronization, greatly shortening the abnormal response time and improving the efficiency of production line quality control.

[0108] Through the automated inspection process in this embodiment, defects in vehicle refrigerators can be quickly identified, corresponding inspection anomaly reports can be generated, inspection efficiency and accuracy can be improved, and defective products can be automatically sorted through the slide transfer mechanism to ensure accurate interception and reduce manual intervention. Therefore, real-time data management and status updates can also enhance production transparency and facilitate rapid response to anomalies; the unique inspection identification code can ensure data traceability, optimize quality control, reduce the risk of defective vehicle refrigerators entering the market, and further improve overall production efficiency and product quality.

[0109] See also Figure 2According to one aspect of the present application, a multi-faceted automatic detection device for a vehicle refrigerator is provided, comprising an image acquisition module 7100, an excitation application module 7200, an abnormal sound recognition module 7300, a defect detection module 7400, and an abnormality handling module 7500. The image acquisition module 7100 is configured to acquire an image of the outer surface of the vehicle refrigerator and extract image features of each of the outer surface images to generate an image static detection result; the excitation application module 7200 is configured to apply a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator and synchronously collect the corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation; the abnormal sound recognition module 7300 is configured to In order to extract the abnormal sound energy characteristics corresponding to the abnormal sound audio data, the abnormal sound energy characteristics are input into the voiceprint classification model pre-trained to a convergence state to obtain the abnormal sound type identification result; the defect detection module 7400 is configured to determine the current vehicle operating condition type according to the vibration feedback parameter, and input the abnormal sound type identification result, the vehicle operating condition type and the image static detection result into the defect classification model pre-trained to a convergence state to obtain a defect probability list of the vehicle refrigerator; the exception handling module 7500 is configured to generate a detection exception report according to the defect probability list if any defect probability in the defect probability list exceeds a preset defect threshold, and move the vehicle refrigerator to an abnormality waiting area.

[0110] Based on any embodiment of the device of the present application, the device of the present application further includes:

[0111] An illumination normalization module is configured to perform illumination normalization processing on the outer surface image, separate the illumination component image and the reflection component image of the outer surface image through a preset component analysis algorithm, and retain the reflection component image as the basic image; an equalization processing module is configured to perform adaptive histogram equalization processing on the basic image to generate an outer surface image after illumination normalization.

[0112] Based on any embodiment of the device of the present application, the image acquisition module 7100 includes: an image segmentation module, configured to perform image segmentation processing on each of the external surface images after illumination normalization, and calculate the first eigenvector and the second eigenvector of each image block respectively; a descriptor generation module, configured to splice the first eigenvector and the second eigenvector into a joint feature vector in a preset order, and normalize the joint feature vector to obtain a feature descriptor characterizing illumination robustness; a descriptor matching module, configured to match the feature descriptor with the feature descriptor corresponding to the preset standard vehicle refrigerator template. If the obtained feature matching degree is lower than the set threshold, it is determined that there is an abnormal condition on the corresponding external surface of the vehicle refrigerator.

[0113] Based on any embodiment of the device of the present application, the excitation application module 7200 includes: a vibration instruction generation module, which is configured to generate a corresponding vibration control instruction through a preset six-degree-of-freedom vibration table according to the vehicle operating condition type corresponding to the vibration excitation, and the vibration control instruction includes the acceleration value instruction, vibration frequency instruction and action time instruction of the six-degree-of-freedom vibration table in different directions, and the vehicle operating condition type includes acceleration condition, braking condition, steering condition and bumping action condition; a vibration execution module, which is configured to execute acceleration, braking, steering and bumping actions respectively according to the vibration control instruction, and collect the abnormal sound audio data and the vibration feedback parameters during the execution of the vibration control instruction.

[0114] Based on any embodiment of the device of the present application, the abnormal sound identification module 7300 includes: an abnormal sound feature acquisition module, configured to perform wavelet packet decomposition on the abnormal sound audio data, extract the energy proportion of a preset frequency band, and obtain the abnormal sound energy feature based on the energy proportion; an abnormal sound type identification module, configured to input the abnormal sound energy feature into the voiceprint classification model to obtain the abnormal sound type identification result, wherein the voiceprint classification model is constructed based on Mel spectrum and convolutional neural network, and the abnormal sound type output by it is any one or more abnormal sound types of preset high-frequency abnormal sound, low-frequency abnormal sound and friction noise.

[0115] Based on any embodiment of the device of the present application, the defect detection module 7400 includes: a working condition determination module, which is configured to match a preset vehicle working condition type mapping table according to the vibration feedback parameters to determine whether the current vehicle working condition type is any one of an acceleration condition, a braking condition, a steering condition and a bumpy action condition; a defect probability acquisition module, which is configured to input the vehicle working condition type, the abnormal sound type recognition result and the image static detection result of the corresponding vehicle refrigerator into the defect classification model constructed based on the decision tree algorithm, and the defect classification model obtains the defect type probability contained in the vehicle refrigerator through the decision tree rule to obtain the defect probability list according to the defect type probability.

[0116] Based on any embodiment of the device of the present application, the exception handling module 7500 includes: a report generation module, which is configured to mark the defect type that exceeds the defect threshold if the probability of any defect type in the defect type list exceeds the defect threshold, and generate the detection exception report according to the defect type, wherein the detection exception report includes the defect type, the defect type probability, the abnormal sound audio data, the vibration feedback parameter and the preset detection identification code of the vehicle refrigerator, and the detection identification code is a unique identifier to distinguish each independent vehicle refrigerator; a product transfer module, which is configured to control the preset slide transfer mechanism to transfer the vehicle refrigerator to the abnormal waiting area, and send the detection exception report to the regional detection terminal corresponding to the abnormal waiting area, so that the regional detection terminal marks the vehicle refrigerator as an abnormal waiting state based on the abnormal detection report.

[0117] Another embodiment of the present application also provides a multi-faceted automatic detection device for a vehicle refrigerator. Figure 3 Figure 2 shows the internal structure of a vehicle-mounted refrigerator multi-surface automatic detection device. The device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The non-volatile computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may contain information sequences. When executed by the processor, the computer-readable instructions enable the processor to implement a method for automatically detecting multiple surfaces of a vehicle-mounted refrigerator.

[0118] The processor of this vehicle-mounted refrigerator multi-surface automatic detection device is used to provide computing and control capabilities, supporting the operation of the entire vehicle-mounted refrigerator multi-surface automatic detection device. The memory of this vehicle-mounted refrigerator multi-surface automatic detection device may store computer-readable instructions. When executed by the processor, these computer-readable instructions cause the processor to perform the vehicle-mounted refrigerator multi-surface automatic detection method of this application. The network interface of this vehicle-mounted refrigerator multi-surface automatic detection device is used to connect and communicate with a terminal.

[0119] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the multi-sided automatic detection device for vehicle-mounted refrigerators to which the solution of the present application is applied. The specific multi-sided automatic detection device for vehicle-mounted refrigerators may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In this embodiment, the processor is used to execute Figure 2The memory stores the program code and various data required to execute the modules or submodules described above. The network interface is used to facilitate data transmission between user terminals or servers. The non-volatile, readable storage medium in this embodiment stores the program code and data required to execute all modules in the multi-surface automatic detection device for vehicle-mounted refrigerators of this application. The server can call upon the server's program code and data to execute the functions of all modules.

[0121] The present application also provides a non-volatile readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the multi-faceted automatic detection method for a vehicle refrigerator of any embodiment of the present application.

[0122] The present application also provides a computer program product, comprising a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] In summary, the present application has realized a complete set of multi-faceted automatic detection technology solutions for vehicle-mounted refrigerators, which significantly improves the accuracy and efficiency of vehicle-mounted refrigerator quality inspection, can eliminate the influence of ambient light changes on image detection, enhance the stability and accuracy of image feature extraction, further optimize image quality, and make tiny defects more clearly visible in the image. At the same time, it can not only obtain static image detection results, but also expose dynamic defects that are difficult to detect with static detection. In addition, the present application accurately identifies high-frequency and low-frequency abnormal sound types by quantifying abnormal sound features, providing strong support for defect location. Through the fusion analysis of multimodal data, it not only improves the reliability of the detection results, but also enhances the interpretability of the detection results. Finally, through a complete and automated exception handling process, it realizes the immediate interception and accurate tracing of defective products, reduces manual intervention, and improves production efficiency. Through the application of the technical solution of the present application, manufacturers can effectively avoid the limitations of a single detection method, greatly shorten the abnormal response time, and provide a high-precision and high-efficiency quality control solution for the intelligent manufacturing of vehicle-mounted refrigerators, ensuring the high quality and high reliability of the products.

Claims

1. A multi-faceted automatic detection method for a vehicle refrigerator, characterized in that: include: Acquire an outer surface image of the vehicle refrigerator, and extract image features of each outer surface image to generate an image static detection result; Applying a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collecting corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation; Extracting an abnormal sound energy feature corresponding to the abnormal sound audio data, and inputting the abnormal sound energy feature into a voiceprint classification model that has been pre-trained to a convergence state to obtain an abnormal sound type recognition result; determining a current vehicle operating condition type based on the vibration feedback parameter, inputting the abnormal noise type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtaining a defect probability list for the vehicle refrigerator; If any defect probability in the defect probability list exceeds a preset defect threshold, a detection abnormality report is generated according to the defect probability list, and the vehicle refrigerator is moved to an abnormality waiting area.

2. The multi-surface automatic detection method for a vehicle refrigerator according to claim 1, characterized in that: Before the step of extracting the image features of each of the outer surface images, the method further includes: performing illumination normalization processing on the outer surface image, separating the illumination component image and the reflection component image of the outer surface image by a preset component analysis algorithm, and retaining the reflection component image as the base image; Adaptive histogram equalization is performed on the basic image to generate an outer surface image after illumination normalization.

3. The multi-surface automatic detection method for a vehicle refrigerator according to claim 2, characterized in that: The extracting of image features of each of the outer surface images comprises: Performing image block processing on each of the illumination-normalized outer surface images, and respectively calculating a first eigenvector and a second eigenvector of each image block; splicing the first eigenvector and the second eigenvector into a joint eigenvector in a preset order, and normalizing the joint eigenvector to obtain a feature descriptor representing illumination robustness; The feature descriptor is matched with the feature descriptor corresponding to the preset standard vehicle refrigerator template. If the obtained feature matching degree is lower than a set threshold, it is determined that the corresponding outer surface of the vehicle refrigerator has an abnormal condition.

4. The multi-surface automatic detection method for a vehicle refrigerator according to claim 1, characterized in that: The step of applying a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator and synchronously collecting corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation comprises: According to the vehicle operating condition type corresponding to the vibration excitation, a corresponding vibration control instruction is generated by a preset six-degree-of-freedom vibration table, wherein the vibration control instruction includes acceleration value instructions, vibration frequency instructions, and action time instructions of the six-degree-of-freedom vibration table in different directions, and the vehicle operating condition type includes acceleration condition, braking condition, steering condition, and bumping action condition; Acceleration, braking, steering and bumping actions are respectively performed according to the vibration control instruction, and the abnormal sound audio data and the vibration feedback parameters are collected during the execution of the vibration control instruction.

5. The multi-surface automatic detection method for a vehicle refrigerator according to any one of claims 1 to 4, characterized in that: The extracting the abnormal sound energy feature corresponding to the abnormal sound audio data, and inputting the abnormal sound energy feature into a voiceprint classification model pre-trained to a convergent state to obtain an abnormal sound type recognition result, includes: Performing wavelet packet decomposition on the abnormal sound audio data, extracting the energy proportion of a preset frequency band, and obtaining the abnormal sound energy feature according to the energy proportion; The abnormal sound energy feature is input into the voiceprint classification model to obtain the abnormal sound type identification result, wherein the voiceprint classification model is constructed based on Mel spectrum and convolutional neural network, and the abnormal sound type output by it is any one or more of the preset high-frequency abnormal sound, low-frequency abnormal sound and friction noise.

6. The multi-surface automatic detection method for a vehicle refrigerator according to any one of claim 5, characterized in that: The determining of the current vehicle operating condition type based on the vibration feedback parameter, inputting the abnormal sound type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtaining a defect probability list of the vehicle refrigerator includes: Matching a preset vehicle operating condition type mapping table according to the vibration feedback parameters to determine whether the current vehicle operating condition type is any one of an acceleration condition, a braking condition, a steering condition, and a bumping motion condition; The vehicle operating condition type, the abnormal sound type identification result, and the image static detection result of the corresponding vehicle refrigerator are input into the defect classification model constructed based on the decision tree algorithm. The defect classification model obtains the probability of the defect type contained in the vehicle refrigerator through the decision tree rule, so as to obtain the defect probability list according to the defect type probability.

7. The multi-surface automatic detection method for a vehicle refrigerator according to any one of claim 6, characterized in that: If any defect probability in the defect probability list exceeds a preset defect threshold, generating a detection abnormality report according to the defect probability list and moving the vehicle refrigerator to an abnormality pending processing area includes: If the probability of any defect type in the defect type list exceeds the defect threshold, the defect type that exceeds the defect threshold is marked accordingly, and the detection abnormality report is generated according to the defect type, wherein the detection abnormality report includes the defect type, the defect type probability, the abnormal sound audio data, the vibration feedback parameter and the preset detection identification code of the vehicle refrigerator, and the detection identification code is a unique identifier to distinguish each independent vehicle refrigerator; Control the preset slide transfer mechanism to transfer the vehicle refrigerator to the abnormal pending area, and send the detection abnormality report to the area detection terminal corresponding to the abnormal pending area, so that the area detection terminal marks the vehicle refrigerator as an abnormal pending state based on the abnormal detection report.

8. A multi-surface automatic detection device for a vehicle refrigerator, characterized in that: include: an image acquisition module configured to acquire an image of an outer surface of the vehicle refrigerator and extract image features of each outer surface image to generate an image static detection result; an excitation applying module, configured to apply a vibration excitation simulating a vehicle driving condition to the vehicle refrigerator, and synchronously collect corresponding abnormal sound audio data and vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation; an abnormal sound recognition module, configured to extract abnormal sound energy features corresponding to the abnormal sound audio data, and input the abnormal sound energy features into a voiceprint classification model pre-trained to a convergence state to obtain an abnormal sound type recognition result; a defect detection module configured to determine a current vehicle operating condition type based on the vibration feedback parameter, input the abnormal noise type recognition result, the vehicle operating condition type, and the image static detection result into a defect classification model pre-trained to a convergence state, and obtain a defect probability list of the vehicle refrigerator; The abnormality handling module is configured to generate a detection abnormality report according to the defect probability list if any defect probability in the defect probability list exceeds a preset defect threshold, and move the vehicle refrigerator to an abnormality waiting area.

9. A multi-faceted automatic detection device for a vehicle refrigerator, characterized in that: The vehicle refrigerator comprises a six-degree-of-freedom vibration platform, a visual camera, a multimodal sensor, an audio acquisition device, and a control unit. The six-degree-of-freedom vibration platform is used to apply vibration excitation simulating a vehicle driving condition to the vehicle refrigerator. The visual camera and the multimodal sensor are used to acquire an image of the outer surface of the vehicle refrigerator and corresponding vibration feedback parameters of the vehicle refrigerator during the application of the vibration excitation. The audio acquisition device is used to acquire corresponding abnormal sound audio data of the vehicle refrigerator during the application of the vibration excitation. The control unit comprises a central processing unit and a memory. The central processing unit is used to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7, so as to realize multi-faceted automatic detection of the vehicle refrigerator.

10. A non-volatile readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images