New energy automobile part surface defect testing method based on computer vision

Abstract

The invention relates to the field of new energy automobile part detection, in particular to a new energy automobile part surface defect testing method based on computer vision, and the method comprises the steps: carrying out the multi-source data collection and preprocessing, dynamically adjusting a light source, carrying out the feature fusion through a deep learning technology, and automatically extracting the multi-level features of defects. And in combination with transfer learning and reinforcement learning, intelligent defect classification, positioning and severity evaluation are realized. And through dynamic threshold adjustment and manual verification, the detection accuracy is ensured. And finally, predicting a defect trend based on historical data, analyzing reasons, and preventing and optimizing production in advance. According to the method, the detection efficiency and precision are remarkably improved, the system intelligence and adaptability are enhanced, and production optimization and quality control are promoted.

Description

Technical Field

[0001] The present application relates to the field of new energy vehicle parts detection, and in particular to a surface defect testing method for new energy vehicle parts based on computer vision. Background Art

[0002] New energy vehicle parts, such as motor core inspection, need to be inspected for appearance defects after production. Currently, conventional appearance inspection generally uses a three-axis appearance inspection instrument. By shining a specific light source on the surface of the manufactured part to be tested, a high-definition high-speed camera is used to obtain the surface image of the manufactured part. Image processing technology is used to extract image features, and then classification technology is used to detect and classify surface defects.

[0003] However, for some new energy vehicle accessories with complex structures, the detection accuracy is low. Summary of the invention

[0004] In order to achieve rapid detection of surface defects of new energy vehicle accessories and improve detection efficiency, the present application provides a surface defect testing method for new energy vehicle accessories based on computer vision.

[0005] In a first aspect, the present application provides a method for testing surface defects of new energy vehicle parts based on computer vision, comprising the following steps: Collect accessory surface information from multiple data sources including high-definition images, 3D point cloud data, and thermal imaging, and then perform preprocessing, including data cleaning, format conversion, and preliminary feature extraction; Dynamically adjust the direction and intensity of the light source according to the surface material and shape of the accessory, and apply image enhancement technology; Using a combination of deep convolutional neural networks and recurrent neural networks, feature fusion is performed on multimodal data to automatically learn and extract multi-level features of surface defects, including texture, shape, location, and time series information; Based on the multi-level features of the extracted surface defects, the model is trained using transfer learning and reinforcement learning to intelligently classify and accurately locate surface defects, while also assessing the severity of the defects. According to the real-time detection results and changes in the surface characteristics of the parts, the threshold of defect detection is dynamically adjusted, and a manual verification mechanism is introduced to review the detection results of the model; Based on historical inspection data, time series analysis and prediction models are used to predict potential surface defect trends, take preventive measures in advance, and analyze the causes of defects.

[0006] Optionally, the acquisition of accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, and thermal imaging, and then preprocessing, including data cleaning, format conversion, and preliminary feature extraction, comprises the following steps: Determine the type of data to be collected and configure the corresponding sensors and collection equipment; Ensure that multimodal data are synchronized in time and perform preliminary data fusion; Perform data cleaning, remove noise, convert data formats, and prepare input for deep learning models; Use the preprocessed data to preliminarily extract surface features and provide basic features for subsequent deep learning models.

[0007] Optionally, dynamically adjusting the direction and intensity of the light source according to the surface material and shape of the accessory and applying image enhancement technology comprises the following steps: Analyze the surface material and shape of the accessories to determine the impact of lighting conditions on the test results; Based on the analysis results, the direction and intensity of the light source are dynamically adjusted to reduce shadows and reflections; Apply image enhancement technology to improve image quality; Evaluate the effects of lighting adjustment and image enhancement to ensure that the image quality meets the detection requirements.

[0008] Optionally, the method utilizes a combination of a deep convolutional neural network and a recurrent neural network to perform feature fusion on multimodal data, automatically learn and extract multi-level features of surface defects, including texture, shape, position and time series information, including the following steps: Design a combined model of deep convolutional neural network and recurrent neural network for feature fusion of multimodal data; Use labeled datasets to train models and optimize model performance by adjusting hyperparameters; Using the trained model, feature fusion is performed on multimodal data to extract multi-level features of surface defects; Verify the effectiveness of the extracted features, adjust the model parameters as needed, and optimize the feature extraction effect.

[0009] Optionally, based on the extracted multi-level features of the surface defects, using transfer learning and reinforcement learning, training the model to intelligently classify and accurately locate the surface defects, and assess the severity of the defects, including the following steps: Use pre-trained models for transfer learning to optimize the model's adaptability to new data; Design reinforcement learning strategies to optimize the model’s classification and location of surface defects; Using the trained model, surface defects can be intelligently classified and accurately located; Evaluate the accuracy of classification and localization, and optimize model performance through feedback mechanisms.

[0010] Optionally, dynamically adjusting the threshold of defect detection according to the real-time detection results and changes in the surface characteristics of the parts, and introducing a manual verification mechanism to review the detection results of the model, includes the following steps: Analyze real-time inspection results to identify changes in the surface characteristics of accessories; According to the analysis results, the threshold of defect detection is dynamically adjusted to adapt to the changes in the surface characteristics of the parts; Introduce a manual verification mechanism to review the model's test results to ensure accuracy; Feedback the manual verification results to the model to improve the accuracy and reliability of detection through continuous optimization.

[0011] Optionally, the method of predicting potential surface defect trends based on historical inspection data using time series analysis and prediction models, taking preventive measures in advance, and analyzing the causes of defects includes the following steps: Collect historical test data and preprocess them in preparation for time series analysis; Use time series analysis methods to predict potential surface defect trends; Analyze the causes of defects and identify potential problems in the production process; Based on the analysis results, optimize the production process and take preventive measures.

[0012] In a second aspect, the present application provides a new energy automobile parts surface defect testing system based on computer vision, which is used to execute the above-mentioned new energy automobile parts surface defect testing method based on computer vision, including: Data acquisition and preprocessing module, used to collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, thermal imaging, etc., and perform data cleaning, format conversion and preliminary feature extraction; Multimodal data fusion and feature extraction module, which is used to receive preprocessed data, use deep learning models to perform multimodal feature fusion, and automatically learn and extract multi-level features of surface defects, including texture, shape, and location information; The defect intelligent classification and location module is used to train the model to intelligently classify and accurately locate surface defects based on the features extracted by the multimodal data fusion and feature extraction module, using transfer learning and reinforcement learning, and to evaluate the severity of the defects; Dynamic threshold adjustment and result verification module, which is used to dynamically adjust the threshold of defect detection according to real-time detection results and changes in the surface characteristics of accessories. At the same time, a manual verification mechanism is introduced to review the detection results of the model to ensure the accuracy and reliability of the detection; The defect prediction and production optimization module is used to predict potential surface defect trends based on historical inspection data, using time series analysis and prediction models, and take preventive measures in advance. At the same time, it analyzes the causes of defects, optimizes production processes, and reduces the defect rate in the production process.

[0013] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the surface defect testing method for new energy vehicle accessories based on computer vision as described in any one of the above are implemented.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the surface defect testing method for new energy vehicle accessories based on computer vision as described in any one of the above are implemented.

[0015] In summary, this application includes the following beneficial technical effects: 1. Through multimodal data collection and preprocessing, combined with deep learning feature fusion and extraction, this method can automatically and quickly identify and locate tiny defects on the surface of accessories, significantly improving the accuracy and efficiency of detection and reducing the error and time cost of manual detection; 2. Using dynamic threshold adjustment and result verification, as well as defect prediction and production optimization modules, the system can automatically adjust the inspection strategy according to real-time inspection results and changes in the surface characteristics of accessories, realize intelligent defect classification and prediction, and enhance the system's adaptability and flexibility; 3. By analyzing the causes of defects, the system can guide the optimization of the production process and take preventive measures to reduce the defect rate. At the same time, it ensures the accuracy of the test results through a manual verification mechanism, effectively improving the production quality and efficiency of new energy vehicle accessories. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of a surface defect testing method for new energy vehicle accessories based on computer vision in an embodiment of the present application.

[0017] Figure 2 It is a flowchart of the sub-steps of step S1 in the embodiment of the present application.

[0018] Figure 3 It is a flowchart of the sub-steps of step S2 in the embodiment of the present application.

[0019] Figure 4 It is a flowchart of the sub-steps of step S3 in the embodiment of the present application.

[0020] Figure 5 It is a flowchart of the sub-steps of step S4 in the embodiment of the present application.

[0021] Figure 6 It is a flowchart of the sub-steps of step S5 in the embodiment of the present application.

[0022] Figure 7 It is a flowchart of the sub-steps of step S6 in the embodiment of the present application.

[0023] Figure 8 It is a principle block diagram of a surface defect testing system for new energy vehicle parts based on computer vision in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following is combined with Figure 1-Figure 8 This application is described in further detail.

[0025] Reference Figure 1 The embodiment of the present application discloses a surface defect testing method for new energy vehicle accessories based on computer vision, which includes the following steps.

[0026] S1: Collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, and thermal imaging, and then perform preprocessing, including data cleaning, format conversion, and preliminary feature extraction.

[0027] Reference Figure 2 , for step S1, it includes the following sub-steps.

[0028] S11: Determine the type of data to be collected and configure the corresponding sensors and collection equipment.

[0029] S12: Ensure that multimodal data are synchronized in time and perform preliminary data fusion.

[0030] S13: Perform data cleaning, remove noise, convert data formats, and prepare input for deep learning models.

[0031] S14: Use the preprocessed data to preliminarily extract surface features to provide basic features for subsequent deep learning models.

[0032] Correspondingly, based on steps S11-S14, the embodiment of the present application provides the following examples.

[0033] For data collection, use a high-resolution industrial camera to ensure that the details of the accessory surface are clearly visible. For example, choose an industrial camera with 20 million pixels to capture tiny defects. At the same time, use a structured light scanner or lidar to obtain three-dimensional information on the accessory surface. For example, use Kinect v2 or Velodyne VLP-16 lidar to obtain accurate three-dimensional point cloud data. In addition, use an infrared thermal imager to capture the temperature distribution on the accessory surface. For example, choose a FLIR T640bx infrared thermal imager to detect potential thermal anomalies.

[0034] Furthermore, timestamp synchronization technology is used to ensure the time consistency of high-definition images, 3D point cloud data, and thermal imaging data during acquisition. For example, NTP (Network Time Protocol) is used to synchronize the clocks of all sensors to ensure that the synchronization accuracy of data acquisition is within 1 millisecond.

[0035] In addition, the collected images are denoised, for example, using median filtering or Gaussian filtering to remove image noise. 3D point cloud data is converted into point cloud images for deep learning model processing. For example, the PCL (Point Cloud Library) library is used to convert point cloud data into grayscale images. Thermal imaging data is converted into grayscale images to unify the data format. For example, the OpenCV library is used to convert thermal imaging data into grayscale images.

[0036] Finally, edge detection algorithms (such as Canny edge detection) are used to extract edge features from high-definition images. Surface texture features are extracted from 3D point cloud data, such as using the local binary pattern (LBP) algorithm. Temperature distribution features are extracted from thermal imaging data, such as calculating the temperature gradient of the image.

[0037] Replay Figure 1 , the following steps are also included after step S1.

[0038] S2: Dynamically adjust the direction and intensity of the light source according to the surface material and shape of the accessory, and apply image enhancement technology.

[0039] Reference Figure 3 , corresponding to step S2, including the following sub-steps.

[0040] S21: Analyze the surface material and shape of the accessory to determine the impact of lighting conditions on the test results.

[0041] S22: Based on the analysis results, dynamically adjust the direction and intensity of the light source to reduce shadows and reflections.

[0042] S23: Apply image enhancement technology to improve image quality.

[0043] S24: Evaluate the effects of lighting adjustment and image enhancement to ensure that the image quality meets the detection requirements.

[0044] Correspondingly, based on steps S21-S24, the embodiment of the present application provides the following examples.

[0045] For example, for highly reflective metal accessories, such as aluminum alloy wheels, its surface reflection characteristics are analyzed to determine the distribution of surface reflections and shadows under specific angles and intensities of light. This can be done by establishing a reflection model of the accessory surface, such as the BRDF (bidirectional reflectance distribution function) model, to simulate and analyze the reflection effect under different lighting conditions.

[0046] For accessories with complex surfaces, such as car body panels, the effects of their curved shapes on light occlusion and reflection are analyzed. For example, using CAD models or 3D scan data, the effects of different angles of light on the shadows and highlights of the curved surfaces are simulated.

[0047] Furthermore, for example, for highly reflective metal accessories, a multi-angle light source layout, such as a ring light source or a strip light source, can be used to illuminate the accessory surface from multiple directions to reduce highlight areas and shadows. The optimal light source angle is determined through experiments to ensure uniform illumination of the accessory surface.

[0048] Or the brightness of the light source can be dynamically adjusted according to the reflectivity of the accessory surface material and the detection requirements. For example, for plastic accessories with low reflectivity, the light intensity can be appropriately reduced to avoid overexposure; for metal accessories with high reflectivity, the light intensity can be appropriately increased to ensure clear details.

[0049] On the other hand, regarding image processing, histogram equalization or adaptive contrast enhancement technology is used to improve the contrast of the image and make the defect area more prominent. At the same time, denoising algorithms such as median filtering or bilateral filtering are applied to reduce image noise and improve image clarity. Furthermore, high-pass filtering or Laplace operator is used for sharpening processing to enhance the image edge and make the defect boundary clearer.

[0050] In addition, the quality of the adjusted image is evaluated through manual visual inspection to ensure that the defect area is clearly visible without obvious shadows and reflections. At the same time, image quality evaluation indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index) are used to quantitatively evaluate the image quality to ensure that the image quality meets the input requirements of the detection algorithm.

[0051] Replay Figure 1 , after step S2, the following steps are also included.

[0052] S3: Using a combination of deep convolutional neural networks and recurrent neural networks, we perform feature fusion on multimodal data to automatically learn and extract multi-level features of surface defects, including texture, shape, location, and time series information.

[0053] Reference Figure 4 , step S3 correspondingly includes the following sub-steps.

[0054] S31: Design a combined model of deep convolutional neural networks and recurrent neural networks for feature fusion of multimodal data.

[0055] S32: Train the model using the labeled dataset and optimize the model performance by adjusting the hyperparameters.

[0056] S33: Use the trained model to perform feature fusion on multimodal data and extract multi-level features of surface defects.

[0057] S34: Verify the effectiveness of the extracted features, adjust the model parameters as needed, and optimize the feature extraction effect.

[0058] Correspondingly, based on steps S31-S34, the embodiment of the present application provides the following examples.

[0059] For example, you can use ResNet or Inception network architectures, which have multiple convolutional layers and pooling layers that can capture features at different scales. At the same time, you can use LSTM (Long Short-Term Memory) networks that can remember long-term dependencies and are suitable for processing sequence data.

[0060] Furthermore, a multimodal dataset containing surface defect annotations is prepared. For example, using the ImageNet or COCO dataset as a basis, annotations of surface defects of new energy vehicle accessories are added.

[0061] At the same time, through cross-validation, adjust hyperparameters such as learning rate, batch size, regularization parameters, etc. to optimize model performance. For example, use grid search or random search methods to find the best hyperparameter combination.

[0062] On the other hand, the output features of DCNN and RNN are fused, for example, using an attention mechanism or a gating mechanism to dynamically adjust the weights according to the importance of the features to achieve weighted fusion of features. From the fused features, the texture, shape, location and time series information of the surface defects are extracted to form a multi-level feature representation.

[0063] Finally, use cross validation and confusion matrix to evaluate the classification performance of the model, for example, calculate the accuracy, recall and F1 score. At the same time, according to the evaluation results, adjust the model structure or hyperparameters, for example, increase or decrease the number of network layers, adjust the learning rate, to optimize the feature extraction effect.

[0064] Replay Figure 1 , after step S3, the following steps are also included.

[0065] S4: Based on the multi-level features of the extracted surface defects, the model is trained using transfer learning and reinforcement learning to intelligently classify and accurately locate surface defects, while also evaluating the severity of the defects.

[0066] Reference Figure 5 , for step S4, it includes the following sub-steps.

[0067] S41: Use pre-trained models for transfer learning to optimize the model's adaptability to new data.

[0068] S42: Design reinforcement learning strategies to optimize the model for classification and localization of surface defects.

[0069] S43: Use the trained model to intelligently classify and accurately locate surface defects.

[0070] S44: Evaluate the accuracy of classification and localization, and optimize model performance through feedback mechanisms.

[0071] Correspondingly, based on steps S41-S44, the embodiment of the present application provides the following examples.

[0072] First, choose a deep learning model pre-trained on a large-scale dataset (such as ImageNet), such as VGG, ResNet, or EfficientNet. The above models have learned rich image features.

[0073] At the same time, the first few layers of the pre-trained model are used as feature extractors, the weights of these layers are frozen, and only the last few layers (such as fully connected layers) are trained to adapt to the classification task of surface defects of new energy vehicle parts. For example, you can use a transfer learning library, such as TensorFlow's Keras API, to easily achieve model migration.

[0074] Furthermore, a reinforcement learning framework, such as OpenAI Gym or PyTorch's RL library, is used to build a reinforcement learning environment, in which the agent (model) learns to optimize the classification and localization strategies through interaction with the environment (surface defect images). A reward function is designed to reward or penalize the model based on the classification accuracy and localization accuracy of surface defects. For example, positive rewards are given for defects that are correctly classified and accurately located, and negative penalties are given for defects that are misclassified or have large localization deviations.

[0075] Furthermore, the test image is input into the trained model, and the model outputs the category and location information of the surface defects. For example, using an object detection model such as YOLO or Faster R-CNN, classification and positioning can be performed simultaneously. At the same time, the classification results and positioning boxes are drawn on the image to intuitively demonstrate the detection effect of the model.

[0076] Furthermore, standard evaluation indicators such as accuracy, recall, F1 score and mean average precision (mAP) are used to evaluate the classification and localization performance of the model. Based on the evaluation results, the model performance is continuously optimized by adjusting model parameters, optimizing algorithms or improving data preprocessing methods. For example, strategies such as learning rate decay, data enhancement or ensemble learning can be used to improve the generalization ability and robustness of the model.

[0077] Reference Figure 1 , after step S4, the following steps are also included.

[0078] S5: Dynamically adjust the threshold of defect detection based on real-time detection results and changes in the surface characteristics of accessories. At the same time, introduce a manual verification mechanism to review the detection results of the model.

[0079] Reference Figure 6 , step S5 correspondingly includes the following sub-steps.

[0080] S51: Analyze the real-time detection results and identify changes in the surface characteristics of the accessories.

[0081] S52: According to the analysis results, the threshold of defect detection is dynamically adjusted to adapt to the changes in the surface characteristics of the accessories.

[0082] S53: Introduce a manual verification mechanism to review the model's detection results to ensure accuracy.

[0083] S54: Feedback the manual verification results to the model to improve the accuracy and reliability of detection through continuous optimization.

[0084] Correspondingly, based on steps S51-S54, the embodiment of the present application provides the following examples.

[0085] First, a real-time detection model, such as YOLOv5 or EfficientDet, is used to perform real-time detection on continuous accessory surface images and output defect categories and location information. At the same time, by analyzing the detection results, changes in accessory surface characteristics are identified, for example, changes in surface roughness, glossiness, or color that may affect defect detection results.

[0086] In addition, the threshold of defect detection can be dynamically adjusted according to changes in the surface characteristics of the accessories. For example, for accessories with higher surface roughness, the threshold of defect detection can be appropriately lowered to avoid missed detection; for accessories with higher surface glossiness, the threshold of defect detection can be appropriately increased to avoid false detection.

[0087] Furthermore, an adaptive algorithm is designed to automatically adjust the threshold of defect detection according to the real-time inspection results and the surface characteristics of the accessories. For example, a machine learning algorithm such as support vector machine (SVM) or random forest (RF) is used to learn the threshold adjustment strategy according to the historical inspection results and the surface characteristics of the accessories.

[0088] On the other hand, a manual verification mechanism is introduced to review the model's detection results. For example, a human-computer interaction interface is used to display the model's detection results for manual review and correction. Review standards are established. For example, for defects detected by the model, manual verification is required to confirm whether the category and location are correct. For defects not detected by the model, manual verification is required to mark the category and location.

[0089] In addition, the manual verification results are fed back to the model, for example, using online learning algorithms such as online gradient descent (OGD) or stochastic gradient descent (SGD) to update the model parameters in real time according to the manual verification results. Improve the accuracy and reliability of detection through continuous optimization, for example, using strategies such as transfer learning, reinforcement learning, or meta-learning to continuously optimize the performance of the model based on real-time detection results and manual verification results.

[0090] Replay Figure 1 , the following steps are also included after step S5.

[0091] S6: Based on historical inspection data, use time series analysis and prediction models to predict potential surface defect trends, take preventive measures in advance, and analyze the causes of defects.

[0092] Reference Figure 7 , step S6 correspondingly includes the following sub-steps.

[0093] S61: Collect historical detection data and perform preprocessing in preparation for time series analysis.

[0094] S62: Use time series analysis methods to predict potential surface defect trends.

[0095] S63: Analyze the causes of defects and identify potential problems in the production process.

[0096] S64: Based on the analysis results, optimize the production process and take preventive measures.

[0097] Correspondingly, based on steps S61-S64, the embodiment of the present application provides the following examples.

[0098] First, historical inspection data is collected, including information such as inspection time, component type, defect category and location, etc. At the same time, the collected data is preprocessed, such as cleaning outliers, filling missing values, and standardizing data, in preparation for time series analysis.

[0099] In addition, time series analysis methods such as ARIMA, LSTM or Prophet are used to analyze historical inspection data and predict potential surface defect trends. Based on the time series analysis results, the potential trend of surface defects in the future is predicted, for example, the change trend of the number, category and location of defects is predicted.

[0100] Further, the causes of defects are analyzed, for example, using causal inference methods such as causal graphs or causal Bayesian networks to identify potential causes of defects. At the same time, potential problems in the production process are identified, for example, using fault tree analysis (FTA) or failure mode and effect analysis (FMEA) to identify potential problems in the production process.

[0101] Finally, based on the analysis results, the production process is optimized, for example, process parameters are adjusted, process flow is improved, or new process technologies are introduced to reduce the occurrence of surface defects. Preventive measures are also taken, for example, a quality control system is established, preventive maintenance is implemented, or an online inspection system is introduced to prevent the occurrence of surface defects.

[0102] Reference Figure 8 This embodiment also provides a new energy automobile parts surface defect testing system based on computer vision, which is used to execute the above-mentioned new energy automobile parts surface defect testing method based on computer vision, including: Data acquisition and preprocessing module, used to collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, thermal imaging, etc., and perform data cleaning, format conversion and preliminary feature extraction; Multimodal data fusion and feature extraction module, which is used to receive preprocessed data, use deep learning models to perform multimodal feature fusion, and automatically learn and extract multi-level features of surface defects, including texture, shape, and location information; The defect intelligent classification and location module is used to train the model to intelligently classify and accurately locate surface defects based on the features extracted by the multimodal data fusion and feature extraction module, using transfer learning and reinforcement learning, and to evaluate the severity of the defects; Dynamic threshold adjustment and result verification module, which is used to dynamically adjust the threshold of defect detection according to real-time detection results and changes in the surface characteristics of accessories. At the same time, a manual verification mechanism is introduced to review the detection results of the model to ensure the accuracy and reliability of the detection; The defect prediction and production optimization module is used to predict potential surface defect trends based on historical inspection data, using time series analysis and prediction models, and take preventive measures in advance. At the same time, it analyzes the causes of defects, optimizes production processes, and reduces the defect rate in the production process.

[0103] This embodiment also provides an electronic device. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a surface defect testing method for new energy vehicle accessories based on computer vision is implemented: S1: Collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, and thermal imaging, and then perform preprocessing, including data cleaning, format conversion, and preliminary feature extraction.

[0104] S2: Dynamically adjust the direction and intensity of the light source according to the surface material and shape of the accessory, and apply image enhancement technology.

[0105] S3: Using a combination of deep convolutional neural networks and recurrent neural networks, we perform feature fusion on multimodal data to automatically learn and extract multi-level features of surface defects, including texture, shape, location, and time series information.

[0106] S4: Based on the multi-level features of the extracted surface defects, the model is trained using transfer learning and reinforcement learning to intelligently classify and accurately locate surface defects, while also evaluating the severity of the defects.

[0107] S5: Dynamically adjust the threshold of defect detection based on real-time detection results and changes in the surface characteristics of accessories. At the same time, introduce a manual verification mechanism to review the detection results of the model.

[0108] S6: Based on historical inspection data, use time series analysis and prediction models to predict potential surface defect trends, take preventive measures in advance, and analyze the causes of defects.

[0109] When the computer program is executed by a processor, it can implement any of the surface defect testing methods for new energy vehicle parts based on computer vision in the above method embodiments.

[0110] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: S1: Collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, and thermal imaging, and then perform preprocessing, including data cleaning, format conversion, and preliminary feature extraction.

[0111] S2: Dynamically adjust the direction and intensity of the light source according to the surface material and shape of the accessory, and apply image enhancement technology.

[0112] S3: Using a combination of deep convolutional neural networks and recurrent neural networks, we perform feature fusion on multimodal data to automatically learn and extract multi-level features of surface defects, including texture, shape, location, and time series information.

[0113] S4: Based on the multi-level features of the extracted surface defects, the model is trained using transfer learning and reinforcement learning to intelligently classify and accurately locate surface defects, while also evaluating the severity of the defects.

[0114] S5: Dynamically adjust the threshold of defect detection based on real-time detection results and changes in the surface characteristics of accessories. At the same time, introduce a manual verification mechanism to review the detection results of the model.

[0115] S6: Based on historical inspection data, use time series analysis and prediction models to predict potential surface defect trends, take preventive measures in advance, and analyze the causes of defects.

[0116] When the computer program is executed by a processor, it can implement any of the surface defect testing methods for new energy vehicle parts based on computer vision in the above method embodiments.

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0118] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0119] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail in the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A surface defect testing method for new energy vehicle parts based on computer vision, characterized in that: The following steps are involved: Collect accessory surface information from multiple data sources including high-definition images, 3D point cloud data, and thermal imaging, and then perform preprocessing, including data cleaning, format conversion, and preliminary feature extraction; Dynamically adjust the direction and intensity of the light source according to the surface material and shape of the accessory, and apply image enhancement technology; Using a combination of deep convolutional neural networks and recurrent neural networks, feature fusion is performed on multimodal data to automatically learn and extract multi-level features of surface defects, including texture, shape, location, and time series information; Based on the multi-level features of the extracted surface defects, the model is trained using transfer learning and reinforcement learning to intelligently classify and accurately locate surface defects, while also assessing the severity of the defects. According to the real-time detection results and changes in the surface characteristics of the parts, the threshold of defect detection is dynamically adjusted, and a manual verification mechanism is introduced to review the detection results of the model; Based on historical inspection data, time series analysis and prediction models are used to predict potential surface defect trends, take preventive measures in advance, and analyze the causes of defects.

2. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The surface information of the accessories is collected from multiple data sources such as high-definition images, 3D point cloud data, and thermal imaging, and then preprocessed, including data cleaning, format conversion, and preliminary feature extraction, including the following steps: Determine the type of data to be collected and configure the corresponding sensors and collection equipment; Ensure that multimodal data are synchronized in time and perform preliminary data fusion; Perform data cleaning, remove noise, convert data formats, and prepare input for deep learning models; Use the preprocessed data to preliminarily extract surface features and provide basic features for subsequent deep learning models.

3. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The method of dynamically adjusting the direction and intensity of the light source according to the surface material and shape of the accessory and applying image enhancement technology includes the following steps: Analyze the surface material and shape of the accessories to determine the impact of lighting conditions on the test results; Based on the analysis results, the direction and intensity of the light source are dynamically adjusted to reduce shadows and reflections; Apply image enhancement technology to improve image quality; Evaluate the effects of lighting adjustment and image enhancement to ensure that the image quality meets the detection requirements.

4. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The method utilizes a combination of a deep convolutional neural network and a recurrent neural network to fuse features of multimodal data, automatically learn and extract multi-level features of surface defects, including texture, shape, position and time series information, including the following steps: Design a combined model of deep convolutional neural network and recurrent neural network for feature fusion of multimodal data; Use labeled datasets to train models and optimize model performance by adjusting hyperparameters; Using the trained model, feature fusion is performed on multimodal data to extract multi-level features of surface defects; Verify the effectiveness of the extracted features, adjust the model parameters as needed, and optimize the feature extraction effect.

5. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The method uses transfer learning and reinforcement learning to train a model based on the extracted multi-level features of surface defects to intelligently classify and accurately locate surface defects and evaluate the severity of the defects, including the following steps: Use pre-trained models for transfer learning to optimize the model's adaptability to new data; Design reinforcement learning strategies to optimize the model’s classification and location of surface defects; Using the trained model, surface defects can be intelligently classified and accurately located; Evaluate the accuracy of classification and localization, and optimize model performance through feedback mechanisms.

6. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The method of dynamically adjusting the threshold of defect detection according to the real-time detection results and the changes in the surface characteristics of the parts, and introducing a manual verification mechanism to review the detection results of the model, includes the following steps: Analyze real-time inspection results to identify changes in the surface characteristics of accessories; According to the analysis results, the threshold of defect detection is dynamically adjusted to adapt to the changes in the surface characteristics of the parts; Introduce a manual verification mechanism to review the model's test results to ensure accuracy; Feedback the manual verification results to the model to improve the accuracy and reliability of detection through continuous optimization.

7. The surface defect testing method for new energy automobile parts based on computer vision according to claim 1 is characterized in that: The method of predicting potential surface defect trends based on historical inspection data and using time series analysis and prediction models, taking preventive measures in advance, and analyzing the causes of defects includes the following steps: Collect historical test data and preprocess them in preparation for time series analysis; Use time series analysis methods to predict potential surface defect trends; Analyze the causes of defects and identify potential problems in the production process; Based on the analysis results, optimize the production process and take preventive measures.

8. A computer vision-based new energy automobile parts surface defect testing system, used to execute the computer vision-based new energy automobile parts surface defect testing method as described in any one of claims 1 to 7, characterized in that: include: Data acquisition and preprocessing module, used to collect accessory surface information from multiple data sources such as high-definition images, 3D point cloud data, thermal imaging, etc., and perform data cleaning, format conversion and preliminary feature extraction; Multimodal data fusion and feature extraction module, which is used to receive preprocessed data, use deep learning models to perform multimodal feature fusion, and automatically learn and extract multi-level features of surface defects, including texture, shape, and location information; The defect intelligent classification and location module is used to train the model to intelligently classify and accurately locate surface defects based on the features extracted by the multimodal data fusion and feature extraction module, using transfer learning and reinforcement learning, and to evaluate the severity of the defects; Dynamic threshold adjustment and result verification module, which is used to dynamically adjust the threshold of defect detection according to real-time detection results and changes in the surface characteristics of accessories. At the same time, a manual verification mechanism is introduced to review the detection results of the model to ensure the accuracy and reliability of the detection; The defect prediction and production optimization module is used to predict potential surface defect trends based on historical inspection data, using time series analysis and prediction models, and take preventive measures in advance. At the same time, it analyzes the causes of defects, optimizes production processes, and reduces the defect rate in the production process.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the surface defect testing method for new energy vehicle accessories based on computer vision are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the surface defect testing method for new energy vehicle parts based on computer vision as described in any one of claims 1 to 7 are implemented.

Images