Automatic defect detection system and method based on unsupervised learning

Abstract

The invention discloses an automatic defect detection system and method based on unsupervised learning, and belongs to the technical field of automatic detection. The method comprises the following steps: S1, collecting data including image data, temperature and humidity data and vibration data; s2, carrying out preprocessing and feature extraction on the collected data; s3, performing defect detection on the extracted feature data by adopting a trained YOLOv4 model and a trained Faster R-CNN model to obtain the type and the position of a defect; s4, performing corresponding execution and / or feedback operation according to the detected defect type and position; in the step S3, the image is quickly scanned by using a YOLOv4 model to generate a large number of candidate frames; inputting the candidate box into a Faster R-CNN model to further screen out a high-quality candidate region, and performing fine classification and bounding box regression; and results of the two models are fused to obtain a final detection result. The method can insight in fine defects which are difficult to capture by a traditional algorithm, and the accuracy and reliability of detection are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated detection, and particularly relates to an automated defect detection system and method based on unsupervised learning. Background Art

[0002] With the vigorous progress of cutting-edge technologies such as artificial intelligence, machine learning, and deep learning, automated detection products actively absorb and integrate these advanced technologies, aiming to improve the accuracy and efficiency of defect detection. For example, some systems adopt sophisticated image recognition and data analysis algorithms to achieve precise capture and identification of minute defects, demonstrating the powerful force of technological innovation. Automated detection is widely applied in many key fields such as manufacturing, aerospace, automotive industry, energy development, and modern agriculture. These systems can closely meet the actual needs of different industries and provide customized solutions, thus effectively ensuring the high quality and safety of products in various industries.

[0003] However, the existing automated detection generally has the following several defects:

[0004] ① The dual challenges of detection accuracy and false detection rate: Although many systems have adopted advanced algorithms and technologies to achieve higher detection accuracy, in complex and changeable production environments, they still face the problems of insufficient detection accuracy and high false detection rate. Such deficiencies may lead to the neglect or misidentification of key defects, thereby posing potential threats to the overall quality and safety of products, and affecting the reputation and market competitiveness of enterprises;

[0005] ② Difficulty in meeting customized requirements: Given the uniqueness and diversity of different industries and products, the requirements for defect detection systems naturally vary widely. Unfortunately, some similar products fail to fully consider this difference in design, resulting in difficulty in meeting the customized requirements of specific industries or products in actual applications. This may not only weaken the detection effect but also limit the adaptability of the system in complex and changeable production environments, affecting the production efficiency and product quality of enterprises;

[0006] ③ Potential risks of system stability and reliability: During long-term operation or under harsh production conditions, some similar products may expose problems of insufficient system stability and decreased reliability. Such instability may lead to serious consequences such as detection interruption or data loss, which not only affects the smooth progress of the production process but also may cause irreversible damage to product quality, thereby increasing the operation risk of enterprises;

[0007] ④The dual burden of high costs and complex maintenance: High-quality automated defect detection systems often come with high acquisition costs, as well as subsequent cumbersome installation, debugging, and maintenance work. This not only increases the initial investment pressure on enterprises but may also bring additional time and labor costs due to the need to hire professional technicians. These factors jointly limit the widespread application of similar products, especially in enterprises with limited resources or tight budgets. Summary of the Invention

[0008] The present invention aims to solve at least one of the technical problems in the above-related technologies to some extent.

[0009] To this end, the object of the present invention is to provide an automated defect detection system and method based on unsupervised learning, which can detect subtle defects that are difficult to capture by traditional algorithms and significantly improve the accuracy and reliability of detection.

[0010] To solve the above technical problems, the present invention is implemented as follows:

[0011] The embodiment of the present invention provides an automated defect detection method based on unsupervised learning, and the method includes:

[0012] S1. Collect data, including image data, temperature and humidity data, and vibration data;

[0013] S2. Preprocess and extract features from the collected data:

[0014] S3. Use the trained YOLOv4 model and Faster R-CNN model to detect defects in the extracted feature data to obtain the type and location of the defects;

[0015] S4. Perform corresponding execution and / or feedback operations according to the detected defect type and location.

[0016] In addition, according to the automated defect detection method based on unsupervised learning of the present invention, the following additional technical features may also be provided:

[0017] In some of the embodiments, the preprocessing in step S2 includes:

[0018] Perform denoising, contrast adjustment, and color enhancement processing on the collected image data;

[0019] Perform standardization processing on the temperature and humidity data and vibration data.

[0020] In some of the embodiments, the YOLOv4 model and the Faster R-CNN model in step S3 are implemented to cooperate in a manner of first connecting in series and then fusing;

[0021] First, use the YOLOv4 model to quickly scan the image and generate a large number of candidate boxes;

[0022] Then, input the candidate boxes generated by the YOLOv4 model into the Faster R-CNN model, use its RPN to further screen out high-quality candidate regions, and then perform fine classification and bounding box regression on the candidate regions through its ROI Pooling and fully connected layers;

[0023] Finally, fuse the results of the two models to obtain the final detection result.

[0024] In some embodiments, fusing the results of the two models is specifically performed by non-maximum suppression or score-weighted fusion.

[0025] In some embodiments, perform various data augmentations on the collected image data, including rotation, scaling, and brightness adjustment.

[0026] In some embodiments, data denoising processing is implemented using median filtering and / or Gaussian filtering algorithms.

[0027] In some embodiments, the data for training the YOLOv4 model and the Faster R-CNN model in step S3 is from historical data; all historical data is defect-labeled using a combination of manual annotation and automatic annotation; the model is trained with the labeled data.

[0028] In some embodiments, a pre-trained deep learning model is used to extract features from the preprocessed data.

[0029] The embodiment of the present invention also provides an unsupervised learning-based automated defect detection system for implementing the steps of the unsupervised learning-based automated defect detection method described in any one of the above; the system includes:

[0030] A data acquisition module for acquiring image data, temperature and humidity data, and vibration data;

[0031] A data processing and analysis module for preprocessing and feature extraction of the acquired data;

[0032] A target localization and defect recognition module for defect detection of the extracted feature data through the YOLOv4 model and the Faster R-CNN model to obtain the type and location of the defect; and,

[0033] An execution and feedback control module for performing corresponding execution and / or feedback operations according to the detected defect type and location.

[0034] In addition, the automated defect detection method based on unsupervised learning according to the present invention may further have the following additional technical features:

[0035] In some of these embodiments, the system further includes:

[0036] A cloud data storage and remote monitoring module that receives data and results from each module and provides a remote access interface for users to access various types of data in real time through a web page or application; and

[0037] A user interface for graphically displaying real-time data, analysis results, and detection information for intuitive viewing by operators; and at the same time providing an artificial interaction interface for operators to set parameters, initiate repairs, and / or perform manual interventions.

[0038] Compared with the prior art, the present invention has at least the following beneficial effects:

[0039] In the embodiments of the present invention, the provided automated defect detection method based on unsupervised learning is a profound insight and active response to a series of key challenges in the current field of automated defect detection; it not only aims to solve the problems of insufficient detection accuracy and high false detection rate commonly existing in similar products, but also realizes unprecedented high-precision recognition and precise positioning of micro defects by introducing more advanced algorithms and deep learning technologies. This breakthrough progress enables the present application to capture those subtle defects that are difficult to reach by traditional methods, thus ensuring the flawless and excellent quality of the product;

[0040] In the embodiments of the present invention, the provided automated defect detection method based on unsupervised learning, through deep learning algorithms (such as YOLO, Faster R-CNN), in target localization and defect recognition, the model can accurately judge the position of the bounding box of the object; especially in complex environments (such as overlapping objects or distant objects), the model can still maintain a high positioning accuracy; for defects such as micro cracks, corrosion, and lesions, the detection accuracy of the model reaches more than 90%, especially in low-contrast areas and occlusions, the recognition accuracy is improved by 10 - 15% compared with traditional algorithms;

[0041] In the embodiments of the present invention, the provided automated defect detection method based on unsupervised learning significantly reduces manual intervention, simplifies the operation process, reduces the error rate, and thus significantly improves the work efficiency;

[0042] In the embodiments of the present invention, the provided automated defect detection system based on unsupervised learning captures environmental data in real time through the sensor module, quickly outputs results in combination with the data processing and analysis module, and the execution and feedback control module immediately takes actions according to the detection results, such as repair or alarm, to ensure the immediacy of the response;

[0043] In the embodiments of the present invention, the provided automated defect detection system based on unsupervised learning deeply integrates high-performance hardware and deep learning algorithms. Through optimized hardware designs (such as efficient data acquisition sensors and high-performance processing units) and efficient algorithms (such as convolutional neural networks and deep learning models), a double leap in processing speed and accuracy is achieved.

[0044] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. Brief Description of the Drawings

[0045] Figure 1 It is a flowchart of an automated defect detection method based on unsupervised learning disclosed in an embodiment of the present invention;

[0046] Figure 2 It is a schematic structural diagram of an automated defect detection system based on unsupervised learning disclosed in an embodiment of the present invention. Detailed Embodiments

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] Next, the embodiments of the present invention will be described in detail in conjunction with the drawings through specific embodiments and their application scenarios.

[0049] In some embodiments of the present invention, an automated defect detection system and method based on unsupervised learning are provided. By introducing more advanced algorithms and deep learning technologies, unprecedented high-precision identification and precise positioning of micro defects are achieved, enabling the system to capture those subtle defects that are difficult to reach by traditional methods, thereby ensuring the flawless and excellent quality of the product.

[0050] Please refer to Figure 1 As shown, in some embodiments of the present invention, the steps of the automated defect detection method based on unsupervised learning include:

[0051] Step 1: Collect data.

[0052] It is collected by high-precision sensors. The high-precision sensors include image sensors, temperature and humidity sensors, and vibration sensors. For example, the resolution of the image sensor reaches above 4K, enabling it to clearly capture subtle disease spots and cracks; the accuracy of the temperature and humidity sensors reaches ±0.1°C and ±0.5%RH, ensuring high-precision environmental monitoring; the vibration sensors are installed on key moving parts of the equipment (such as bearings, gears, motors) or structural components (such as pipes, racks) to collect the frequency, amplitude, and harmonic components of vibration signals in real time. These vibration data can reflect the operating state of the equipment. Through the fusion with image data, the system can synchronously detect mechanical defects (such as bearing wear, abnormal gear meshing) and environmental disturbances (such as equipment jitter caused by wind speed). These several sensors have high stability. For example, under different lighting conditions (including low light, strong light, shadow, etc.), the data collected by the sensors remains accurate, avoiding the influence of external environmental changes on data quality.

[0053] Experimental data shows that in different application scenarios such as farmland, industrial sites, and infrastructure, the data error collected by the sensors is less than 1%, and it can be stably transmitted to the subsequent data processing and analysis module, providing high-quality input data for the subsequent deep learning model.

[0054] Step 2: Preprocess and extract features from the collected data.

[0055] Image data augmentation: By performing various data augmentations (such as rotation, scaling, brightness adjustment, etc.) on the collected images, the robustness of the deep learning model under various changing conditions is significantly improved, and both the accuracy and recall rate are significantly increased.

[0056] Denoising and Feature Extraction: After the data preprocessing stage is completed, the system uses a multi-scale deep feature fusion network to automatically extract defect features: By using the YOLOv4 model and the Faster R-CNN model as the backbone networks, combining the spatial pyramid pooling of the SPP module and the path aggregation network of PANet, multi-level fusion of low-level detailed features (such as edge textures) and high-level semantic features (such as global semantic information) is achieved. Specifically, after the input image is standardized, first, the backbone network generates multi-scale feature maps, and then the FPN is used to fuse high-level semantics and low-level detailed features from top to bottom to enhance the adaptability to tiny defects and complex backgrounds. Subsequently, YOLOv4 directly predicts the bounding box coordinates and class probabilities through the Darknet53 head, while Faster R-CNN generates high-quality candidate regions through the RPN network. Finally, the system synchronously extracts visual features (such as lesion shape, color, texture), environment-related features (temperature and humidity sensitive features), and vibration-related features (frequency domain harmonic components) to generate a multi-dimensional feature vector for subsequent classification and localization. This feature extraction method significantly improves the anti-interference ability through data augmentation (rotation, scaling, brightness adjustment) and adaptive normalization, effectively improves the defect detection rate in low-contrast scenarios, and supports cross-domain deployment through pre-trained model transfer learning, while meeting the real-time requirement of single-frame processing time < 100ms in industrial scenarios.

[0057] The experimental results show that the accuracy of the preprocessed image data in agricultural pest and disease identification has increased by more than 5%, especially in complex backgrounds (such as occlusions, environmental interferences, etc.), it can still maintain a high recognition accuracy.

[0058] Step 3: Use the trained deep learning model for detection.

[0059] In the object detection experiment, using the trained YOLOv4 and Faster R-CNN models, the performance on the agricultural pest and disease and infrastructure detection datasets exceeded the mean average precision (mAP) of traditional methods, and the accuracy of lesion detection in the agricultural field increased by more than 10%.

[0060] In the defect detection of industrial products, the deep learning model performed particularly well in the identification of tiny cracks, successfully increasing the recognition rate by 15%.

[0061] The function of the above YOLOv4 model is to achieve fast object localization. As a single-stage object detection model, by scanning the entire image at once, the YOLOv4 model can quickly locate the approximate positions of pests and diseases and generate multiple candidate boxes. The YOLOv4 model can directly predict the bounding boxes and classes of objects on the image without generating candidate regions. Its advantage lies in its fast speed, making it suitable for real-time or near-real-time detection. The YOLOv4 model uses CSPDarknet53 as the backbone network and combines SPP (Spatial Pyramid Pooling) and PANet (Path Aggregation Network), enhancing the detection ability for multi-scale objects (such as pests and diseases of different sizes), so it has the characteristic of multi-scale feature fusion.

[0062] The function of the above Faster R-CNN model is to achieve high-precision classification and localization. Faster R-CNN generates high-quality candidate boxes through RPN (Region Proposal Network), and then performs fine-grained classification and bounding box regression on the candidate boxes through ROIPooling and fully connected layers. Its advantage lies in its high accuracy, especially being good at dealing with dense or small objects. As a two-stage model, for possible occlusions, overlaps, or complex backgrounds (such as leaf texture interference) in agricultural scenarios, the two-stage design of the Faster R-CNN model can more robustly extract features and distinguish pests and diseases.

[0063] In some embodiments of the present invention, if the pre-trained model performs well on a general dataset, Faster R-CNN can quickly adapt to specific classes of agricultural pests and diseases through fine-tuning, that is, the method of transfer learning optimization. By adopting this method, model training in different application scenarios can be quickly achieved, so as to be used for classification in corresponding fields. Four embodiments in different fields and scenarios are provided later in the present invention, which are quickly obtained based on this training method.

[0064] In some embodiments of the present invention, the YOLOv4 model and the Faster R-CNN model cooperate in a cascaded manner. First, the YOLOv4 is used to quickly scan the image to generate a large number of candidate boxes (such as the approximate positions of pests and diseases). The candidate boxes generated by the YOLOv4 model are input into the Faster R-CNN model, and its RPN is used to further screen out high-quality candidate regions, and the detection accuracy is improved through classification and regression. The YOLOv4 solves the real-time problem, reduces the number of candidate boxes that the Faster R-CNN model needs to process, and reduces the computational load. The Faster R-CNN model makes up for the lack of accuracy of the YOLOv4 model in small target or complex scenarios, and improves the reliability of the final detection result. The two models cooperate with each other to achieve complementary advantages. Finally, non-maximum suppression (NMS) or score-weighted fusion can be used, that is: for overlapping candidate boxes, set the IoU threshold to 0.3 to retain high-confidence detection results, and at the same time introduce the Soft-NMS algorithm (Gaussian weighting parameter σ = 0.5) to alleviate the missed detection problem of dense small targets; for the outputs of the two types of models, the weights are dynamically allocated based on the performance of the validation set (YOLOv4 weight 0.3, Faster R-CNN weight 0.7), only the prediction boxes with a combined confidence level ≥ 0.6 are retained, and the image features (such as lesion shape, color), temperature time-series data (such as environment-sensitive features), and vibration frequency-domain features (such as equipment fault harmonics) are weighted and fused through the attention mechanism. The prediction results of the two models are combined to further optimize the detection performance. The experimental results show that the error rate of the present invention in identifying and locating complex defects is less than 5%, the recognition accuracy rate in agricultural pest and disease detection reaches 96.55%, which is 8% higher than that of a single model, and the missed detection rate in small target scenarios is reduced by 12%; in infrastructure detection, both the accuracy rate and the recall rate can reach more than 90%, far exceeding the effects of traditional image processing and machine learning methods; fully verifying the technical advantages of the multi-scale feature pyramid (such as the PANet-D structure) and cross-modal data fusion.

[0065] In the above embodiments, the data for training the model is sourced from historical data. First, it is labeled, and then the same preprocessing and feature extraction as above are performed. A combination of manual and automatic labeling methods can be used to ensure that each image in the training set and validation set is accurately labeled. The accuracy of labeling directly improves the learning quality of the trained model. Additionally, the data of the present invention is derived from the fusion of multiple data sources; in addition to traditional visual data, it also includes sensor data; the sensor data (such as temperature, humidity, vibration) is fused with the image data, so as to provide more feature information for model training. Among them, the temperature and humidity data is mainly used for environmental state perception. For example, in agricultural pest and disease detection, there is a strong correlation between temperature and humidity fluctuations and pathogen growth (such as high humidity accelerating the spread of rust). After extracting the temporal features through the LSTM network and performing attention mechanism weighted fusion with the morphological features of the disease spots in the image, environmental interference (such as leaf surface dew) can be distinguished from real lesions; the vibration data is used for equipment status diagnosis. For example, in industrial equipment detection, the frequency domain features of the vibration signal (such as the harmonic frequency of gear faults) are extracted through 1D-CNN and then concatenated with the feature map of the surface defect image of the equipment in the channel dimension. After screening the key features through the residual shrinkage module, both internal mechanical faults (abnormal vibration) and external damage (image cracks) can be captured simultaneously. The data diversity enables the model to perform more stably in different application scenarios. When multiple sensors and processing modules are running simultaneously, the system can still operate stably without performance degradation or data loss during the processing of large-scale data. The present invention has the ability to process real-time data. The latency of the entire process from data acquisition to target recognition and feedback control is less than 200 ms, ensuring that the system can respond in real time and meet the requirements of efficient automated operations.

[0066] Please refer to Figure 2 As shown, in some embodiments of the present invention, an automated defect detection system based on unsupervised learning is provided, including: a data acquisition module, a data processing and analysis module, a target location and defect recognition module, an execution and feedback control module, a cloud data storage and remote monitoring module, and a user interface.

[0067] Specifically, the data acquisition module is used to collect environmental and device data in real time through sensors and cameras. These data cover images captured by the camera, temperature and humidity monitored by environmental sensors, vibration sensed by vibration sensors, etc. Subsequently, these data are seamlessly transmitted to the data processing and analysis module through interfaces such as USB, serial ports, and wireless networks for in-depth processing, unification, and refinement. This module integrates image sensors (cameras), temperature and humidity sensors, vibration sensors, etc., captures data in real time, and quickly transmits the data (image frames, temperature and humidity values, etc.) to the data processing module through wired or wireless interfaces. For video stream data, high-speed channels such as Wi-Fi and Ethernet are used to ensure the smoothness of data transmission.

[0068] Furthermore, the data processing and analysis module receives sensor data, performs preliminary purification, noise reduction, and format preprocessing on it (such as grayscale conversion, noise elimination), as well as standardization of sensor data (such as normalization of temperature and humidity data). After that, these carefully processed data are delivered to the target location and defect identification module for higher-order analysis and detection work. The operation mechanism of this module is as follows: perform denoising, contrast adjustment, color enhancement, etc. on the image data; standardize sensor data such as temperature and humidity to present them in a unified standard scale; at the same time, load pre-trained deep learning models (CNN, ResNet, VGG, etc.) to extract data features and ensure that the data is transmitted to the next module in a form acceptable to the model.

[0069] Furthermore, the target location and defect identification module receives the results of the data processing and analysis module, and relies on deep learning models (such as convolutional neural network YOLOv4 model and Faster R-CNN model, etc.) to accurately locate the target and identify defects. The details of the detected target and defects (location, category, severity, etc.) are immediately transmitted to the execution and feedback control module, which triggers corresponding control actions accordingly. The operation mechanism of this module is as follows: load the deep learning models YOLOv4 and Faster R-CNN, and use techniques such as image classification and bounding box regression to locate the target object and identify its location and category; at the same time, the model can also identify defects on the target, such as cracks and scratches, and output the target location, category, and defect level information to provide guidance for subsequent operations.

[0070] Further, the execution and feedback control module performs specific operations (such as adjusting the equipment layout, starting the repair process, alarming, etc.) based on the detection results of the target positioning and defect identification module; meanwhile, the real-time status and operation effectiveness of the system are immediately fed back to the user interface, facilitating the operator to monitor in real time and take manual measures as needed (such as parameter adjustment or manual repair). The operation mechanism of this module is: after receiving the defect information and target position information, it evaluates the severity of the defect and initiates corresponding operations (such as alarming, equipment adjustment, repair, etc.). The execution results are fed back to the user interface in real time, facilitating the operator to monitor and intervene.

[0071] Further, the cloud data storage and remote monitoring module presents all the treasured data and historical analysis results on the user interface. With just a click of the mouse, the user can access real-time data, alarm information, and system status, easily realizing remote monitoring and management. The operation mechanism of this module is: it receives the data and results from each module (processed sensor data, target detection and defect identification results, etc.) and uploads them to the cloud; the cloud platform provides a remote access interface, and the user can view the device status, analysis results, and alarm information in real time through a web page or application program for subsequent decision-making.

[0072] Further, the user interface displays real-time data, analysis results, and detection information in a graphical manner, facilitating the operator to intuitively understand the system status. At the same time, the operator can set system parameters, start the repair process, or perform manual intervention through the interface, and make decision adjustments according to the feedback information of the execution and feedback control module.

[0073] In some embodiments of the present invention, the content of the cloud data storage and remote monitoring function includes:

[0074] 1. Fast upload and storage: It can upload approximately 100MB of data per second and perform efficient storage and retrieval in the cloud; the cloud platform provides fast data access and real-time analysis functions to ensure that users can obtain data in a timely manner and make decisions.

[0075] 2. Historical data analysis: Through the analysis of historical data, the model can discover the trends of potential problems and provide early warnings, which provides strong data support for subsequent decision-making.

[0076] In remote monitoring and data storage, the access delay of the system does not exceed 1 second, and the data analysis and storage capabilities of the cloud can support high-frequency data upload and real-time monitoring. The analysis capabilities of the cloud module help users identify potential risks and problems in the fields of agriculture, infrastructure, and industry, and take preventive or maintenance measures in advance, thereby reducing the operating costs.

[0077] Example 1: Pest and disease detection and prevention in smart agriculture

[0078] Imagine a modern corn farm covering thousands of acres. The traditional pest and disease monitoring method relies on manual field inspections, which is not only time-consuming and labor-intensive, but also easy to miss early symptoms. Now, the farm has introduced an intelligent pest and disease monitoring system. High-definition cameras and temperature and humidity sensors are deployed in key areas of the farmland to monitor environmental changes and crop growth status 24 hours a day. One day, the system captured a set of abnormal data: image features of suspected rust spots appeared on the leaves of a certain corn area. Through the efficient analysis of the deep learning algorithm, the system quickly confirmed that this was an early sign of corn rust, and automatically triggered the precision spraying system to apply pesticides only to the infected areas, effectively curbing the further spread of the disease. At the same time, all monitoring data is securely uploaded to the cloud server, and farm managers can remotely access it through mobile phones or computers anytime, anywhere, to grasp the health status of the farmland in real time, and adjust field management measures accordingly.

[0079] Example 2: Infrastructure defect detection during drone inspection

[0080] Consider an ultra-high voltage transmission line located in a remote area. The traditional inspection method relies on manual inspection along the line, which is not only inefficient but also has safety hazards. Now, the line adopts a drone inspection solution. The drone is equipped with a high-resolution camera and flies autonomously according to the preset route to take a comprehensive picture of the line. During a routine inspection, the system identified abnormal reflections on the surface of the insulator of a certain section of the line. Through the detailed analysis of the deep learning model, it was accurately determined that this was a sign of discharge caused by the accumulation of dirt on the surface of the insulator, and there was a potential risk of line short circuit. The system immediately issued an early warning to the ground control center and provided detailed defect location information and severity assessment. Based on this, the control center staff quickly formulated a maintenance plan and arranged a professional team to go to the site for cleaning and maintenance, effectively avoiding possible power outages.

[0081] Example 3: Real-time detection and maintenance of oil production equipment damage

[0082] On a deep-sea oil production platform, oil production equipment has been operating under high pressure, high temperature and corrosive environment for a long time. Traditional manual inspection methods are difficult to fully cover and have poor timeliness. Now, the platform has deployed an intelligent oil production equipment monitoring system. Vibration sensors, pressure sensors and temperature sensors are installed on key equipment components to monitor the operating status of the equipment in real time. Once, the system detected an abnormal increase in the vibration frequency of a centrifugal pump. Through comprehensive analysis of the deep learning algorithm, it was determined that this was a signal of increased bearing wear. The system immediately issued an alarm to the platform control room and provided a quantitative assessment of the degree of bearing wear. The control room staff responded quickly and arranged for technicians to shut down the centrifugal pump for inspection and replaced the worn bearings in time, effectively avoiding production interruptions and safety hazards caused by equipment failure.

[0083] Example 4: Quality Inspection and Production Optimization in Industrial Manufacturing

[0084] In a highly automated automobile manufacturing plant, each production line is equipped with an intelligent quality inspection system. High-precision detection devices such as vision sensors and laser rangefinders are integrated at key workstations on the production line to collect and process product data in real time. During a production process, the system detected a slight scratch defect on the body of a car being assembled. Through the rapid analysis of the deep learning algorithm, the position and length of the scratch were accurately located. The system immediately triggered an emergency stop signal for the production line and guided the car to the defective product handling area. At the same time, all quality inspection data was automatically uploaded to the cloud database, and quality control engineers used data analysis tools to deeply explore potential problems in the production process, timely adjust production process parameters and process layout, and continuously improve product quality and production efficiency.

[0085] The technical advantages of the present invention stem from the collaborative design of its multi-modal data fusion architecture and real-time stream processing engine. The specific technical points include:

[0086] 1. Stability guarantee for data diversity: Through cross-modal feature alignment techniques (such as multi-head attention mechanism) and distributed computing frameworks (such as Apache Flink), the system maps multi-source data such as images, temperature and humidity, and vibration to a unified feature space, and realizes parallel processing through a redundant data buffer pool (based on the in-memory database Redis) to ensure the high fault tolerance of sensor data in heterogeneous scenarios. For example, the spatio-temporal alignment module of vibration signals and image crack features in industrial inspection uses Kalman filtering to eliminate data asynchronous errors and avoid performance fluctuations during multi-sensor concurrency.

[0087] 2. Low-latency real-time processing ability: Based on lightweight stream processing technologies (such as Kafka Streams) and model acceleration strategies (such as CSPNet structure pruning), the system controls the entire data processing process within 200 ms. Specifically, image data quickly generates candidate boxes through the lightweight backbone network of Mobilenetv2 of YOLOv4 (single-frame time consumption < 100 ms). At the same time, the frequency domain features of vibration signals are compressed into low-dimensional tensors through 1D-CNN. After the two are fused in the Feature Pyramid Network (PANet), they are further screened by the RPN network, and finally the detection results are merged through the Soft-NMS algorithm. The entire process uses pipeline parallel acceleration.

[0088] 3. Efficient automated operation mechanism: Through decision-level fusion (such as D-S evidence theory) and pre-trained model transfer technology (such as reusing the BERT embedding layer), the system dynamically matches the multi-modal reasoning results with the domain knowledge base (such as the equipment failure mode library), and uses reinforcement learning (such as Q-Learning) to optimize the control strategy. For example, in agricultural pest and disease detection, the model combines the real-time temperature and humidity LSTM prediction with the image lesion detection results to automatically trigger precise pesticide application instructions, realizing a closed-loop control from perception to execution.

[0089] For the parts not detailed in the present invention, reference can be made to the prior art or they are well-known technologies to those skilled in the art. This embodiment does not make any limitations in this regard and will not be described in detail here.

[0090] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the claims of the present invention, and all of them fall within the protection scope of the present invention.

Claims

1. An automated defect detection method based on unsupervised learning, characterized in that, The method includes: S1. Collect data, including image data, temperature and humidity data, and vibration data; S2. Preprocess and extract features from the collected data; S3. Use the trained YOLOv4 model and Faster R-CNN model to perform defect detection on the extracted feature data to obtain the type and location of the defects; S4. Perform corresponding execution and / or feedback operations according to the detected defect type and location.

2. The automated defect detection method based on unsupervised learning according to claim 1, characterized in that The preprocessing in step S2 includes: Perform denoising, contrast adjustment, and color enhancement on the collected image data; Perform standardization on the temperature and humidity data and vibration data.

3. The automated defect detection method based on unsupervised learning according to claim 2, wherein In step S3, the YOLOv4 model and Faster R-CNN model cooperate in a way of first being connected in series and then fused; First, use the YOLOv4 model to quickly scan the image to generate a large number of candidate boxes; Then, input the candidate boxes generated by the YOLOv4 model into the Faster R-CNN model, use its RPN to further screen out high-quality candidate regions, and then perform fine classification and bounding box regression on the candidate regions through its ROI Pooling and fully connected layers; Finally, fuse the results of the two models to obtain the final detection result.

4. The automated defect detection method based on unsupervised learning according to claim 3, wherein Fusing the results of the two models is specifically performed by non-maximum suppression or score-weighted fusion.

5. The automated defect detection method based on unsupervised learning according to claim 2, wherein Perform various data augmentations on the collected image data, including rotation, scaling, and brightness adjustment.

6. The automated defect detection method based on unsupervised learning according to claim 2, wherein The data denoising process is implemented using median filtering and / or Gaussian filtering algorithms.

7. The automated defect detection method based on unsupervised learning according to claim 1, characterized in that In step S3, the data for training the YOLOv4 model and Faster R-CNN model comes from historical data; all historical data is defect-labeled using a combination of manual annotation and automatic annotation; the model is trained with the labeled data.

8. The automated defect detection method based on unsupervised learning according to claim 1, characterized in that, Use a pre-trained deep learning model to extract features from the preprocessed data.

9. An automated defect detection system based on unsupervised learning, characterized in that, For implementing the steps of the unsupervised learning-based automated defect detection method according to any one of claims 1-8; the system includes: A data acquisition module for collecting image data, temperature and humidity data, and vibration data; A data processing and analysis module for preprocessing and extracting features from the collected data; A target localization and defect recognition module for performing defect detection on the extracted feature data through the YOLOv4 model and Faster R-CNN model to obtain the type and location of the defects; and An execution and feedback control module for performing corresponding execution and / or feedback operations according to the detected defect type and location.

10. The automated defect detection system based on unsupervised learning according to claim 9, wherein, The system further includes: A cloud data storage and remote monitoring module that receives data and results from each module and provides a remote access interface for users to access various types of data in real time through a web page or application; and A user interface for graphically displaying real-time data, analysis results, and detection information for intuitive viewing by operators; and at the same time providing a human-computer interaction interface for operators to set parameters, initiate repairs, and / or perform manual intervention.

Images