A reducer appearance detection method and system based on deep learning
Through the combination of high-precision imaging and deep learning models, feature pyramid network and spectral analysis are used to solve the problems of low efficiency and insufficient adaptability of traditional detection methods, high accuracy and real-time detection are achieved, and detailed fault description and maintenance are provided. It is recommended to extend the life of the equipment.
Patent Information
- Application Number
- CN202411261105.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-10
AI Technical Summary
Traditional reducer appearance detection methods are inefficient and easily subjective, making them difficult to meet the high requirements of modern industry for detection accuracy and speed. In addition, deep learning models lack adaptive learning capabilities when facing new failure modes.
The high-precision imaging module is used for multi-angle image acquisition and preprocessing, combined with deep learning models for preliminary feature extraction and defect recognition, and dynamically adjust model parameters through adaptive learning mechanisms, use feature pyramid networks to fuse features at different scales, and combine spectral analysis. The data is evaluated for appearance quality, and a comprehensive inspection report and preventive maintenance recommendations are generated.
Improves the accuracy and efficiency of detection, can identify known defects and adapt to new failure modes, comprehensively evaluate appearance quality, reduce manual intervention, provide real-time inspection reports and preventive maintenance recommendations, and extend the service life of the equipment.
Smart Images

Figure CN118799592B_ABST
Abstract
Description
Technical Field
[0001] The present invention proposes a reducer appearance detection method and system based on deep learning, belonging to the technical field of deep learning and reducer appearance detection. Background Art
[0002] As an indispensable transmission component in mechanical equipment, the performance stability and appearance quality of reducers directly impact the overall equipment's operating efficiency and lifespan. Traditional reducer appearance inspection methods rely primarily on manual visual inspection, which is not only inefficient but also susceptible to subjective judgment, leading to frequent missed inspections or misjudgments. Furthermore, with the increasing variety and complexity of reducer structures, the requirements for inspection accuracy and speed are becoming increasingly stringent, making traditional methods unable to meet the needs of modern industrial production.
[0003] In recent years, the rapid rise of deep learning technology has brought new solutions to reducer visual inspection. Deep learning models, particularly convolutional neural networks (CNNs), have demonstrated powerful feature extraction and pattern recognition capabilities in image processing, automatically learning and identifying complex features and defect patterns in images. This technological advantage holds broad application prospects for deep learning in reducer visual inspection.
[0004] However, relying solely on deep learning models for preliminary feature extraction and defect identification in images still has certain limitations. Firstly, due to the complexity and diversity of reducer appearance, feature extraction at a single scale may not fully reflect its appearance quality. Secondly, traditional deep learning models often lack the ability to adaptively learn emerging fault modes, requiring frequent model updates and retraining. Summary of the Invention
[0005] The present invention provides a method and system for speed reducer appearance inspection based on deep learning to solve the problems mentioned in the above background technology:
[0006] The present invention proposes a method for reducing machine appearance inspection based on deep learning, the method comprising:
[0007] S1. Use a high-precision imaging module to capture comprehensive, multi-angle images of the reducer and pre-process the captured images;
[0008] S2. Apply deep learning models to perform preliminary feature extraction on preprocessed images, identify appearance defects, and use adaptive learning mechanisms to dynamically adjust model parameters based on emerging failure modes;
[0009] S3. Use a feature pyramid network to fuse features of different scales, and use spectral analysis data combined with visual features to evaluate appearance quality.
[0010] S4. Analyze the assessment results in real time and automatically generate a comprehensive inspection report. Through the fault prediction model, based on historical data and current inspection reports, generate preventive maintenance recommendations.
[0011] Furthermore, the S2 includes:
[0012] S21. Select a deep learning model and pre-train the deep learning model using a labeled reducer appearance image dataset;
[0013] S22, inputting the preprocessed image into a pre-trained deep learning model, and automatically extracting multi-level features in the image through a convolutional layer, wherein the multi-level features include edges, textures, and shapes;
[0014] S23. Generate a defect candidate region on the feature map through a region proposal network, and determine whether the region contains a defect through a classifier;
[0015] S24. If defects are included, the identified defects are sub-classified using a hierarchical classification strategy;
[0016] S25. Integrate an online learning module to automatically collect and annotate samples of emerging failure modes or misidentification cases, and dynamically adjust deep learning model parameters using incremental learning.
[0017] Furthermore, the S22 includes:
[0018] Adaptive median filtering or bilateral filtering algorithms are used to dynamically adjust filtering parameters based on local image features to remove random noise while retaining edge details.
[0019] Apply the Retinex algorithm or a deep learning-based illumination normalization model to perform global and local illumination adjustments on the image to reduce the impact of illumination changes on feature extraction;
[0020] Using histogram equalization or its improved method to enhance the contrast of the image;
[0021] Construct an image pyramid and apply convolutional layers at different scales to extract multi-scale features from coarse to fine;
[0022] Design a feature fusion module to fuse features extracted by convolutional layers of different scales;
[0023] Residual connections are introduced in the convolutional layer, and principal component analysis is used to evaluate the importance of features, identify and retain the most critical features for defect detection, and perform feature sparsification through L1 regularization.
[0024] Furthermore, the S23 includes:
[0025] Anchor boxes are designed based on the shapes and sizes of common defects in reducer appearance images. An adaptive anchor box generation mechanism is also introduced to dynamically adjust the size and proportion of the anchor box based on the image content.
[0026] Using the feature pyramid network structure, feature maps of different scales are fused, enabling the region proposal network to detect defects at multiple scales;
[0027] The region proposal network is used to slide the window on the feature map to generate preliminary defect candidate regions; non-maximum suppression is used to remove redundant and overlapping candidate regions based on the scores and overlap of the candidate regions.
[0028] Based on the complexity and real-time requirements of defect classification, a suitable classifier architecture is selected and a feature recalibration module is introduced before the classifier to adaptively recalibrate the features of the candidate regions.
[0029] Design a context information fusion mechanism to incorporate image information around the candidate area into the classification decision; use the trained classifier to classify the screened candidate area to determine whether it contains defects;
[0030] Calculate a confidence score for each candidate area determined to be a defect, and sort or further process the defects based on the confidence score;
[0031] Review the classification results, identify the defects of false detection and missed detection, and analyze their causes; based on the results of false detection and missed detection analysis, adjust the parameters of the region proposal network, optimize the structure of the classifier, or introduce new feature representation methods.
[0032] Furthermore, the S3 includes:
[0033] S31. Use a feature pyramid network to fuse features of different scales extracted by the deep learning model; and combine high-level semantic features with low-level detail features through top-down paths and lateral connections;
[0034] S32. Based on the feature pyramid network, a multi-scale feature fusion strategy is designed; according to the characteristics of the reducer appearance inspection task, the feature fusion method is selected;
[0035] S33. Compare the effects of different fusion methods through simulation tests and select a fusion strategy; further optimize the fused features through the attention mechanism;
[0036] S34. In the feature fusion process, context information is introduced; by considering the correlation between different regions in the image, context features are extracted and combined with local features;
[0037] S35, collecting spectral data of the reducer surface, measuring it using a spectrometer; and preprocessing the spectral data;
[0038] S36. Analyze the correlation between spectral data and visual features through statistical analysis and machine learning algorithms; obtain the intrinsic connection between spectral features and visual features, and establish a multi-dimensional appearance quality assessment model;
[0039] S37. Apply machine learning algorithms to establish a mapping relationship between spectral features and appearance quality; through training models, learn the nonlinear relationship between spectral features and appearance quality, and obtain appearance quality prediction results based on spectral data.
[0040] Furthermore, the S31 includes:
[0041] Based on the appearance characteristics of the reducer, a multi-level feature extraction network was designed. The network depth was selected to balance computing resources with feature representation capabilities, and the number and size of convolution kernels were adjusted.
[0042] The resolution of high-level feature maps is increased to match that of low-level feature maps through upsampling; at each level, the upsampled high-level features are fused with the corresponding low-level features through horizontal connections;
[0043] A channel attention module is embedded in the feature fusion process, and the weights of different channels or spatial positions in the feature map are adaptively adjusted through the attention module.
[0044] Furthermore, the S36 includes:
[0045] Remove noise and outliers from spectral data and normalize visual features extracted from deep learning models;
[0046] According to the acquisition location and time of the reducer appearance image and spectral data, data alignment is performed, and the linear or nonlinear correlation between the spectral data and visual features is analyzed using the Pearson correlation coefficient;
[0047] Random forest was used to evaluate the importance of each feature to the appearance quality of the reducer. Based on the results of feature selection, principal component analysis (PCA) was used to reduce the dimension of the features.
[0048] The data is fused through various fusion strategies, and the key parameters of the fusion strategies are tuned. The performance of different fusion strategies on the reducer appearance quality assessment task is evaluated, and a deep learning model is selected based on the characteristics of the task.
[0049] The spectral data and visual features are embedded into the hidden layers of the deep learning model respectively, and the features are deeply fused through the attention mechanism;
[0050] An end-to-end training method is used to simultaneously optimize the spectral data processing module and the visual feature processing module.
[0051] Furthermore, the S4 includes:
[0052] S41. Analyze the fused features in real time and apply a classification algorithm to determine the appearance quality grade of the reducer.
[0053] S42. Predict potential faults and their development trends based on historical data and current test results using a fault prediction model; conduct a comprehensive analysis of the classification results and fault prediction results;
[0054] S43. Automatically generate a comprehensive test report based on the results of real-time evaluation and analysis;
[0055] S44. Generate preventive maintenance recommendations based on the results of the fault prediction model; and continuously adjust and optimize the maintenance plan based on real-time detection data and historical maintenance records.
[0056] The present invention proposes a system for implementing the reducer appearance inspection method based on deep learning, the system comprising:
[0057] Image acquisition module: Use high-precision imaging module to perform comprehensive and multi-angle image acquisition of the reducer and pre-process the acquired images;
[0058] Feature extraction module: This module applies a deep learning model to perform preliminary feature extraction on preprocessed images, identifies appearance defects, and uses an adaptive learning mechanism to dynamically adjust model parameters based on emerging fault modes.
[0059] Feature fusion module: This module uses a feature pyramid network to fuse features at different scales and combines spectral analysis data with visual features to evaluate appearance quality.
[0060] Report generation module: Analyzes the assessment results in real time, automatically generates comprehensive inspection reports, and generates preventive maintenance recommendations based on historical data and current inspection reports through fault prediction models.
[0061] The beneficial effects of the present invention are as follows: through the high-precision imaging module, images are comprehensively collected from multiple angles and preprocessed to ensure image quality, laying the foundation for subsequent accurate detection; the deep learning model is used for feature extraction and defect recognition, combined with the adaptive learning mechanism, known defects can be identified, and the model can be continuously optimized by learning new fault modes; the feature pyramid network is used for multi-scale feature fusion, combined with spectral analysis data, so that the method can more comprehensively evaluate the appearance quality from different scales, thereby improving the accuracy of detection; the channel attention module is embedded in the feature pyramid network, which can adaptively adjust the weights of different channels or spatial positions in the feature map, enhance important features, suppress unimportant features, and improve detection efficiency; context information is introduced in the feature fusion process Information is collected, enabling the model to understand the correlation between different areas in the image, thereby more accurately identifying defects; combining spectral data and visual features, a multi-dimensional appearance quality assessment model is established, so that the detection method is not limited to the visual information on the image surface, but can also analyze quality problems inside the material from a spectral perspective; it can perform real-time analysis of the evaluation results and automatically generate comprehensive inspection reports to provide timely information for production management and quality control; through the fault prediction model, based on historical data and current inspection reports, it can not only discover existing problems, but also predict potential failures and make preventive maintenance suggestions, which helps to reduce downtime and extend equipment service life; this method can continuously adjust and optimize maintenance plans based on real-time inspection data and historical maintenance records, realizing closed-loop management of inspection and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 A diagram showing the steps of the method of the present invention;
[0063] Figure 2 This is a system module diagram of the present invention. DETAILED DESCRIPTION
[0064] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein may be combined with each other.
[0065] The following description sets forth numerous specific details to facilitate a thorough understanding of the present invention. The embodiments described are merely a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.
[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0067] One embodiment of the present invention, as Figure 1 As shown, a method for reducing gearbox appearance detection based on deep learning, the method includes:
[0068] S1. Using a high-precision imaging module, perform comprehensive, multi-angle image acquisition of the reducer, and pre-process the acquired images; the pre-processing includes denoising, contrast enhancement, and color correction;
[0069] S2. Apply deep learning models to perform preliminary feature extraction on preprocessed images, identify appearance defects, and use adaptive learning mechanisms to dynamically adjust model parameters based on emerging failure modes;
[0070] S3 uses a feature pyramid network (FPN) to fuse features of different scales and evaluates appearance quality by combining spectral analysis data with visual features.
[0071] S4. Analyze the assessment results in real time and automatically generate a comprehensive inspection report containing a detailed fault description, repair recommendations, and life expectancy predictions; generate preventive maintenance recommendations based on historical data and current inspection reports using a fault prediction model.
[0072] The working principle of the above technical solution is as follows: Utilizing high-precision imaging modules (such as high-resolution cameras and 3D scanners), comprehensive, multi-angle image acquisition of the reducer is performed. This ensures that images of all key parts and details of the reducer are captured, providing a rich data foundation for subsequent analysis. The captured images are then subjected to a series of pre-processing operations, specifically:
[0073] Denoising: Remove noise from the image, such as random pixels and image sensor noise, to make the image clearer.
[0074] Enhance contrast: Adjust the contrast of the image to make the details in the image more prominent, which facilitates subsequent feature extraction and defect identification.
[0075] Color correction: Correct the color deviation of the image to ensure that the image color accurately reflects the actual appearance of the reducer and avoid misjudgment due to color distortion.
[0076] A deep learning model (such as a convolutional neural network (CNN)) is used to extract features from preprocessed images. Deep learning models can automatically learn complex features in images, providing strong support for subsequent defect identification. Leveraging an adaptive learning mechanism, the model dynamically adjusts its parameters to adapt to the new data distribution when encountering new fault modes. The model continuously learns and improves its recognition capabilities, enhancing detection accuracy and robustness. A feature pyramid network (FPN) is used to fuse features at different scales. The FPN leverages multi-scale feature information to improve the detection of defects of varying sizes and locations. By fusing features at different levels, the model achieves a more comprehensive understanding of image content, improving detection accuracy and efficiency. In addition to visual features, spectral analysis data (such as infrared and ultraviolet spectra) is combined to further assess the appearance quality of the reducer. Spectral analysis data provides additional information on material properties and surface condition, helping to more accurately determine the health of the reducer. Real-time analysis of the assessment results allows for rapid identification of cosmetic defects, wear, and other issues within the reducer. Furthermore, a fault prediction model is used to predict the future condition of the reducer based on historical data and current inspection reports. Automatically generate a comprehensive inspection report containing a detailed fault description, repair recommendations, and life expectancy prediction. This report not only identifies the specific problem with the reducer but also provides targeted repair suggestions, helping users take timely action to prevent further problems. Furthermore, by predicting the reducer's life expectancy, users can rationally plan maintenance and reduce costs.
[0077] The effects of the above technical solutions are: using high-precision imaging modules, it is possible to capture tiny details and defects on the reducer surface, providing high-quality image data for subsequent image processing and analysis; the deep learning model has powerful feature extraction and pattern recognition capabilities, which can automatically learn and identify various appearance defects, improving the accuracy and reliability of detection; by fusing features of different scales through FPN, it can capture information in the image more comprehensively, further improving the accuracy of detection; the model can dynamically adjust parameters according to emerging fault modes, which means that as the design and manufacturing technology of reducers continue to develop, the detection model can continue to learn and evolve, and maintain sensitivity to new fault modes; combined with spectral analysis data, it can provide more dimensional information, enhance the comprehensive judgment ability of the detection system, and make it more flexible to deal with the appearance inspection of reducers under different materials and different working conditions; reduce the need for manual intervention and improve Improved detection efficiency; the system can automatically analyze the evaluation results in real time, and generate a comprehensive detection report containing detailed fault descriptions, maintenance suggestions and expected life predictions, which greatly improves the automation and efficiency of detection; through the fault prediction model, preventive maintenance suggestions are generated based on historical data and current detection reports, which can help companies plan maintenance plans in advance and avoid downtime losses and increased repair costs caused by sudden failures; timely detection and treatment of appearance defects can effectively prevent the expansion of defects from causing more serious damage to the reducer, thereby extending the overall service life of the equipment; detailed fault descriptions, accurate maintenance suggestions and expected life predictions can provide users with high-quality detection service experience, enhance users' trust and satisfaction with the company's brand; this technical solution demonstrates the company's professional capabilities and technological innovation strength in the field of reducer detection, and helps to enhance the company's brand image and market competitiveness.
[0078] In one embodiment of the present invention, the S2 includes:
[0079] S21. Select a deep learning model and pre-train the deep learning model using a labeled reducer appearance image dataset, including samples of normal states and multiple known defect types, to enhance the model's basic recognition capabilities.
[0080] S22, inputting the preprocessed image into a pre-trained deep learning model, and automatically extracting multi-level features in the image through a convolutional layer, wherein the multi-level features include edges, textures, and shapes;
[0081] S23, generating possible defect candidate regions on the feature map through a region proposal network (RPN), and determining whether the region contains a defect through a classifier;
[0082] S24. If defects are present, the identified defects are sub-classified using a hierarchical classification strategy, such as cracks, wear, rust, etc.
[0083] S25. Integrate an online learning module to automatically collect and annotate samples of emerging failure modes or misidentification cases, and dynamically adjust deep learning model parameters using incremental learning.
[0084] The working principle of the above technical solution is as follows: First, based on the specific requirements of reducer appearance inspection, an appropriate deep learning model is selected, such as a convolutional neural network (CNN) or more advanced variants (such as ResNet and VGG). These models have demonstrated strong performance in image recognition. The deep learning model is pre-trained using a dataset of labeled reducer appearance images. This dataset should include images of reducers in normal conditions as well as samples of various known defect types (such as cracks, wear, and rust). Through pre-training, the model learns these basic features and patterns, thereby improving its basic recognition capabilities. The images, which have undergone pre-processing steps such as denoising, contrast enhancement, and color correction, are then fed into the pre-trained deep learning model. The convolutional layers in the model automatically extract multi-level features from the images. These features, including edges, textures, and shapes, together form a high-level representation of the image, facilitating subsequent defect detection. Based on the feature maps, the RPN is used to generate potential defect candidate regions. The RPN can quickly scan the entire feature map and predict regions that may contain defects and their corresponding confidence scores. For each candidate region, a classifier (such as a softmax classifier) is used to determine whether it actually contains a defect. This step helps filter out non-defective regions, reducing the computational complexity of subsequent processing. If a candidate region is determined to contain a defect, a hierarchical classification strategy is used to further refine the defect classification. This step typically involves multiple classifiers or classification layers, which gradually perform more detailed defect classification, such as distinguishing between different types of defects such as cracks, wear, and rust. An integrated online learning module collects samples of emerging fault modes or misidentification cases encountered during the inspection process in real time and automatically annotates them (or prompts manual annotation). Incremental learning techniques are used to dynamically adjust the parameters of the deep learning model. Incremental learning allows the model to gradually adapt to new fault modes while maintaining its ability to recognize old ones. In this way, the model's recognition ability and robustness will continue to improve as the inspection process continues.
[0085] The effect of the above technical solution is: using a labeled reducer appearance image dataset for pre-training, the deep learning model can learn the basic features of the normal state and multiple known defect types, thereby improving the basic recognition ability of the model.
[0086] Multi-level feature extraction: Convolutional layers automatically extract multi-level features (edges, textures, shapes, etc.) from images. These features provide a more comprehensive description of image content, facilitating more accurate defect identification. Identified defects are sub-classified (e.g., cracks, wear, rust, etc.), further improving identification accuracy and specificity, facilitating subsequent repair and maintenance. An integrated online learning module collects and annotates samples of emerging fault modes or misidentification cases in real time. This enables the model to continuously learn and adapt to new data distributions, enhancing its adaptability and flexibility. Incremental learning techniques are used to dynamically adjust deep learning model parameters, enabling the model to gradually adapt to new fault modes while maintaining its ability to recognize existing ones. This capability is particularly important for handling complex and changeable reducer appearance defect detection scenarios; from image preprocessing, feature extraction, defect candidate area generation, defect judgment to sub-classification, the entire detection process is highly automated, reducing the need for manual intervention and improving detection efficiency; it can analyze the evaluation results in real time and automatically generate a comprehensive detection report, so that the detection results can be quickly fed back to relevant personnel so that timely measures can be taken; by accurately identifying reducer appearance defects and generating preventive maintenance recommendations, it helps companies plan maintenance plans in advance and avoid downtime losses and increased repair costs caused by sudden failures; timely detection and treatment of appearance defects can effectively prevent the expansion of defects from causing more serious damage to the reducer, thereby extending the overall service life of the equipment; providing accurate detection results and detailed maintenance recommendations improves user satisfaction and trust in the company's services; the use of advanced deep learning technology and online learning mechanisms demonstrates the company's strength in technological innovation and helps enhance the company's brand image and market competitiveness.
[0087] In one embodiment of the present invention, the step S22 includes:
[0088] Adaptive median filtering or bilateral filtering algorithms are used to dynamically adjust filtering parameters based on local image features to remove random noise while retaining edge details.
[0089] Apply the Retinex algorithm or a deep learning-based illumination normalization model to perform global and local illumination adjustments on the image to reduce the impact of illumination changes on feature extraction;
[0090] Using histogram equalization or its improved methods (such as contrast-limited adaptive histogram equalization (CLAHE)) to enhance the contrast of the image;
[0091] Construct an image pyramid and apply convolutional layers at different scales to extract multi-scale features from coarse to fine;
[0092] Design a feature fusion module to fuse features extracted by convolutional layers of different scales; introduce residual connections in the convolutional layers;
[0093] Principal component analysis (PCA) is used to evaluate the importance of features, identify and retain the most critical features for defect detection, and perform feature sparsification through L1 regularization.
[0094] The working principle of the above technical solution is as follows: adaptive median filtering or bilateral filtering algorithms dynamically adjust filtering parameters based on local image features (such as pixel value differences and texture complexity). These algorithms can effectively preserve image edge details while removing random noise, laying a good foundation for subsequent feature extraction. The Retinex algorithm or a deep learning-based illumination normalization model is used to perform global and local illumination adjustments on the image. The Retinex algorithm, based on color constancy theory, simulates the human eye's adaptability to illumination changes, reducing the impact of illumination changes on image content. The deep learning-based illumination normalization model automatically adjusts image illumination to a more uniform level by learning from a large amount of data, facilitating subsequent feature extraction. Histogram equalization or its improved methods (such as CLAHE) are used to enhance image contrast. These methods adjust the image's histogram distribution to make the pixel values more uniform, thereby enhancing the image's detail and making defects easier to detect. An image pyramid is constructed, and convolutional layers are applied at different scales for feature extraction. This approach captures multi-scale information in images, from coarse to fine scales, enabling the model to detect defects of varying sizes. A feature fusion module is designed to fuse features extracted by convolutional layers at different scales. Using techniques such as the attention mechanism or DenseNet, the importance of features at each scale is automatically learned and effectively integrated, improving the expressiveness and robustness of the features. This allows the model to comprehensively consider information from different scales and make more accurate judgments. Residual connections (such as the ResNet architecture) are introduced into convolutional layers, allowing the network to directly learn the residual between input and output. This structure helps alleviate the vanishing or exploding gradient problem in deep network training, improving the accuracy and efficiency of feature extraction. Principal component analysis (PCA) is used to assess feature importance, identifying and retaining the most critical features for defect detection. Furthermore, L1 regularization is used to sparsify features, reducing redundant information and improving feature compactness and generalization. This allows the model to focus more on learning features useful for defect detection during training, improving detection performance.
[0095] The above technical solutions achieve the following: Adaptive median filtering or bilateral filtering algorithms effectively remove random noise from images while preserving edge details. This is crucial for subsequent feature extraction and defect detection, as reducing noise improves feature quality, while preserving edge details helps accurately identify object boundaries and potential defects in the image. Using the Retinex algorithm or a deep learning-based illumination normalization model, the model automatically adjusts global and local illumination in the image. This significantly reduces the impact of illumination variations on feature extraction, enabling the model to maintain stable performance under varying lighting conditions. Illumination normalization helps the model focus more on image content rather than variations in illumination conditions. Using histogram equalization or its improved methods (such as CLAHE), image contrast is significantly enhanced, making details more visible. This is particularly important for defect detection, as the enhanced detail information helps detect otherwise subtle defects. By constructing an image pyramid and applying convolutional layers at different scales, multi-scale features are extracted, from coarse to fine scales. This multi-scale feature extraction strategy captures information at different scales within the image, thereby improving the model's ability to detect defects of varying sizes. In addition, a feature fusion module is designed to fuse features of different scales, further improving the expressiveness and robustness of features. The introduction of residual connections (such as the ResNet structure) in convolutional layers effectively alleviates the problem of vanishing or exploding gradients in deep network training. This enables more stable network training and improves the accuracy and efficiency of feature extraction. Residual connections also promote the flow of information, allowing the network to better utilize deep features. Principal component analysis (PCA) is used to assess the importance of features and identify the most critical features for defect detection. Features are sparsified through L1 regularization, reducing redundant information and improving feature compactness and generalization capabilities. This feature optimization strategy helps reduce model complexity, improve computational efficiency, and enhance the model's generalization capabilities on unseen data.
[0096] In one embodiment of the present invention, the step S23 includes:
[0097] Based on the shapes and sizes of common defects in reducer appearance images, a variety of anchor boxes are designed. At the same time, an adaptive anchor box generation mechanism is introduced to dynamically adjust the size and proportion of the anchor box according to the image content.
[0098] Using the Feature Pyramid Network (FPN) structure, feature maps of different scales are fused, enabling the region proposal network to detect defects at multiple scales.
[0099] The region proposal network slides a window on the feature map to generate a large number of preliminary defect candidate regions. Non-maximum suppression (NMS) is used to remove redundant and overlapping candidate regions based on their scores and overlap.
[0100] Based on the complexity and real-time requirements of defect classification, select an appropriate classifier architecture, such as convolutional neural network (CNN), recurrent neural network (RNN), or a combination of the two; introduce a feature recalibration module (such as SEBlock in SENet) before the classifier to adaptively recalibrate the features of the candidate area.
[0101] Considering the relationship between defects and the surrounding environment, a context information fusion mechanism is designed to incorporate image information around the candidate area into the classification decision. The trained classifier is used to classify the screened candidate area to determine whether it contains defects.
[0102] A confidence score is calculated for each candidate region identified as a defect. This score reflects the classifier's degree of certainty that the region contains a defect. Based on the confidence score, the defects are sorted or further processed.
[0103] Review the classification results, identify the defects of false detection and missed detection, and analyze their causes; based on the results of false detection and missed detection analysis, adjust the parameters of the region proposal network, optimize the structure of the classifier, or introduce new feature representation methods.
[0104] The working principle of the above technical solution is as follows: Based on the shapes and sizes of common defects in reducer appearance images, a variety of anchor boxes (AnchorBoxes) are pre-designed. These anchor boxes serve as the initial shape and size of defect candidate regions, helping to improve recall rate. An adaptive anchor box generation mechanism is introduced to dynamically adjust the size and proportion of the anchor boxes based on image content. This mechanism more accurately matches the actual defect shape in the image, further improving detection accuracy. Feature Pyramid Network (FPN) architecture is used to fuse feature maps of different scales. Through upsampling and lateral connections, FPN combines high-level semantic information with low-level detail information, enabling the Region Proposal Network (RPN) to perform defect detection at multiple scales. The FPN architecture enables the RPN to slide a window across feature maps of different scales to generate a large number of preliminary defect candidate regions. This multi-scale detection strategy improves sensitivity to defects of varying sizes. The RPN slides a window across the feature map to generate a large number of preliminary defect candidate regions. A non-maximum suppression algorithm is used to remove redundant and overlapping candidate regions based on their scores and overlap, retaining the regions most likely to contain defects. This step reduces the computational burden of subsequent classifiers and improves detection accuracy. Based on the complexity and real-time requirements of defect classification, an appropriate classifier architecture is selected, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a combination of the two. A feature recalibration module (such as SEBlock in SENet) is introduced before the classifier to adaptively recalibrate the features of the candidate regions. This step enhances the expressive power of key features, suppresses useless features, and improves classification accuracy. A contextual information fusion mechanism is designed to incorporate image information surrounding the candidate region into the classification decision. This mechanism considers the relationship between the defect and its surroundings, helping to more accurately determine whether a candidate region contains a defect. A confidence score is calculated for each candidate region identified as a defect, reflecting the classifier's degree of certainty that the region contains a defect. Classification results are reviewed to identify falsely detected and missed defects and analyze their causes. Based on the review results, the parameters of the region proposal network can be adjusted, the classifier structure can be optimized, or new feature representation methods can be introduced to further improve detection accuracy and robustness.
[0105] The above technical solution achieves the following results: Based on the shapes and sizes of common defects in reducer appearance images, diverse anchor boxes (AnchorBoxes) are designed to more comprehensively cover possible defect morphologies, thereby increasing the probability of initial defect detection. An adaptive anchor box generation mechanism is introduced to dynamically adjust the size and proportion of the anchor boxes based on the image content, ensuring that the anchor boxes more closely match the actual defect shape, further improving the recall rate. Feature maps at different scales are fused using the FPN architecture, enabling the Region Proposal Network (RPN) to detect defects at multiple scales. This multi-scale detection mechanism improves sensitivity to defects of varying sizes, particularly for small and difficult-to-detect defects. The NMS algorithm removes redundant and overlapping candidate regions based on their scores and overlap, retaining the regions most likely to contain defects. This step effectively reduces the computational burden of the subsequent classifier and improves classification accuracy. A feature recalibration module (such as SEBlock in SENet) is introduced before the classifier to adaptively recalibrate the features of the candidate regions, enhancing the representation of key features and suppressing useless features. This mechanism helps improve the classifier's ability to identify defect features. A contextual information fusion mechanism is designed to incorporate image information surrounding the candidate region into the classification decision. This helps the classifier more comprehensively understand the environment in which the candidate region is located, thereby improving classification accuracy. A confidence score is calculated for each candidate region determined to be a defect. This score reflects the classifier's degree of certainty that the region contains a defect. This step provides an important reference for subsequent defect processing. The classification results are reviewed to identify falsely detected and missed defects and analyze their causes. This helps to continuously optimize and adjust the detection scheme and improve the accuracy and reliability of the detection results. Based on the results of the false detection and missed detection analysis, the parameters of the region proposal network are adjusted, the structure of the classifier is optimized, or new feature representation methods are introduced. This continuous optimization and iteration mechanism ensures that the detection scheme can continuously adapt to new application scenarios and defect forms.
[0106] In one embodiment of the present invention, S3 includes:
[0107] S31. Use Feature Pyramid Network (FPN) to fuse features of different scales extracted by deep learning models; and combine high-level semantic features with low-level detail features through top-down paths and lateral connections;
[0108] S32. Based on the feature pyramid network, a multi-scale feature fusion strategy is designed; according to the characteristics of the reducer appearance inspection task, the feature fusion method is selected.
[0109] S33, and compare the effects of different fusion methods through simulation tests, select the optimal fusion strategy; and further optimize the fused features through the attention mechanism;
[0110] S34. In the feature fusion process, context information is introduced; by considering the correlation between different regions in the image, context features are extracted and combined with local features;
[0111] S35, collect the spectral data of the reducer surface and measure it using a high-precision spectrometer. And pre-process the spectral data.
[0112] S36. Analyze the correlation between spectral data and visual features through statistical analysis and machine learning algorithms; obtain the intrinsic connection between spectral features and visual features, and establish a multi-dimensional appearance quality assessment model;
[0113] S37. Apply machine learning algorithms, such as support vector machines (SVM) or random forests, to establish a mapping relationship between spectral features and appearance quality. By training the model, the nonlinear relationship between spectral features and appearance quality can be learned to achieve appearance quality prediction based on spectral data.
[0114] The working principle of the above technical solution is to use FPN to fuse features of different scales extracted by the deep learning model. FPN combines high-level semantic features with low-level detailed features through a top-down pathway and lateral connections. High-level features are rich in semantic information and help identify object categories, while low-level features contain more detailed information and help accurately locate object locations. This fusion approach makes the model more robust in detecting defects of different scales, capturing both global and local information. A multi-scale feature fusion strategy is designed based on the characteristics of the reducer appearance inspection task. Strategies may include feature weighting, feature concatenation, and feature selection. Each strategy has its applicable scenarios and advantages and disadvantages. Simulation tests are conducted to compare the performance of different fusion methods and select the optimal one. This step is crucial to ensuring the performance of the final inspection system. After feature fusion, an attention mechanism is introduced to further optimize the features. The attention mechanism simulates the focusing process of the human visual system, weighting important areas in the feature map to highlight key features and suppress irrelevant features. Optimizing the attention mechanism can further improve detection accuracy, allowing the model to focus more on features that are critical for determining defect type. Contextual information is incorporated into the feature fusion process. By considering the correlation between different regions in the image, contextual features are extracted and combined with local features. The inclusion of contextual information helps the model better understand the overall structure and background information of the image, thereby improving detection accuracy and robustness. Spectral data from the reducer surface is collected using a high-precision spectrometer. Spectral data can reflect the chemical composition and physical state of the material surface, providing important information for appearance quality assessment. Preprocessing steps such as denoising, baseline correction, and feature extraction are performed on the spectral data to improve the signal-to-noise ratio and feature separability. Statistical analysis and machine learning algorithms are used to analyze the correlation between spectral data and visual features. The intrinsic relationship between spectral and visual features is then extracted to establish a multidimensional appearance quality assessment model. This multidimensional assessment model can more comprehensively reflect the appearance quality of the reducer. Machine learning algorithms (such as support vector machines or random forests) are used to establish a mapping between spectral features and appearance quality. The model is trained to learn the nonlinear relationship between spectral features and appearance quality. Ultimately, the model can predict the appearance quality of the reducer based on spectral data, providing powerful support for preventive maintenance and fault warning.
[0115] The effects of the above technical solution are as follows: by fusing features of different scales through FPN, combining high-level semantic features with low-level detail features, the model becomes more accurate and robust when detecting defects of different sizes and shapes. This fusion method helps to capture more comprehensive image information and improve detection accuracy; according to the characteristics of the reducer appearance inspection task, designing and selecting appropriate feature fusion methods (such as feature weighting, splicing, and selection) can further optimize feature representation and improve the generalization ability and detection accuracy of the model; weighting the fused features through the attention mechanism to highlight key features and suppress irrelevant features, so that the model is more focused on areas that have a significant impact on the inspection task, thereby improving detection accuracy; introducing contextual information in the feature fusion process and considering the correlation between different areas in the image helps the model understand the overall structure and background information of the image, and improves the comprehensive judgment ability of the reducer appearance quality; collecting spectral data of the reducer surface, and measuring and preprocessing it with a high-precision spectrometer to improve the signal-to-noise ratio and feature separability of the data. Spectral data can reveal the chemical composition and physical state of a material's surface, providing unique and crucial information for appearance quality assessment. By using statistical analysis and machine learning algorithms to analyze the correlation between spectral data and visual features, a multidimensional appearance quality assessment model is established. This multidimensional assessment approach provides a more comprehensive picture of the reducer's appearance quality. Machine learning algorithms (such as SVM and random forests) are applied to establish a mapping between spectral features and appearance quality. By learning the nonlinear relationship between spectral features and appearance quality, appearance quality prediction based on spectral data is achieved. This approach is not only fast and efficient, but also provides more objective and accurate assessment results. The entire inspection process is highly automated, from feature extraction and fusion to spectral data acquisition, processing, model training, and prediction, all performed automatically on a computer, reducing the need for manual intervention and improving inspection efficiency and accuracy. By integrating information from visual features and spectral data, the model can make more intelligent decisions, providing a scientific basis for reducer maintenance, repair, and replacement.
[0116] In one embodiment of the present invention, the step S31 includes:
[0117] Based on the complex appearance of the reducer (such as gears of varying sizes and bearing seats), a multi-level feature extraction network was designed. The network depth was selected to balance computing resources with feature representation capabilities, and the number and size of convolution kernels were adjusted.
[0118] The resolution of high-level feature maps is increased to match that of low-level feature maps through upsampling (such as bilinear interpolation or transposed convolution); at each level, the upsampled high-level features are fused with the corresponding low-level features through lateral connections (element-wise sum or concatenation), and weight distribution is considered during fusion to balance the contribution of features at different levels.
[0119] A channel attention module is embedded in the feature fusion process, and the weights of different channels or spatial positions in the feature map are adaptively adjusted through the attention module.
[0120] The working principle of the above technical solution is as follows: Given the complexity of the reducer's appearance, such as gears of varying sizes and bearing seats, a multi-level feature extraction network was designed. This network structure ensures that features at all scales, from coarse to fine, are effectively captured to meet the detection requirements for defects of varying sizes and shapes. During the design process, a balance was struck between computational resources and feature representation capabilities. By selecting an appropriate network depth and adjusting the number and size of convolution kernels, sufficient feature extraction capabilities were ensured while avoiding unnecessary computational overhead. Upsampling techniques (such as bilinear interpolation or transposed convolution) were used to increase the resolution of high-level feature maps to match that of lower-level feature maps. This step ensures spatial alignment of features during feature fusion, making the fusion process more efficient. At each level, the upsampled high-level features are fused with the corresponding low-level features. The fusion method can be element-wise summation or concatenation, depending on the task requirements and best practices for feature representation. During the fusion process, weight distribution is also considered to balance the contributions of features at different levels, ensuring that the fused features contain rich semantic information while retaining sufficient detail information. A channel attention module is embedded in the feature fusion process. This module can adaptively adjust the weights of different channels or spatial locations in the feature map, allowing the model to focus more on feature regions that are important for the detection task. Through the attention mechanism, the model can simulate the focusing process of the human visual system, highlighting key features and suppressing irrelevant features, thereby improving detection accuracy. After the fusion and optimization of the above steps, the feature pyramid network can generate a series of feature maps that fuse information at different scales. These feature maps retain high-level semantic information while also incorporating low-level detail information, providing richer feature representations for subsequent detection or classification tasks.
[0121] The above technical solution achieves the following: Designing a multi-level feature extraction network ensures that features at different scales, from coarse to fine, are effectively captured. This is particularly important for objects with complex appearances, such as reducers, as features at different scales are crucial for detecting defects of varying sizes and shapes. By properly adjusting the network depth, number of convolution kernels, and sizes, this solution achieves a good balance between computational resources and feature representation capabilities. This ensures sufficient feature extraction capabilities while avoiding unnecessary computational overhead, improving overall detection efficiency. Upsampling is used to increase the resolution of high-level feature maps to match that of low-level feature maps, ensuring spatial alignment of features. This step is crucial for feature fusion, as only aligned features can be effectively fused, preserving more useful information. At each level, the upsampled high-level features are fused with the corresponding low-level features, and weight distribution is used to balance the contributions of features from different levels. This fusion approach preserves high-level semantic information while incorporating low-level details, making the fused features richer and more comprehensive. A channel attention module is embedded in the feature fusion process to adaptively adjust the weights of different channels or spatial locations in the feature map. This allows the model to focus more on feature areas that are crucial to the inspection task, thereby improving both its focus and detection accuracy. By integrating multi-level features and adaptively adjusting weights, this solution makes the model more robust to subtle changes in the reducer's appearance or occlusions. Even in complex environments, the model can accurately detect defects or anomalies. Effectively extracted and fused features provide strong support for subsequent inspection, classification, and evaluation tasks. This rich feature representation enables subsequent tasks to be performed more accurately and efficiently.
[0122] In one embodiment of the present invention, the step S36 includes:
[0123] Remove noise and outliers from spectral data, such as fluctuations caused by instrument errors or measurement environment, and normalize visual features extracted from deep learning models.
[0124] Based on the acquisition location and time of the reducer appearance image and spectral data, accurate data alignment is performed, and the linear or nonlinear correlation between the spectral data and visual features is analyzed using the Pearson correlation coefficient;
[0125] Random forest was used to evaluate the importance of each feature to the appearance quality of the reducer. Based on the results of feature selection, principal component analysis (PCA) was used to reduce the dimension of the features.
[0126] The data is fused through various fusion strategies, such as early fusion (fusion in the feature extraction stage), mid-term fusion (fusion in the feature processing stage) and late fusion (fusion in the decision stage), and the key parameters in the fusion strategy are tuned, such as fusion weights, fusion layer positions, etc.
[0127] The performance of different fusion strategies in the reducer appearance quality assessment task is evaluated, including indicators such as accuracy, recall rate, and F1 score; deep learning models are selected according to the characteristics of the task, such as convolutional neural networks (CNN) for processing visual features, and recurrent neural networks (RNN) or long short-term memory networks (LSTM) for processing the temporal information in spectral data.
[0128] The spectral data and visual features are embedded into the hidden layers of the deep learning model respectively, and the features are deeply fused through the attention mechanism;
[0129] An end-to-end training method is used to simultaneously optimize the spectral data processing module and the visual feature processing module, as well as the fusion method between them.
[0130] The working principle of the above technical solution is as follows: First, noise and outliers in the spectral data are removed. This noise may be caused by instrument errors or changes in the measurement environment. Simultaneously, the visual features extracted from the deep learning model are normalized to ensure consistency in magnitude across different features, providing clean, comparable data for subsequent analysis. Precise data alignment is performed based on the acquisition location and time of the reducer appearance image and spectral data. This is a key step in ensuring the accuracy of the correspondence between the two, providing a foundation for subsequent correlation analysis and feature fusion. The Pearson correlation coefficient is used to analyze the linear or nonlinear correlation between the spectral data and the visual features. A random forest algorithm is used to assess the importance of each feature to the reducer appearance quality. This step identifies key features that contribute significantly to the evaluation task. Based on the results of feature selection, PCA is used to reduce the feature dimensionality. PCA retains the main directions of variation in the data (i.e., principal components) and removes redundant information, thereby reducing computational complexity and preserving key information. Data fusion is performed using various fusion strategies, including early, mid-term, and late fusion. Each fusion strategy has its own unique advantages and applicable scenarios. By adjusting key parameters in the fusion strategy (such as fusion weights and fusion layer positions), the fusion effect can be optimized. The performance of different fusion strategies on the reducer appearance quality assessment task is evaluated, including metrics such as accuracy, recall, and F1 score. The appropriate deep learning model is selected based on the specifics of the task. For example, a CNN is used to process visual features, leveraging its powerful spatial feature extraction capabilities. An RNN or LSTM is used to process the temporal information in spectral data to capture temporal dependencies in the data. Spectral data and visual features are embedded separately in the hidden layers of the deep learning model. Deep feature fusion using an attention mechanism enables the model to simultaneously utilize information from both data sources for decision making. An end-to-end training approach is employed to simultaneously optimize the spectral data processing module, the visual feature processing module, and the fusion method between them. This training approach helps achieve optimal fusion effects and overall performance.
[0131] The effects of the above technical solutions are as follows: by removing noise and outliers in spectral data and visual features, the quality of data can be significantly improved, and the impact of instrument errors or changes in the measurement environment on the analysis results can be reduced; the visual features are normalized to ensure the consistency of the magnitude of different features, providing a more stable and reliable data basis for subsequent model training and data analysis; the reducer appearance image and the spectral data are accurately aligned according to the acquisition position and time, ensuring the accurate correspondence between the two, providing a solid foundation for subsequent correlation analysis and fusion strategies; the importance of each feature is evaluated using the random forest algorithm, which helps to identify the most critical features for the reducer appearance quality assessment task; the feature dimensionality reduction processing through PCA not only reduces the computational complexity, but also retains the key information in the data, improving the generalization ability and prediction accuracy of the model; provides early There are various fusion strategies such as fusion, mid-term fusion and late fusion. The appropriate fusion method can be selected according to the specific task requirements to achieve effective data fusion; the key parameters in the fusion strategy are tuned, such as fusion weights, fusion layer positions, etc., which can further optimize the fusion effect and improve the performance of the evaluation task; according to the characteristics of the task, a suitable deep learning model is selected, such as CNN for processing visual features, RNN or LSTM for processing time series information in spectral data, giving full play to the advantages of different models in processing different types of data; the spectral data and visual features are respectively embedded in the hidden layer of the deep learning model, and the features are deeply fused through the attention mechanism, realizing the effective integration and utilization of the information of the two data sources; an end-to-end training method is adopted to simultaneously optimize the spectral data processing module, the visual feature processing module and the fusion method between them, so as to obtain the best fusion effect and overall performance. This training method not only improves the training efficiency, but also ensures the stability and reliability of the model in practical applications. In one embodiment of the present invention, the S4 includes:
[0132] S41. Perform real-time analysis on the fused features and apply classification algorithms (such as SVM, neural network, etc.) to determine the appearance quality grade of the reducer;
[0133] S42. Use a fault prediction model to predict potential faults and their development trends based on historical data and current test results. The fault prediction model should use methods such as time series analysis and machine learning, taking into account factors such as the frequency, severity, and development trend of faults, to provide fault warnings and estimated remaining service life. Conduct a comprehensive analysis of the classification results and fault prediction results; the fault prediction results are obtained using the following formula:
[0134]
[0135] in, is the fault prediction result at time t, N is the number of different prediction models, is the i-th prediction model The weight of , is the predicted value of the i-th prediction model at time t, is the trend adjustment coefficient based on time series analysis, is the error term, which represents the uncertainty of the prediction.
[0136] S43. Based on the results of real-time evaluation and analysis, a comprehensive inspection report is automatically generated, including detailed fault description, fault location marking, repair suggestions, and life expectancy prediction;
[0137] S44. Based on the results of the fault prediction model, it generates targeted preventive maintenance recommendations; and continuously adjusts and optimizes maintenance plans based on real-time detection data and historical maintenance records.
[0138] The working principle of this technical solution is as follows: First, the fused features are analyzed in real time. These features contain various information about the reducer's appearance, such as shape, color, and texture. Classification algorithms (such as support vector machines and neural networks) are then applied to these features to determine the reducer's appearance quality grade. Through training, the classification algorithm can accurately distinguish between different grades, such as normal state, minor defects, and major defects, and assign corresponding confidence levels. Based on historical data and current inspection results, a fault prediction model is established using time series analysis and machine learning methods. This model considers factors such as fault frequency, severity, and development trends to provide fault warnings and estimated remaining service life. The classification results are then combined with the fault prediction results for analysis. For example, if the classification result indicates a major defect, an alarm is immediately triggered. If the fault prediction model indicates a high failure risk, preventive maintenance is recommended. Furthermore, a comprehensive assessment is provided, taking into account the impact of different defect grades and failure risks on the reducer's overall performance. Based on the results of the real-time evaluation and analysis, a comprehensive inspection report is automatically generated, including a detailed fault description, fault location, repair recommendations, and expected service life. This report is designed to provide users with comprehensive information on the reducer's status so they can make reasonable decisions. It also generates targeted preventive maintenance recommendations based on the results of the fault prediction model. These recommendations include maintenance time windows, maintenance content, maintenance methods, and required resources to ensure timely maintenance intervention before a failure occurs. Maintenance plans are continuously adjusted and optimized based on real-time detection data and historical maintenance records. When new defects or failure modes emerge, maintenance recommendations are updated promptly to address new risks. When maintenance resources change, maintenance plans are re-optimized to ensure efficient resource utilization. A maintenance recommendation history function is provided to facilitate users to track and compare changes in maintenance recommendations at different time points. This helps users understand the evolution of maintenance recommendations and make more reasonable decisions.
[0139] The above technical solution achieves the following benefits: By analyzing the fused features in real time, the system can quickly and accurately determine the reducer's appearance quality level, including normal condition, minor defects, and severe defects. This real-time nature ensures timely detection and resolution of problems. The application of classification algorithms ensures high confidence in the evaluation results, reduces the possibility of misjudgment, and improves the accuracy of the assessment. The fault prediction model predicts potential faults and their development trends based on historical data and current test results. This predictive capability enables users to take preventive measures before a fault occurs, avoiding downtime and financial losses. When the fault prediction model indicates a high risk of failure, the system recommends preventive maintenance. This proactive maintenance strategy reduces the likelihood of failure and extends the reducer's service life. The generated comprehensive inspection report includes a detailed fault description, fault location, repair recommendations, and life expectancy prediction. This comprehensive report provides users with clear information on the reducer's condition, helping them make informed decisions. Based on the results of the fault prediction model, preventive maintenance recommendations are generated, including targeted maintenance time windows, maintenance content, maintenance methods, and required resources. These recommendations provide users with clear maintenance guidelines, ensuring effective and efficient maintenance work. The system continuously adjusts and optimizes maintenance plans based on real-time inspection data and historical maintenance records. This dynamic adjustment mechanism ensures the timeliness and adaptability of maintenance plans, adapting to new defects or failure modes, as well as resource changes. By optimizing maintenance plans, the system ensures efficient resource utilization and avoids waste and overconsumption. A historical record of maintenance recommendations allows users to track and compare changes in maintenance recommendations over time. This feature enhances user engagement and trust, improving the user experience. A comprehensive assessment is provided, considering the impact of different levels of defects and failure risks on the overall performance of the reducer. This comprehensive assessment helps users fully understand the reducer's condition and improves user satisfaction. The above formula allows the output of multiple different prediction models to be combined, which helps improve prediction accuracy and robustness. Different models may capture different characteristics and patterns, and by combining this information, a more comprehensive prediction result can be obtained. Weighting factors can be introduced to ensure that important models have a greater impact on the final results, while less reliable models have a smaller impact. The weighting can be selected based on the model's historical performance or expert experience. Using a trend adjustment factor can make forecasts more consistent with actual trends, helping to correct for biases caused by short-term fluctuations and making long-term forecasts more accurate. Introducing an error term reflects the uncertainty of forecast results, which is crucial for risk management and decision-making. Understanding the confidence interval of forecast results can help make more informed decisions. By combining the outputs of multiple models, the bias of individual models can be reduced, thereby improving the accuracy of the overall forecast.In the face of abnormal data or environmental changes, the integration of multiple models can provide additional information, making the system more robust. The integration of multiple models and trend adjustments can effectively reduce false positives and missed negatives, improving the reliability of the fault prediction system. The provided prediction results and their uncertainty can assist decision makers in better planning maintenance strategies, such as scheduling preventive maintenance measures or component replacement. Combining historical data with current test results can identify potential failure signs earlier, allowing timely intervention to avert major incidents.
[0140] One embodiment of the present invention, as Figure 2 As shown, a reducer appearance inspection system based on deep learning, the system includes:
[0141] Image acquisition module: Use high-precision imaging module to perform comprehensive and multi-angle image acquisition of the reducer and pre-process the acquired images;
[0142] Feature extraction module: This module applies a deep learning model to perform preliminary feature extraction on preprocessed images, identifies appearance defects, and uses an adaptive learning mechanism to dynamically adjust model parameters based on emerging fault modes.
[0143] Feature Fusion Module: This module uses a Feature Pyramid Network (FPN) to fuse features at different scales and evaluates appearance quality by combining spectral analysis data with visual features.
[0144] Report Generation Module: Analyzes assessment results in real time and automatically generates a comprehensive inspection report containing detailed fault descriptions, repair recommendations, and life expectancy predictions. It also generates preventive maintenance recommendations based on historical data and current inspection reports using a fault prediction model.
[0145] The working principle of the above technical solution is as follows: Utilizing high-precision imaging modules (such as high-resolution cameras and 3D scanners), comprehensive, multi-angle image acquisition of the reducer is performed. This ensures that images of all key parts and details of the reducer are captured, providing a rich data foundation for subsequent analysis. The captured images are then subjected to a series of pre-processing operations, specifically:
[0146] Denoising: Remove noise from the image, such as random pixels and image sensor noise, to make the image clearer.
[0147] Enhance contrast: Adjust the contrast of the image to make the details in the image more prominent, which facilitates subsequent feature extraction and defect identification.
[0148] Color correction: Correct the color deviation of the image to ensure that the image color accurately reflects the actual appearance of the reducer and avoid misjudgment due to color distortion.
[0149] A deep learning model (such as a convolutional neural network (CNN)) is used to extract features from preprocessed images. Deep learning models can automatically learn complex features in images, providing strong support for subsequent defect identification. Leveraging an adaptive learning mechanism, the model dynamically adjusts its parameters to adapt to the new data distribution when encountering new fault modes. The model continuously learns and improves its recognition capabilities, enhancing detection accuracy and robustness. A feature pyramid network (FPN) is used to fuse features at different scales. The FPN leverages multi-scale feature information to improve the detection of defects of varying sizes and locations. By fusing features at different levels, the model achieves a more comprehensive understanding of image content, improving detection accuracy and efficiency. In addition to visual features, spectral analysis data (such as infrared and ultraviolet spectra) is combined to further assess the appearance quality of the reducer. Spectral analysis data provides additional information on material properties and surface condition, helping to more accurately determine the health of the reducer. Real-time analysis of the assessment results allows for rapid identification of cosmetic defects, wear, and other issues within the reducer. Furthermore, a fault prediction model is used to predict the future condition of the reducer based on historical data and current inspection reports. Automatically generate a comprehensive inspection report containing a detailed fault description, repair recommendations, and life expectancy prediction. This report not only identifies the specific problem with the reducer but also provides targeted repair suggestions, helping users take timely action to prevent further problems. Furthermore, by predicting the reducer's life expectancy, users can rationally plan maintenance and reduce costs.
[0150] The effects of the above technical solutions are as follows: using high-precision imaging modules, it is possible to capture tiny details and defects on the reducer surface, providing high-quality image data for subsequent image processing and analysis; the deep learning model has powerful feature extraction and pattern recognition capabilities, which can automatically learn and identify various appearance defects, improving the accuracy and reliability of detection; by fusing features of different scales through FPN, it can capture information in the image more comprehensively, further improving the accuracy of detection; the model can dynamically adjust parameters according to emerging fault modes, which means that as the design and manufacturing technology of reducers continue to develop, the detection model can continue to learn and evolve, and maintain sensitivity to new fault modes; combined with spectral analysis data, it can provide more dimensional information, enhance the comprehensive judgment ability of the detection system, and make it more flexible to deal with different materials and different Appearance inspection of reducers under working conditions; reduces the need for manual intervention and improves inspection efficiency; the system can automatically analyze the evaluation results in real time and generate a comprehensive inspection report containing detailed fault descriptions, maintenance suggestions and expected life predictions, greatly improving the degree of automation and efficiency of inspections; through the fault prediction model, preventive maintenance suggestions are generated based on historical data and current inspection reports, which can help companies plan maintenance plans in advance and avoid downtime losses and increased repair costs caused by sudden failures; timely detection and treatment of appearance defects can effectively prevent the expansion of defects to cause more serious damage to the inside of the reducer, thereby extending the overall service life of the equipment; detailed fault descriptions, accurate maintenance suggestions and expected life predictions can provide users with a high-quality inspection service experience and enhance users' trust and satisfaction with the company's brand.
[0151] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.
Claims
1. A reducer appearance detection method based on deep learning, It is characterized in that The method comprises: S1. Use a high-precision imaging module to collect comprehensive and multi-angle images of the reducer and pre-process the collected images; S2. Apply deep learning models to perform preliminary feature extraction on preprocessed images, identify appearance defects, and use adaptive learning mechanisms to dynamically adjust model parameters based on emerging failure modes; S3, using feature pyramid network to fuse features of different scales, and using spectral analysis data combined with visual features to evaluate appearance quality; S4. Analyze the evaluation results in real time, automatically generate comprehensive test reports, and generate preventive maintenance recommendations based on historical data and current test reports through fault prediction models; The S2 comprises: S21. Select a deep learning model, and use a labeled reducer appearance image dataset to pre-train the deep learning model; S22, inputting the preprocessed image into the pre-trained deep learning model, and automatically extracting multi-level features in the image through a convolutional layer, wherein the multi-level features include edges, textures, and shapes; S23, generating a defect candidate region on the feature map through a region proposal network, and determining whether the region contains a defect through a classifier; S24, if defects are included, the identified defects are sub-classified through a hierarchical classification strategy; S25, integrated online learning module, automatically collects and annotates new failure modes or misidentification cases, and dynamically adjusts deep learning model parameters using incremental learning; The S23 comprises: According to the shape and size of common defects in the reducer appearance image, the anchor box is designed. At the same time, an adaptive anchor box generation mechanism is introduced to dynamically adjust the size and proportion of the anchor box according to the image content. Using the feature pyramid network structure, feature maps of different scales are fused, allowing the region proposal network to detect defects at multiple scales; The region proposal network is used to generate preliminary defect candidate regions by sliding the window on the feature map. Non-maximum suppression is used to remove redundant and overlapping candidate regions based on the scores and overlaps of the candidate regions. According to the complexity and real-time requirements of defect classification, the classifier architecture is selected, and a feature recalibration module is introduced before the classifier to adaptively recalibrate the features of the candidate area; Design a context information fusion mechanism to integrate the image information around the candidate area into the classification decision; use the trained classifier to classify the screened candidate area to determine whether it contains defects; Calculate a confidence score for each candidate area determined to be a defect, and sort or further process the defects based on the confidence score; Review the classification results, identify the defects of false detection and missed detection, and analyze their causes; according to the results of false detection and missed detection analysis, adjust the parameters of the region proposal network, optimize the structure of the classifier, or introduce new feature representation methods; The S3 includes: S31. Use feature pyramid network to fuse different scale features extracted by deep learning model; and combine high-level semantic features with low-level detail features through top-down paths and lateral connections; S32. Based on the feature pyramid network, a multi-scale feature fusion strategy is designed; according to the characteristics of the reducer appearance inspection task, the feature fusion method is selected; S33. Compare the effects of different fusion methods through simulation tests and select fusion strategies; and further optimize the fused features through the attention mechanism; S34, in the feature fusion process, context information is introduced; by considering the correlation between different regions in the image, context features are extracted, and the context features are combined with local features; S35, collecting spectral data of the reducer surface, measuring it using a spectrometer; and preprocessing the spectral data; S36. Analyze the correlation between spectral data and visual features through statistical analysis and machine learning algorithms; obtain the intrinsic connection between spectral features and visual features, and establish a multi-dimensional appearance quality assessment model; S37. Apply machine learning algorithms to establish a mapping relationship between spectral features and appearance quality. By training the model, learn the nonlinear relationship between spectral features and appearance quality, and obtain appearance quality prediction results based on spectral data.
2. According to the method for reducing machine appearance detection based on deep learning as claimed in claim 1, It is characterized in that The S22 comprises: Adopt adaptive median filtering or bilateral filtering algorithm to dynamically adjust filtering parameters according to local features of the image, remove random noise in the image, and retain edge details; Apply the Retinex algorithm or a deep learning-based illumination normalization model to adjust the global and local illumination of the image; Using histogram equalization or its improved method to enhance the contrast of the image; Construct an image pyramid and apply convolutional layers at different scales to extract multi-scale features from coarse to fine. Design a feature fusion module to fuse the features extracted by convolutional layers of different scales; Residual connections are introduced in the convolutional layer, and principal component analysis is used to evaluate the importance of features, identify and retain the most critical features for defect detection, and use L1 regularization to sparse the features.
3. According to the reducer appearance detection method based on deep learning as claimed in claim 1, It is characterized in that The S31 includes: Design a multi-level feature extraction network based on the appearance characteristics of the reducer; balance computing resources and feature representation capabilities, select network depth, and adjust the number and size of convolution kernels; The resolution of high-level feature maps is increased to match that of low-level feature maps through upsampling; at each level, the upsampled high-level features are fused with the corresponding low-level features through horizontal connections; A channel attention module is embedded in the feature fusion process, and the weights of different channels or spatial positions in the feature map are adaptively adjusted through the attention module.
4. According to the reducer appearance detection method based on deep learning as claimed in claim 1, It is characterized in that The S36 comprises: Remove noise and outliers from spectral data and normalize visual features extracted from deep learning models; According to the acquisition position and time of the reducer appearance image and spectral data, data alignment is performed, and the linear or nonlinear correlation between the spectral data and the visual features is analyzed using the Pearson correlation coefficient; Random forest is used to evaluate the importance of each feature to the appearance quality of the reducer. Based on the results of feature selection, principal component analysis is used to reduce the dimension of the features. Through a variety of fusion strategies, the data is fused and the key parameters in the fusion strategy are tuned; the performance of different fusion strategies in the reducer appearance quality assessment task is evaluated, and the deep learning model is selected according to the task characteristics; The spectral data and visual features are embedded into the hidden layers of the deep learning model respectively, and the features are deeply fused through the attention mechanism; An end-to-end training method is adopted to simultaneously optimize the spectral data processing module and the visual feature processing module.
5. According to the method for reducing machine appearance detection based on deep learning as claimed in claim 1, It is characterized in that The S4 comprises: S41, performing real-time analysis on the fused features and applying a classification algorithm to determine the appearance quality grade of the reducer; S42. Predict potential faults and their development trends through the fault prediction model based on historical data and current test results; conduct a comprehensive analysis of the classification results and fault prediction results; S43, automatically generate a comprehensive test report based on the results of real-time evaluation and analysis; S44. Generate preventive maintenance recommendations based on the results of the fault prediction model; and continuously adjust and optimize the maintenance plan based on real-time detection data and historical maintenance records.
6. A system for implementing the reducer appearance detection method based on deep learning as claimed in claim 1, It is characterized in that The system comprises: Image acquisition module: Use high-precision imaging module to perform comprehensive and multi-angle image acquisition of the reducer, and pre-process the acquired images; Feature extraction module: Apply deep learning models to perform preliminary feature extraction on preprocessed images, identify appearance defects, and use adaptive learning mechanisms to dynamically adjust model parameters based on emerging fault modes; Feature fusion module: It uses a feature pyramid network to fuse features of different scales, and uses spectral analysis data combined with visual features to evaluate the appearance quality; Report generation module: Analyze the evaluation results in real time, automatically generate comprehensive inspection reports, and generate preventive maintenance recommendations based on historical data and current inspection reports through fault prediction models.
Citation Information
Patent Citations
Method for detecting bolt denitration defect of power transmission line by introducing attention mechanism
CN114677339A
Wafer defect unknown category detection and classification method and system
CN116188877A
