Internal mixer rotor defect detection method, system and medium based on image recognition

Abstract

The present invention relates to the technical field related to image recognition, and specifically includes a method, system, and medium for detecting defects in internal mixer rotors based on image recognition, including: extracting frequency domain and time domain features from the vibration signal of the internal mixer rotor, optimizing the image and matching the defect visual vector, configuring the vibration signal feature vector, and evaluating the defects with a defect classification network after fusion. If the result exceeds the limit, a reminder is triggered and the robot arm is linked to shut down. The method solves the technical problem that the rotor surface imaging is blurred due to oil coverage during operation of the internal mixer, and the rotor is easily interfered by mechanical noise in a high-temperature and high-noise environment, which limits the accuracy of internal mixer rotor defect detection. By fusing the frequency domain and time domain features of the vibration signal with the optimized rotor image visual features, a defect classification network with an attention mechanism is introduced, which can dynamically focus on tiny defect areas, eliminate oil interference, and adaptively optimize contrast to cope with high-temperature environments. It can still maintain clear imaging in an oily environment, thereby ensuring the technical effect of accurate internal mixer rotor defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field related to image recognition, and in particular to a method, system and medium for detecting defects in an internal mixer rotor based on image recognition. Background Art

[0002] As the manufacturing industry develops towards intelligence and high precision, early detection of rotor defects has become a key link in ensuring the safe and stable operation of equipment. However, the working environment of the internal mixer is high temperature, high noise and complex oil pollution. Rotor surface defects (such as cracks, wear, and deformation) can easily cause mechanical failures and even production accidents. Single modal detection technology can no longer meet the modern industry's needs for real-time and accurate detection of rotor defects.

[0003] Current internal mixer rotor defect detection relies on static image analysis, which is unable to identify tiny cracks and internal damage. Although vibration sensors can capture anomalies, they lack multi-source data fusion and are easily affected by mechanical noise, leading to misjudgment. On the other hand, when the internal mixer is working, oil stains cover the rotor surface, resulting in blurred imaging and difficulty in feature extraction. The failure rate of existing detection equipment further increases in high-temperature environments.

[0004] In summary, the existing technology has technical problems such as oil coverage during operation of the internal mixer causing blurred imaging of the rotor surface, susceptibility to mechanical noise interference in a high temperature and high noise environment, and limited accuracy in detecting defects in the internal mixer rotor. Summary of the Invention

[0005] This application provides an image recognition-based internal mixer rotor defect detection method, system and medium, aiming to solve the technical problems in the prior art that the rotor surface imaging is blurred due to oil coverage during operation of the internal mixer, the internal mixer is susceptible to mechanical noise interference in a high-temperature and high-noise environment, and the internal mixer rotor defect detection accuracy is limited.

[0006] In view of the above problems, the technical solution to implement this application is:

[0007] In a first aspect, the present application provides a method for detecting defects in an internal mixer rotor based on image recognition, wherein the method comprises: performing multimodal feature extraction on the rotor surface according to a vibration sensor signal, and determining frequency domain feature parameters and time domain feature parameters associated with the rotor operating state; performing contrast optimization on the internal mixer rotor image, and configuring a defect visual feature vector in combination with the frequency domain feature parameters and time domain feature parameters; decomposing the vibration sensor signal, extracting energy features of different frequency bands, and configuring a vibration signal feature vector; fusing the defect visual feature vector with the vibration signal feature vector, using a defect classification network, introducing an attention mechanism to dynamically focus on tiny defect areas, and outputting quantitative evaluation results of crack grade, deformation amount, and wear degree; when the quantitative evaluation results of the crack grade, deformation amount, and wear degree meet preset threshold conditions, triggering an audio-visual reminder instruction and linking a robotic arm to perform an emergency shutdown operation.

[0008] Preferably, a multispectral imaging device is deployed to eliminate the interference of oil pollution on the rotor surface through phase unwrapping and obtain the surface roughness characteristics; at the same time, the laser point cloud and the rotor surface image are combined to dynamically correct the geometric distortion caused by high temperature deformation.

[0009] Preferably, a dynamic weight map is constructed, which is used to capture early abnormal features including metal cracks, rubber aging and assembly gaps; based on the dynamic weight map, an adaptive contrast stretching variable is introduced to automatically adjust the gamma value of the rubber rotor material / plastic rotor material; according to the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized to configure the defect visual feature vector.

[0010] Preferably, the intrinsic mode function of the vibration sensor signal is extracted by empirical mode decomposition, and noise interference is eliminated by wavelet transform to obtain the frequency band energy distribution characteristics of mechanical failure factors, which include bearing wear and gear tooth breakage; through joint analysis of time and frequency domains, impact events are identified and sparse coding is performed on sudden impact signals.

[0011] Preferably, a graph convolutional network unit is embedded in the encoder to formulate the topological relationship of the rotor joints and capture the local deformation characteristics caused by thermal stress; and a cross-modal attention mechanism is introduced in the decoder to coordinate the weight allocation of the defect visual feature vector and the vibration signal feature vector.

[0012] Preferably, a generative adversarial network is deployed, which is used to repair the image of the internal mixer rotor and generate a synthetic image of the high-temperature deformation area of ​​the rotor through adversarial training; based on the synthetic image, the weight distribution of the defect visual feature vector is updated.

[0013] Preferably, a CBAM unit is embedded in the backbone network to enhance the feature extraction capability of tiny defect areas; based on the backbone network, a multi-scale feature fusion strategy is configured to fuse shallow high-resolution features with deep semantic features, and generate feature maps of different scales through a feature pyramid network.

[0014] Preferably, an adaptive parameter optimization mechanism is set up to trigger reinforcement learning-driven dynamic compensation of exposure parameters when the detected ambient light change exceeds the preset change range; at the same time, a rotor defect knowledge graph is constructed, linking historical maintenance work orders with the material failure database to support natural language queries.

[0015] In a second aspect of the present application, a rotor defect detection system for an internal mixer based on image recognition is provided, wherein the system includes: a feature extraction module, which is used to perform multimodal feature extraction on the rotor surface according to the vibration sensor signal, and determine the frequency domain feature parameters and time domain feature parameters associated with the rotor operating state; a contrast optimization module, which is used to perform contrast optimization on the internal mixer rotor image, and configure a defect visual feature vector in combination with the frequency domain feature parameters and time domain feature parameters; decompose the vibration sensor signal, extract energy features of different frequency bands, and configure a vibration signal feature vector; a defect classification module, which is used to fuse the defect visual feature vector with the vibration signal feature vector, use a defect classification network, introduce an attention mechanism to dynamically focus on tiny defect areas, and output quantitative evaluation results of crack level, deformation and wear degree; a defect reminder module, which is used to trigger an audio-visual reminder instruction and link a robotic arm to perform an emergency shutdown operation when the quantitative evaluation results of the crack level, deformation and wear degree meet preset threshold conditions.

[0016] In a third aspect of the present application, the present invention further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned internal mixer rotor defect detection method based on image recognition when executing the computer program.

[0017] In summary, the one or more technical solutions provided in this application, by integrating the frequency domain and time domain features of the vibration signal with the optimized visual features of the rotor image, introduce a defect classification network with an attention mechanism, which can dynamically focus on tiny defect areas, eliminate oil interference, and adaptively optimize contrast to cope with high temperature environments. It can still maintain clear imaging in an oily environment, thereby ensuring the technical effect of accurate detection of rotor defects in the internal mixer. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flow chart of a method for detecting rotor defects of an internal mixer based on image recognition is provided for this application.

[0019] Figure 2A structural diagram of an internal mixer rotor defect detection system based on image recognition is provided for this application.

[0020] Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0021] Explanation of the reference numerals: feature extraction module M100 , contrast optimization module M200 , defect classification module M300 , defect reminder module M400 , bus 300 , receiver 301 , processor 302 , transmitter 303 , memory 304 , bus interface 305 . DETAILED DESCRIPTION

[0022] Example 1: The present application will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the present application provides a method for detecting rotor defects of an internal mixer based on image recognition, wherein the method comprises:

[0023] S1: Extract multimodal features of the rotor surface based on the vibration sensor signal to determine the frequency domain feature parameters and time domain feature parameters associated with the rotor operating status; S2: Optimize the contrast of the internal mixer rotor image and configure the defect visual feature vector based on the frequency domain feature parameters and time domain feature parameters; Decompose the vibration sensor signal to extract the energy features of different frequency bands and configure the vibration signal feature vector.

[0024] Specifically, multimodal feature extraction refers to the process of obtaining multiple types of features from vibration sensor signals, including frequency domain feature parameters and time domain feature parameters; frequency domain feature parameters are features extracted by converting the vibration signal from the time domain to the frequency domain (usually using methods such as fast Fourier transform), such as frequency components, power spectral density, etc., which can reflect the vibration intensity and distribution of the rotor at different frequencies.

[0025] Time domain feature parameters are directly extracted from the vibration signal in the time domain, such as the amplitude, root mean square value, and kurtosis of the vibration signal, which mainly reflect the intensity and fluctuation characteristics of the vibration signal. The two are jointly related to the operating status of the rotor and provide a basis for subsequent defect analysis; contrast optimization is to process the image of the internal mixer rotor to enhance the contrast between the target area (such as the defect area) and the background area in the image, making the defect clearer and more identifiable.

[0026] This is achieved by adjusting the grayscale distribution of the image and using algorithms such as histogram equalization or adaptive contrast enhancement. This helps to highlight tiny defects such as cracks and wear, and improve the accuracy of image feature extraction. Feature vectors are used to quantify the extracted features into vectors, which facilitates calculation and analysis in the algorithm model. Configuring defect visual feature vectors quantifies and integrates defect-related features (such as shape, texture, grayscale distribution, etc.) in the contrast-optimized image into a vector, comprehensively characterizing the visual features of defects in the image and providing image feature input for subsequent fusion analysis.

[0027] Decomposing vibration sensor signals means using signal processing techniques (such as wavelet decomposition, empirical mode decomposition, etc.) to decompose complex vibration signals into multiple sub-signals or intrinsic mode functions in different frequency bands in order to extract more targeted features; extracting energy characteristics of different frequency bands from the decomposed signals. These energy characteristics reflect the energy distribution of the vibration signal in different frequency bands and can reflect the vibration energy characteristics of the rotor at different frequencies. They are then configured into vibration signal feature vectors to provide vibration feature input for fusion analysis.

[0028] Execution steps: Perform multimodal feature extraction on the vibration sensor signal to determine the frequency domain and time domain feature parameters. Taking a common industrial internal mixer as an example, during normal operation, the vibration signal of the rotor has relatively stable frequency domain and time domain characteristics. Generally, in the frequency domain characteristics of the vibration signal of a normal rotor, the energy near the main operating frequency accounts for up to 60%-70%, while in the time domain characteristics, the root mean square value of the vibration amplitude is usually between 0.5g-1.0g (g is the acceleration of gravity). These characteristic parameters can be used as basic indicators for evaluating the operating status of the rotor.

[0029] The contrast of the internal mixer rotor image is optimized and defect visual feature vectors are configured. Under complex working conditions such as oil pollution and high temperature, the original contrast of the internal mixer rotor image is often low, and the defective areas are difficult to clearly identify. Through contrast optimization algorithms, such as adaptive contrast stretching, the contrast of the image can be improved, effectively highlighting defective areas such as cracks and wear.

[0030] At the same time, the vibration sensor signal is decomposed and the vibration signal feature vector is configured. Taking empirical mode decomposition as an example, the vibration signal can be decomposed into multiple intrinsic mode functions, which correspond to the vibration characteristics of different frequency bands, and the energy characteristics of each intrinsic mode function are extracted. Furthermore, when the rotor has a bearing wear fault, the energy characteristics of the intrinsic mode function in a specific high-frequency band will change significantly, and its energy value may be higher than the normal state; after integrating these energy characteristics of different frequency bands into a vibration signal feature vector, the rotor vibration state can be fully characterized, and together with the defect visual feature vector, it provides key data support for subsequent feature fusion and defect classification, realizing the effective integration of multi-dimensional features and improving the accuracy and reliability of defect detection.

[0031] S3: Fuse the defect visual feature vector with the vibration signal feature vector, use the defect classification network, introduce the attention mechanism to dynamically focus on the tiny defect area, and output the quantitative evaluation results of the crack level, deformation and wear degree; S4: When the quantitative evaluation results of the crack level, deformation and wear degree meet the preset threshold conditions, trigger the sound and light reminder command and link the robotic arm to perform an emergency stop operation.

[0032] Specifically, feature vector fusion refers to the integration of defect visual feature vectors and vibration signal feature vectors, so that the two types of features interact and complement each other in a unified feature space, so as to more comprehensively characterize the defect information of the rotor; the defect classification network is a model based on machine learning or deep learning, which is used to analyze and classify the fused features to identify different defect types and degrees; the attention mechanism is a neural network module that simulates visual attention, which can dynamically focus on the most important part of the input data, that is, the small defect area. The fused features are weighted through the attention mechanism, so that the network pays more attention to the feature parts related to the defects, thereby improving the classification accuracy and sensitivity to small defects.

[0033] The quantitative assessment results are specific numerical representations of the crack level, deformation and wear degree. Through deep learning analysis of the fusion features, the defect information is converted into quantifiable indicators; the preset threshold conditions refer to the pre-set threshold ranges in the quantitative assessment results of the crack level, deformation and wear degree. When the actual detection results reach or exceed these thresholds, it indicates that the rotor defect may affect the normal operation or even safety of the equipment. At this time, it is necessary to trigger the sound and light warning command and link the robotic arm to perform an emergency shutdown operation to ensure the safety of the equipment and production.

[0034] Execution steps: Fuse the defect visual feature vector with the vibration signal feature vector. The visual feature vector and the vibration signal feature vector respectively carry information of different dimensions of the rotor surface and operating status. Specifically, the visual feature vector may contain information such as the shape, size, and texture of the rotor surface crack; the vibration signal feature vector reflects the vibration characteristics of the rotor at different frequencies and times; through feature fusion, information complementarity can be achieved, enabling the model to have a more comprehensive understanding of the rotor defect situation. The fused features combine the two types of information to provide richer input for the defect classification network, thereby improving the classification accuracy.

[0035] After using the defect classification network and introducing the attention mechanism, the network can dynamically focus on tiny defect areas. During the inspection process, tiny defects are often the most easily overlooked but the most critical. The attention mechanism weights key areas by learning the importance of features, enabling the model to identify these areas more accurately. Specifically, when inspecting rubber rotors, for cracks with a width of 0.05 mm, the attention mechanism can enhance the weight of these tiny features in the feature map, thereby improving detection sensitivity.

[0036] The system outputs quantitative assessment results of crack grade, deformation and wear degree. The quantitative assessment results are presented in the form of specific numerical values. For example, the crack grade may be divided into grades 1-10, the deformation is in millimeters, and the wear degree is expressed as a percentage, providing a clear basis for subsequent decision-making. When the quantitative assessment results meet the preset threshold conditions, for example, when the crack grade reaches grade 8 or the deformation exceeds 5mm, an audible and visual reminder command will be triggered and the robotic arm will be linked to perform an emergency shutdown operation, ensuring that timely measures are taken before the defect may cause serious failure, effectively reducing the risk of equipment failure and production accidents, and ensuring the continuity and safety of industrial production.

[0037] Furthermore, the multimodal feature extraction of the rotor surface is performed based on the vibration sensor signal. The method of the present application includes:

[0038] Multispectral imaging equipment is deployed to eliminate oil interference on the rotor surface through phase unwrapping and obtain surface roughness characteristics; at the same time, the laser point cloud and rotor surface image are combined to dynamically correct the geometric distortion caused by high-temperature deformation.

[0039] Specifically, multispectral imaging equipment is an imaging technology that can simultaneously acquire image data in multiple spectral bands (such as visible light, near-infrared, mid-infrared, etc.). Phase unwrapping is a signal processing technology that can eliminate the interference of oil on the rotor surface on imaging, because oil can cause abnormal reflection and scattering of light, affecting the imaging quality. Phase unwrapping can extract real surface morphology information from multispectral images, thereby obtaining the roughness characteristics of the rotor surface. The roughness characteristics are an important indicator for measuring the microscopic roughness of the rotor surface and reflect the quality and wear of the rotor surface; laser point clouds can provide high-precision geometric shape information, and the rotor surface image provides two-dimensional visual information. Due to the high temperature environment, the rotor will deform, resulting in geometric distortion of the image, that is, the shape of the object in the image does not match the actual shape; dynamic correction refers to the real-time correction of distortion during the imaging process, so that the image can accurately reflect the actual geometric shape of the rotor.

[0040] Implementation steps: Deploy multispectral imaging equipment and use phase unwrapping technology to eliminate oil interference on the rotor surface and obtain surface roughness characteristics. Specifically, the thickness of oil on the surface of the internal mixer rotor is generally between 0.1mm and 0.3mm. Oil pollution will seriously affect the imaging quality, resulting in reduced image contrast and blurred features. After using multispectral imaging equipment combined with phase unwrapping technology, it is possible to obtain rotor images in multiple spectral bands, effectively penetrate the oil layer, and extract the true morphology information of the rotor surface, providing high-quality surface feature data for subsequent defect detection.

[0041] At the same time, the laser point cloud and the rotor surface image are combined to dynamically correct the geometric distortion caused by high-temperature deformation. Specifically, during the operation of the internal mixer, the rotor temperature can reach 150℃-200℃, causing the rotor surface to expand and change in shape, resulting in geometric distortion of the image. The laser point cloud provides high-precision three-dimensional geometric information, which is matched and corrected with the rotor surface image in real time. The accuracy of the geometric distortion correction is controlled within a certain range, ensuring a high degree of consistency between the image and the actual rotor shape, and ensuring the data accuracy of subsequent feature extraction and defect analysis. This lays a solid foundation for improving the reliability and accuracy of the detection system, enabling the detection system to operate stably in harsh high-temperature and oily environments, and effectively identify tiny defects on the rotor surface.

[0042] Furthermore, the contrast of the internal mixer rotor image is optimized, and the defect visual feature vector is configured by combining the frequency domain feature parameters and the time domain feature parameters. The method of the present application includes:

[0043] A dynamic weighted atlas is constructed to capture early abnormal features, including metal cracks, rubber aging, and assembly gaps. Based on the dynamic weighted atlas, an adaptive contrast stretching variable is introduced to automatically adjust the gamma value of the rubber rotor material / plastic rotor material. Based on the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized to configure the defect visual feature vector.

[0044] Specifically, the dynamic weight atlas is a mapping tool used to highlight key features in the image (such as early abnormal features such as metal cracks, rubber aging and assembly gaps). It achieves differentiated expression of features by assigning different weights to different areas of the image. Areas with high weights represent potential defect areas that deserve more attention. The adaptive contrast stretching variable automatically adjusts the gamma value according to the differences in the optical properties of the rotor material (rubber or plastic) (the gamma value is used to control the nonlinear mapping relationship of the image grayscale, thereby affecting the image contrast). Contrast optimization refers to processing the image to enhance the contrast between the target and the background in the image, making potential defects clearer and more identifiable. Configuring the defect visual feature vector is to quantify and integrate the defect-related features (such as shape, texture, grayscale distribution, etc.) in the optimized image into a vector, comprehensively characterizing the visual characteristics of the defects in the image, and providing image feature input for subsequent fusion analysis.

[0045] Execution steps: Construct a dynamic weight map to capture early abnormal features. The dynamic weight map assigns regional weights to the image based on prior knowledge of rotor materials and common defect types (such as metal cracks, rubber aging, and assembly gaps). For rubber rotors, the aging area usually manifests as specific texture changes, and its weight will be dynamically increased, thereby improving the detection rate of early abnormal features; based on the dynamic weight map, introduce an adaptive contrast stretching variable to automatically adjust the gamma value of the rubber rotor material or plastic rotor material. Due to the differences in optical properties of different materials, the gamma value of the rubber rotor is between 1.5-2.0, and the gamma value of the plastic rotor is between 2.0-2.5; by automatically adjusting the gamma value, the contrast-optimized image can more clearly present the defect features.

[0046] Based on the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized and a defect visual feature vector is configured. The optimized image provides richer defect information for the feature vector, the dimension of the defect visual feature vector is increased, and the feature expression capability is significantly enhanced. In the above steps, the accuracy and sensitivity of image defect feature extraction are improved, ensuring the reliability of subsequent fusion analysis and defect classification, enabling the system to more accurately identify early defects of the rotor and provide strong support for preventive maintenance of the internal mixer.

[0047] Furthermore, the vibration sensor signal is decomposed, and the method of the present application further includes:

[0048] The intrinsic mode function of the vibration sensor signal is extracted through empirical mode decomposition, and noise interference is eliminated by wavelet transform to obtain the frequency band energy distribution characteristics of mechanical failure factors, such as bearing wear and gear tooth breakage. Through joint analysis of time and frequency domains, impact events are identified and sparse coding is performed on sudden impact signals.

[0049] Specifically, empirical mode decomposition is able to decompose complex vibration sensor signals into several simple intrinsic mode functions with single frequency characteristics. Each intrinsic mode function represents the vibration characteristics of a specific frequency band in the signal, which can more finely analyze the different frequency components in the signal; wavelet transform is used to perform multi-resolution analysis of the signal, which can effectively eliminate noise interference in the signal.

[0050] The frequency band energy distribution characteristics refer to the energy distribution of the vibration signal in different frequency bands. By calculating the energy value of each intrinsic mode function, the frequency band energy distribution characteristics of the entire signal can be obtained; mechanical failure factors such as bearing wear and gear tooth breakage produce specific frequency components and energy changes in the vibration signal; time-frequency domain joint analysis refers to analyzing the signal in the time domain and frequency domain at the same time, which can capture the characteristics of the signal more comprehensively; impact events refer to instantaneous impact signals that appear in the vibration signal, which are usually related to mechanical failures; sparse coding is used to extract and represent features of sudden impact signals.

[0051] Execution steps: Extract the intrinsic mode functions (IMFs) of the vibration sensor signal through empirical mode decomposition. For example, the vibration signal of an internal mixer rotor in an industrial setting can typically be decomposed into 5-8 IMFs, each corresponding to the vibration characteristics of a different frequency band. Combined with wavelet transforms to eliminate noise interference, the frequency band energy distribution characteristics are more accurately determined. Analysis of the frequency band energy distribution characteristics has been shown to reveal that the energy values ​​of the IMFs in specific high-frequency bands are 2-3 times higher than normal, providing a key basis for identifying bearing wear.

[0052] Through joint analysis of the time and frequency domains, impact events are identified and sudden impact signals are sparsely coded. Generally, the joint analysis of the time and frequency domains can capture the instantaneous impact signals caused by broken teeth, and the impact intensity can reach 5-8 times the normal vibration amplitude. By representing these impact signals as a combination of a small number of basic impact signals through sparse coding, the data dimension can be effectively reduced while retaining key feature information. In the above steps, the detection accuracy of faults such as bearing wear and gear tooth breakage is improved, and the reliability of subsequent fusion analysis and fault diagnosis is ensured, providing a strong guarantee for the safe operation of the internal mixer.

[0053] Furthermore, the defect visual feature vector is fused with the vibration signal feature vector, and a defect classification network is used. The method of the present application includes:

[0054] A graph convolutional network unit is embedded in the encoder to propose the topological relationship of the rotor joints and capture the local deformation characteristics caused by thermal stress. A cross-modal attention mechanism is introduced in the decoder to coordinate the weights of the defect visual feature vector and the vibration signal feature vector.

[0055] Specifically, the graph convolutional network unit is used to process data with complex topological relationships, such as the connection relationship of rotor joints; the rotor joint topological relationship refers to the connection structure and spatial relationship between the various components of the rotor, which affects the mechanical properties and thermal stress distribution of the rotor; thermal stress refers to the internal stress of the material caused by temperature changes, which will cause local deformation of the rotor and affect its normal operation; embedding the graph convolutional network unit in the encoder can utilize the graph convolutional network's modeling ability of the rotor joint topological relationship to capture the local deformation characteristics caused by thermal stress; the cross-modal attention mechanism is a mechanism that dynamically adjusts the weights of each modal data during the fusion process of multimodal data (such as visual features and vibration features); introducing the cross-modal attention mechanism in the decoder to coordinate the allocation of weights of the defect visual feature vector and the vibration signal feature vector, so that the model can automatically focus on more important feature modes according to the needs of the current detection task, thereby improving the effect of feature fusion.

[0056] Execution steps: Embed a graph convolutional network unit in the encoder to propose the topological relationship of the rotor joint and capture the local deformation characteristics caused by thermal stress. Specifically, the topological relationship of the rotor joint includes the connection relationship between the rotor blades, rotor shaft, rotor end cover and other components. Under high temperature environment, these components will produce thermal stress due to different thermal expansion coefficients, resulting in local deformation; through the graph convolutional network unit, the rotor joint topological relationship can be modeled, and the local deformation characteristics of the connection between the rotor blade and the rotor shaft caused by thermal stress can be captured, thereby improving the extraction accuracy of local deformation features.

[0057] A cross-modal attention mechanism is introduced into the decoder to coordinate the weights of the defect visual feature vector and the vibration signal feature vector. Specifically, during the feature fusion process, the importance of visual features and vibration features will vary depending on the detection task. Through the cross-modal attention mechanism, the weights of the two can be automatically adjusted according to the requirements of the current detection task. When detecting cracks on the rotor surface, the weight of the visual features is increased; when detecting bearing wear, the weight of the vibration features is increased. In the above steps, it is ensured that the model can fully utilize the feature information of the two modes, improve the pertinence and effectiveness of feature fusion, and provide a more reliable basis for the intelligent operation and maintenance of the internal mixer rotor.

[0058] Furthermore, the present application method also includes:

[0059] A generative adversarial network is deployed to repair the image of the internal mixer rotor and generate a synthetic image of the rotor's high-temperature deformation area through adversarial training; based on the synthetic image, the weight distribution of the defect visual feature vector is updated.

[0060] Specifically, the generative adversarial network is a deep learning model composed of a generator and a discriminator, which is used to generate and repair image data. The generative adversarial network is deployed to repair the image of the internal mixer rotor, and adversarial training is used to generate synthetic images of the high-temperature deformation area of ​​the rotor; the goal of the generator is to generate synthetic images of the high-temperature deformation area that are as realistic as possible, and the goal of the discriminator is to distinguish between the generated images and the real images; through adversarial training, the generator continuously learns how to generate more realistic images, thereby realizing the repair of the high-temperature deformation area; synthetic images refer to images generated by the generator, which are repaired in the high-temperature deformation area and are closer to the real situation; updating the weight distribution of the defect visual feature vector refers to readjusting the weights of each feature in the defect visual feature vector according to the information in the synthetic image, so that the feature vector can better reflect the real defect situation.

[0061] Execution steps: Deploy a generative adversarial network to repair the image of the internal mixer rotor. In a high-temperature environment, the rotor image will exhibit problems such as deformation and blurring. Through adversarial training of the generative adversarial network, the generator can produce synthetic images of the high-temperature deformed area. Through the generative adversarial network repair, the image quality of the deformed area is improved, and the features of the deformed area are clearer and more discernible.

[0062] The weight distribution of the defect visual feature vector is updated according to the synthetic image. During the weight updating process, the feature weight of the repaired deformed area in the synthetic image will increase accordingly, thereby more accurately reflecting the actual defect situation of the rotor. Through the above steps, the image repair quality and the accuracy of the feature vector are improved, ensuring the reliability of feature fusion and defect classification, enabling the detection system to more accurately identify rotor defects, and providing high-quality data support for the intelligent operation and maintenance of the internal mixer.

[0063] Furthermore, an attention mechanism is introduced to dynamically focus on small defect areas. The method of this application also includes:

[0064] A CBAM unit is embedded in the backbone network to enhance the feature extraction capability of tiny defect areas. Based on the backbone network, a multi-scale feature fusion strategy is configured to fuse shallow high-resolution features with deep semantic features, and feature maps of different scales are generated through a feature pyramid network.

[0065] Specifically, the CBAM unit (Convolutional Block Attention Module) can be embedded into backbone networks (such as convolutional neural networks) to enhance the network's feature extraction capabilities for specific image regions (such as small defect regions). Through channel attention and spatial attention mechanisms, it automatically learns and highlights important feature regions while suppressing unimportant feature regions. The multi-scale feature fusion strategy involves fusing features from different levels (such as shallow high-resolution features and deep semantic features) to obtain more comprehensive feature information. Shallow high-resolution features can provide detailed image information, such as the edges and textures of small defects, while deep semantic features can provide high-level semantic information, such as the category and shape of defects. The feature pyramid network is a network structure that generates feature maps of different scales. It can generate a series of feature maps from high resolution to low resolution to capture defect features at different scales.

[0066] Execution steps: Embed CBAM units in the backbone network to enhance the feature extraction capability of tiny defect areas. Tiny cracks on the surface of the internal mixer rotor are easily overlooked. After embedding the CBAM units, tiny defect areas can be automatically learned. The channel attention mechanism will enhance the channel weights related to the crack features, and the spatial attention mechanism will focus on the spatial position of the crack, thereby improving the efficiency of crack feature extraction.

[0067] Based on the backbone network, a multi-scale feature fusion strategy is configured to fuse shallow high-resolution features with deep semantic features. Feature pyramid networks are then used to generate feature maps of different scales. Shallow features provide edge details of cracks, while deep features provide information about the overall shape and category of the cracks. The fused feature maps can contain both detail and semantic information. Specifically, when detecting worn areas on rubber rotors, the fused feature maps can accurately capture the edge details and overall shape of the worn areas, making the assessment of the degree of wear more accurate. These steps improve the accuracy of detecting minor defects and the ability to identify multi-scale defects, ensuring that the model can fully understand rotor defect characteristics at different levels and scales. This provides high-quality feature input for subsequent defect classification and assessment, and enhances the reliability of internal mixer rotor defect detection.

[0068] Furthermore, the present application method includes:

[0069] An adaptive parameter optimization mechanism is set up to trigger reinforcement learning-driven dynamic compensation of exposure parameters when the detected ambient light changes exceed the preset change range. At the same time, a rotor defect knowledge graph is constructed, linking historical maintenance work orders with the material failure database to support natural language queries.

[0070] Specifically, the adaptive parameter optimization mechanism is a mechanism that can automatically adjust system parameters according to environmental changes, which is used to ensure that the detection system can still operate stably and maintain high detection accuracy under different lighting conditions; reinforcement learning-driven exposure parameter dynamic compensation refers to the use of reinforcement learning algorithms to automatically adjust the exposure parameters of the image acquisition equipment according to changes in the detection environment lighting to adapt to different lighting conditions; the rotor defect knowledge graph is a structured knowledge base used to store and associate information related to rotor defects, such as historical maintenance work orders, material failure data, etc.; natural language query means that users can obtain the required information from the knowledge graph by entering query statements in natural language.

[0071] Execution steps: Set up an adaptive parameter optimization mechanism. When the detection environment light changes beyond the preset change range, trigger the reinforcement learning-driven dynamic compensation of exposure parameters. The light intensity in the internal mixer workshop may fluctuate between 200 lux and 1000 lux. This light change will affect image quality, causing the image to be too dark or too bright, affecting the accuracy of defect detection. After adopting the adaptive parameter optimization mechanism, it is possible to monitor light changes in real time. When the light change exceeds the preset range (such as 300 lux to 800 lux), the reinforcement learning algorithm will dynamically adjust the exposure parameters based on historical data and current lighting conditions to keep the image within the optimal brightness range.

[0072] After dynamic compensation, the image's brightness uniformity is improved, boosting the accuracy of defect detection. Simultaneously, a rotor defect knowledge graph is constructed, linking historical maintenance work orders with a material failure database to support natural language queries. By integrating historical maintenance work orders (including information such as maintenance time, fault type, and maintenance measures) with a material failure database (including data such as material properties and failure modes), the knowledge graph can provide comprehensive background information on rotor defects. Specifically, when a user searches for maintenance records due to bearing wear within the past six months, the relevant maintenance work orders can be quickly returned, and combined with material failure data, the causes and trends of bearing wear can be analyzed. These steps improve the system's environmental adaptability and data utilization efficiency, ensuring the detection system's stable operation in complex lighting environments. This provides strong data support for equipment maintenance and enhances the intelligent operation and maintenance of internal mixer rotors.

[0073] In summary, the beneficial effects of the embodiments of the present application are:

[0074] The invention uses multimodal feature extraction of the rotor surface based on the vibration sensor signal to determine the frequency domain feature parameters and time domain feature parameters associated with the rotor operating state; optimizes the contrast of the internal mixer rotor image, and configures the defect visual feature vector by combining the frequency domain feature parameters and time domain feature parameters; decomposes the vibration sensor signal, extracts the energy characteristics of different frequency bands, and configures the vibration signal feature vector; fuses the defect visual feature vector with the vibration signal feature vector, uses a defect classification network, introduces an attention mechanism to dynamically focus on the tiny defect area, and outputs the quantitative evaluation results of the crack grade, deformation amount, and wear degree; when the quantitative evaluation results of the crack grade, deformation amount, and wear degree meet the preset threshold conditions, triggers an audio-visual warning command and links the robotic arm to perform an emergency shutdown operation. This application provides an internal mixer rotor defect detection method, system, and medium based on image recognition. By fusing the frequency domain and time domain features of the vibration signal with the optimized visual features of the rotor image, a defect classification network with an attention mechanism is introduced. This network can dynamically focus on tiny defect areas, eliminate oil interference, and adaptively optimize contrast to cope with high-temperature environments. It can still maintain clear imaging in oily environments, thereby ensuring the technical effect of accurate defect detection of internal mixer rotors.

[0075] Embodiment 2 is based on the same inventive concept as the method for detecting rotor defects of an internal mixer based on image recognition in the aforementioned embodiment. Figure 2 As shown, an embodiment of the present application provides an internal mixer rotor defect detection system based on image recognition, wherein the system includes:

[0076] The feature extraction module M100 is used to extract multi-modal features of the rotor surface according to the vibration sensor signal, and determine the frequency domain feature parameters and time domain feature parameters associated with the rotor operation state.

[0077] The contrast optimization module M200 is used to optimize the contrast of the internal mixer rotor image, configure the defect visual feature vector by combining the frequency domain feature parameters and time domain feature parameters; decompose the vibration sensor signal, extract the energy characteristics of different frequency bands, and configure the vibration signal feature vector.

[0078] The defect classification module M300 is used to fuse the defect visual feature vector with the vibration signal feature vector, use the defect classification network, introduce the attention mechanism to dynamically focus on the tiny defect area, and output the quantitative evaluation results of the crack grade, deformation and wear degree.

[0079] The defect reminder module M400 is used to trigger the sound and light reminder command and link the robotic arm to perform an emergency stop operation when the quantitative evaluation results of the crack level, deformation and wear degree meet the preset threshold conditions.

[0080] Furthermore, the feature extraction module M100 is used to perform the following method:

[0081] Multispectral imaging equipment is deployed to eliminate oil interference on the rotor surface through phase unwrapping and obtain surface roughness characteristics; at the same time, the laser point cloud and rotor surface image are combined to dynamically correct the geometric distortion caused by high-temperature deformation.

[0082] Furthermore, the contrast optimization module M200 is configured to perform the following method:

[0083] A dynamic weighted atlas is constructed to capture early abnormal features, including metal cracks, rubber aging, and assembly gaps. Based on the dynamic weighted atlas, an adaptive contrast stretching variable is introduced to automatically adjust the gamma value of the rubber rotor material / plastic rotor material. Based on the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized to configure the defect visual feature vector.

[0084] Furthermore, the contrast optimization module M200 is further configured to perform the following method:

[0085] The intrinsic mode function of the vibration sensor signal is extracted through empirical mode decomposition, and noise interference is eliminated by wavelet transform to obtain the frequency band energy distribution characteristics of mechanical failure factors, such as bearing wear and gear tooth breakage. Through joint analysis of time and frequency domains, impact events are identified and sparse coding is performed on sudden impact signals.

[0086] Furthermore, the defect classification module M300 is used to perform the following method:

[0087] A graph convolutional network unit is embedded in the encoder to propose the topological relationship of the rotor joints and capture the local deformation characteristics caused by thermal stress. A cross-modal attention mechanism is introduced in the decoder to coordinate the weights of the defect visual feature vector and the vibration signal feature vector.

[0088] Furthermore, the defect classification module M300 is further configured to perform the following method:

[0089] A generative adversarial network is deployed to repair the image of the internal mixer rotor and generate a synthetic image of the rotor's high-temperature deformation area through adversarial training; based on the synthetic image, the weight distribution of the defect visual feature vector is updated.

[0090] Furthermore, the defect classification module M300 is further configured to perform the following method:

[0091] A CBAM unit is embedded in the backbone network to enhance the feature extraction capability of tiny defect areas. Based on the backbone network, a multi-scale feature fusion strategy is configured to fuse shallow high-resolution features with deep semantic features, and feature maps of different scales are generated through a feature pyramid network.

[0092] Furthermore, the defect classification module M300 is further configured to perform the following method:

[0093] An adaptive parameter optimization mechanism is set up to trigger reinforcement learning-driven dynamic compensation of exposure parameters when the detected ambient light changes exceed the preset change range. At the same time, a rotor defect knowledge graph is constructed, linking historical maintenance work orders with the material failure database to support natural language queries.

[0094] Example 3. Based on the same inventive concept of the internal mixer rotor defect detection method based on image recognition in the above-mentioned Example 1, the present invention also provides an electronic device, including: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in the above-mentioned Example 1.

[0095] like Figure 3 As shown, the bus architecture is represented by bus 300, which can include any number of interconnected buses and bridges. Bus 300 connects various circuits including one or more processors represented by processor 302 and memory represented by memory 304. Bus 300 can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. Bus interface 305 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 can be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 302 is responsible for managing bus 300 and general processing, while memory 304 can be used to store data used by processor 302 when performing operations.

[0096] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting defects in internal mixer rotors based on image recognition, characterized in that: The method comprises: Extract multimodal features of the rotor surface based on the vibration sensor signal to determine the frequency domain characteristic parameters and time domain characteristic parameters associated with the rotor operating state; The contrast of the internal mixer rotor image is optimized, and the defect visual feature vector is configured by combining the frequency domain feature parameters and the time domain feature parameters; the vibration sensor signal is decomposed, the energy features of different frequency bands are extracted, and the vibration signal feature vector is configured; The defect visual feature vector is fused with the vibration signal feature vector, and a defect classification network is used to introduce an attention mechanism to dynamically focus on the tiny defect area, and output a quantitative evaluation result of the crack grade, deformation amount and wear degree; When the quantitative evaluation results of the crack level, deformation and wear degree meet the preset threshold conditions, an audible and visual reminder instruction is triggered and the robotic arm is linked to perform an emergency stop operation; Among them, the contrast of the internal mixer rotor image is optimized, and the defect visual feature vector is configured by combining the frequency domain feature parameters and the time domain feature parameters, including: Constructing a dynamic weighted atlas for capturing early abnormal features including metal cracks, rubber aging, and assembly gaps; Based on the dynamic weight map, an adaptive contrast stretching variable is introduced to automatically adjust the gamma value of the rubber rotor material / plastic rotor material; According to the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized to configure the defect visual feature vector; The step of decomposing the vibration sensor signal further includes: Extracting the intrinsic mode function of the vibration sensor signal through empirical mode decomposition, combining with wavelet transform to eliminate noise interference, and obtaining the frequency band energy distribution characteristics of mechanical fault factors, wherein the mechanical fault factors include bearing wear and gear tooth breakage; Through joint analysis in time and frequency domains, shock events are identified and sparse coding is performed on sudden shock signals.

2. The internal mixer rotor defect detection method based on image recognition according to claim 1, characterized in that: Performing multimodal feature extraction on a rotor surface according to a vibration sensor signal, the method comprising: Deploy multispectral imaging equipment to eliminate oil interference on the rotor surface through phase unwrapping and obtain surface roughness characteristics; At the same time, the laser point cloud and the rotor surface image are combined to dynamically correct the geometric distortion caused by high temperature deformation.

3. The method for detecting defects in an internal mixer rotor based on image recognition according to claim 2, wherein: The defect visual feature vector is fused with the vibration signal feature vector, and a defect classification network is used. The method includes: A graph convolutional network unit is embedded in the encoder to propose the topological relationship of the rotor joints and capture the local deformation characteristics caused by thermal stress; A cross-modal attention mechanism is introduced in the decoder to coordinate the weights of the defect visual feature vector and the vibration signal feature vector.

4. The method for detecting defects in an internal mixer rotor based on image recognition according to claim 3, wherein: The method further comprises: Deploying a generative adversarial network, wherein the generative adversarial network is used to repair the image of the internal mixer rotor and generate a synthetic image of the rotor's high-temperature deformation area through adversarial training; According to the synthesized image, the weight distribution of the defect visual feature vector is updated.

5. The internal mixer rotor defect detection method based on image recognition according to claim 1, characterized in that: An attention mechanism is introduced to dynamically focus on small defect areas. The method further includes: Embed CBAM units in the backbone network to enhance the feature extraction capability of tiny defect areas; Based on the backbone network, a multi-scale feature fusion strategy is configured to fuse shallow high-resolution features with deep semantic features, and generate feature maps of different scales through a feature pyramid network.

6. The internal mixer rotor defect detection method based on image recognition according to claim 5, characterized in that: The method comprises: Set up an adaptive parameter optimization mechanism. When the detected ambient light changes exceed the preset change range, it triggers reinforcement learning-driven dynamic compensation of exposure parameters. At the same time, a rotor defect knowledge graph is constructed to associate historical maintenance work orders with the material failure database, supporting natural language queries.

7. The internal mixer rotor defect detection system based on image recognition is characterized by: A system for implementing the internal mixer rotor defect detection method based on image recognition according to any one of claims 1 to 6, comprising: A feature extraction module is used to extract multimodal features of the rotor surface based on the vibration sensor signal and determine the frequency domain feature parameters and time domain feature parameters associated with the rotor operation state; A contrast optimization module is used to optimize the contrast of the internal mixer rotor image, configure a defect visual feature vector by combining the frequency domain feature parameters and the time domain feature parameters; decompose the vibration sensor signal, extract the energy features of different frequency bands, and configure a vibration signal feature vector; A defect classification module is used to fuse the defect visual feature vector with the vibration signal feature vector, use a defect classification network, introduce an attention mechanism to dynamically focus on small defect areas, and output quantitative evaluation results of crack grade, deformation, and wear degree; A defect reminder module is used to trigger an audible and visual reminder command and link the robotic arm to perform an emergency stop operation when the quantitative evaluation results of the crack level, deformation and wear degree meet the preset threshold conditions; Among them, the contrast of the internal mixer rotor image is optimized, and the defect visual feature vector is configured by combining the frequency domain feature parameters and the time domain feature parameters, including: Constructing a dynamic weighted atlas for capturing early abnormal features including metal cracks, rubber aging, and assembly gaps; Based on the dynamic weight map, an adaptive contrast stretching variable is introduced to automatically adjust the gamma value of the rubber rotor material / plastic rotor material; According to the adaptive contrast stretching variable, the contrast of the internal mixer rotor image is optimized to configure the defect visual feature vector; The vibration sensor signal is decomposed, and the system is further configured to perform the following method: Extracting the intrinsic mode function of the vibration sensor signal through empirical mode decomposition, combining with wavelet transform to eliminate noise interference, and obtaining the frequency band energy distribution characteristics of mechanical fault factors, wherein the mechanical fault factors include bearing wear and gear tooth breakage; Through joint analysis in time and frequency domains, shock events are identified and sparse coding is performed on sudden shock signals.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the internal mixer rotor defect detection method based on image recognition are implemented as described in any one of claims 1 to 6.

Images