Equipment defect comprehensive detection system and method based on three modes

Abstract

The invention relates to the technical field of equipment defect detection, and discloses an equipment defect comprehensive detection system and method based on three modes. The system comprises a multi-modal data acquisition module which acquires infrared temperature data, X-ray image data and visual image data of equipment in real time; the comprehensive data processing module processes the data and generates standardized defect features; the defect risk assessment module assesses the comprehensive risk level of the defect based on the characteristics; the detection stage division module divides the detection process into a plurality of stages and determines a detection focus of each stage; the parameter rejection degree evaluation module evaluates the rejection degree of the detection parameters based on the equipment material characteristic data and the detection focus; the adjustment amount determination module determines the adjustment amount of the detection parameter based on the rejection degree and the risk level; and the detection regulation and control module regulates and controls the detection parameters according to the adjustment amount and executes defect processing actions. According to the system, integrated detection of multiple types of defects can be realized, and the detection flexibility and practicability are improved.

Description

Technical Field

[0001] This invention relates to the field of equipment defect detection technology, specifically to a comprehensive equipment defect detection system and method based on three modes. Background Technology

[0002] Equipment defect detection plays a crucial role in industrial production and equipment maintenance, especially in packaging and equipment sealing, where defects can lead to product malfunctions or even safety accidents. Traditional defect detection methods typically rely on a single sensing technology, such as using only one of infrared thermography, X-ray imaging, or visual image analysis. Infrared thermography can detect abnormal temperature rises caused by defects in equipment, but it cannot identify structural damage or surface cracks; X-ray imaging can see through internal structures and detect hidden defects such as internal cracks or foreign objects, but it is not sensitive to surface defects and temperature anomalies; visual image detection is suitable for identifying external defects such as cracks, pinholes, or stains, but it is greatly affected by factors such as ambient light and obstruction, and cannot detect internal or temperature-related defects.

[0003] Due to the limitations of the aforementioned single detection methods, practical applications often require a combination of multiple detection techniques. However, current technologies often process multi-sensor data independently, lacking an effective information fusion mechanism. Different modal data exhibit significant differences in characteristic scale, physical meaning, and data format, making direct correlation and comprehensive analysis difficult, resulting in insufficient accuracy and completeness in defect assessment. Furthermore, the detection process is typically static, failing to dynamically adjust detection parameters based on defect severity or equipment material characteristics, leading to poor flexibility and low resource utilization efficiency. Especially when multiple defect types coexist, a single detection strategy struggles to address the identification needs of all defect types, easily resulting in missed or false detections.

[0004] Existing multimodal inspection systems attempt to integrate multiple sensors, but most remain at the level of simple data overlay, lacking unified quantification and assessment of defect risks, and failing to achieve phased optimization and dynamic parameter adjustment of the inspection process. For example, while some systems can simultaneously collect multiple types of data, the data processing is independent, resulting in fragmented assessments and an inability to form a holistic judgment on the comprehensive impact of defects. Other systems, although incorporating risk assessment, fail to consider the influence of equipment material properties on inspection parameters, leading to limited inspection accuracy. Therefore, there is an urgent need for an integrated system capable of integrating multimodal data, performing comprehensive feature extraction and risk assessment, and adaptively adjusting inspection strategies to improve the comprehensiveness, accuracy, and efficiency of defect detection. Summary of the Invention

[0005] The purpose of this invention is to provide a comprehensive equipment defect detection system and method based on three modes to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a comprehensive equipment defect detection system and method based on three modes, the system comprising: The multimodal data acquisition module is used to acquire infrared temperature data, X-ray image data and visual image data of the equipment in real time; The integrated data processing module is used to process the data collected by the multimodal data acquisition module to generate standardized defect features; The defect risk assessment module is used to assess the overall risk level of defects based on the standardized defect features generated by the integrated data processing module. The detection phase division module is used to divide the detection process into multiple phases and determine the detection focus of each phase based on the evaluation results of the defect risk assessment module. The parameter rejection evaluation module is used to acquire equipment material property data and evaluate the rejection of detection parameters based on the detection focus of the detection stage division module. The adjustment amount determination module is used to determine the adjustment amount of the detection parameter based on the evaluation rejection degree of the parameter rejection degree evaluation module and the risk level of the defect risk evaluation module; The detection and control module is used to adjust the detection parameters and perform defect handling actions based on the adjustment amount of the module.

[0007] Preferably, the multimodal data acquisition module includes an infrared temperature sensing unit, an X-ray defect recognition unit, and a visual inspection unit; The infrared temperature sensing unit is used to monitor temperature anomalies in the equipment. The X-ray defect identification unit is used to identify packaging damage defects; The visual inspection unit is used to detect packaging ruptures and pinhole defects; The infrared temperature sensing unit, the X-ray defect identification unit, and the visual inspection unit include a dot-matrix distribution structure; the dot-matrix distribution structure sets the sensor positions based on the area analysis of the equipment region to achieve uniform data acquisition.

[0008] Preferably, the integrated data processing module includes a data conversion unit and a feature extraction unit; The data conversion unit is used to convert the raw data collected by the multimodal data acquisition module into a time series analysis format; The feature extraction unit is used to extract defect features from the time series analysis format converted by the data conversion unit.

[0009] Preferably, the defect risk assessment module includes a risk analysis unit and a priority determination unit; The risk analysis unit is used to analyze the interactions and potential impacts between defects; The priority determination unit is used to determine the priority order of defect handling based on the analysis results of the risk analysis unit.

[0010] Preferably, the detection stage division module includes a stage definition unit and a focus allocation unit; The stage definition unit is used to divide the detection stage according to a preset time interval; The focus allocation unit is used to allocate a detection focus area for each stage based on the risk level of the defect risk assessment module.

[0011] Preferably, the parameter rejection evaluation module includes a data acquisition unit and a rejection calculation unit; The data acquisition unit is used to acquire equipment material property data in real time; The repulsion calculation unit is used to calculate the repulsion index of the detection parameters based on the detection focus of the detection stage division module and the material property data obtained by the data acquisition unit.

[0012] Preferably, the adjustment amount determination module includes an adjustment calculation unit and an amount allocation unit; The adjustment calculation unit is used to calculate the adjustment value based on the rejection index of the parameter rejection evaluation module and the risk level of the defect risk evaluation module; The value allocation unit is used to allocate the adjustment value calculated by the adjustment calculation unit to the corresponding detection parameter.

[0013] Preferably, the detection and control module includes a parameter adjustment unit and an action execution unit; The parameter adjustment unit is used to modify the detection parameter settings according to the adjustment amount determined by the adjustment amount determination module; The action execution unit is used to control the defect processing executor based on the priority order of the defect risk assessment module.

[0014] Preferably, the system further includes: The human-computer interaction module is used to display and record system data and allow users to change system parameters; The human-computer interaction module includes a data display unit and a parameter input unit; the data display unit is used to display in real time the standardized defect characteristics of the integrated data processing module and the risk level of the defect risk assessment module; the parameter input unit is used to receive parameter change instructions input by the user.

[0015] Preferably, the present invention also includes a comprehensive equipment defect detection method based on three modes, the method comprising all modules and method flow of the above-mentioned comprehensive equipment defect detection system based on three modes.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention achieves multi-dimensional perception and comprehensive analysis of equipment defects by integrating data acquisition and processing from three modalities: infrared temperature, X-ray images, and visual images. The system generates standardized defect features, eliminating scale and format differences between multi-source data, enabling unified expression and comparison of different defect information. The defect risk assessment module quantifies and rates the overall impact of defects based on the fused features, helping users intuitively grasp the severity of defects, prioritize them, and optimize the processing order.

[0017] The testing phase segmentation module divides the testing process into different stages and dynamically allocates the testing focus of each stage based on risk level. This allows the system to adapt to complex and ever-changing testing environments and avoids wasting resources on low-risk stages. The parameter exclusion assessment module incorporates equipment material characteristic data to analyze the mutual exclusion relationships between different testing parameters, reducing parameter setting conflicts and improving testing coordination. The adjustment amount determination module combines exclusion and risk level to calculate the specific adjustment direction and magnitude of testing parameters, providing a clear basis for system control.

[0018] The detection and control module adjusts the detection parameters in real time based on the adjustment amount and triggers corresponding defect handling actions, forming a closed-loop control for the entire detection process and enhancing the system's responsiveness and adaptability. The collaborative use of multimodal data expands the defect identification range, enabling the system to simultaneously detect temperature anomalies, internal structural defects, and surface defects. This reduces the blind spots of single-sensor detection and enhances the ability to identify multiple types of defects, such as simultaneously detecting packaging damage, loose seals, and pinhole leaks.

[0019] The system boasts excellent flexibility and practicality, capable of running three detection modes simultaneously or selecting any combination of modes to meet diverse application scenarios and defect types. The entire system, from data acquisition and processing to execution and control, forms an integrated pipeline, minimizing manual intervention, reducing subjective errors, and improving the level of automation and consistency of results. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the working principle of the trimodal integrated equipment defect detection system described in this invention. Figure 2 This is a schematic diagram illustrating the working principle of the multimodal data acquisition module. Figure 3 This is a schematic diagram of the working principle of the integrated data processing module; Figure 4 This is a schematic diagram illustrating the working principle of the parameter rejection evaluation module. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 The present invention provides a comprehensive equipment defect detection system based on three modes. The system includes: a multimodal data acquisition module, a comprehensive data processing module, a defect risk assessment module, a detection stage division module, a parameter rejection assessment module, an adjustment amount determination module, and a detection control module.

[0023] The multimodal data acquisition module acquires infrared temperature data, X-ray image data, and visual image data from the equipment in real time. The integrated data processing module processes this data to generate standardized defect features. The defect risk assessment module evaluates the overall risk level of defects based on these standardized defect features. The detection stage segmentation module divides the detection process into multiple stages and determines the detection focus for each stage based on the risk level. The parameter rejection assessment module acquires equipment material property data and evaluates the rejection of detection parameters based on the detection focus. The adjustment amount determination module determines the adjustment amount of the detection parameters based on the rejection amount and the risk level. The detection control module adjusts the detection parameters and executes defect handling actions according to the adjustment amount.

[0024] Example 1: See Figure 2 This embodiment relates to the specific structure and operation mechanism of a multimodal data acquisition module. This module consists of three core units: an infrared temperature sensing unit, an X-ray defect identification unit, and a visual inspection unit. These units are deployed through a specific dot-matrix distribution structure to achieve comprehensive monitoring of the equipment status.

[0025] The infrared temperature sensing unit is primarily responsible for monitoring abnormal temperature conditions in the equipment. This unit employs an array-type infrared sensor layout, using a non-contact temperature measurement device based on the thermopile principle (model: MLX90614-DCI). Its specific performance parameters are as follows: temperature measurement range covers -70℃ to 380℃, adaptable to common temperature ranges in industrial equipment; temperature measurement accuracy reaches ±0.5℃ within the core operating range of -10℃ to 85℃, and ±1℃ within the range of 85℃ to 380℃; the default sampling frequency is set to 1Hz, which can be dynamically adjusted to a maximum of 10Hz according to the equipment's operating speed, meeting real-time monitoring needs under different operating conditions; the communication interface uses an I2C bus with a transmission rate of 400kHz, ensuring real-time data transmission. During deployment, the equipment surface area is first divided into a grid, dividing the area to be monitored into several sub-regions. An infrared sensor node is placed at the center point of each sub-region, with the node spacing calculated and determined based on the equipment's thermal conductivity characteristics and monitoring accuracy requirements. The sensor nodes are connected via a CAN bus network, forming a distributed temperature measurement system. Each sensor collects temperature data at a fixed sampling frequency, and the collected data includes temperature values ​​and timestamp information. The data is transmitted to the central processing unit via industrial Ethernet, and data compression algorithms are used during transmission to reduce bandwidth consumption.

[0026] The X-ray defect identification unit focuses on identifying packaging damage defects. This unit consists of an X-ray emitting device and a digital detector array. The X-ray source uses a microfocus X-ray tube (model: Hamamatsu L8121-03), with an adjustable tube voltage range of 50kV~130kV and a tube current range of 50μA~200μA, dynamically adaptable to the thickness of the packaging material. The detector is a flat panel detector (model: Toshiba TDI-4000), with a pixel size of 100μm×100μm, an image resolution of 4096×3000 pixels, a frame rate of 15fps, and an imaging contrast ≥40dB when penetrating 3mm thick aluminum alloy (a common material in equipment packaging), ensuring clear visibility of defects. The emitting device uses a microfocus X-ray source with an adjustable power range covering 50kV to 130kV. The detector array consists of multiple flat panel detectors arranged in a dot matrix around the equipment. The arrangement was determined through simulation calculations to ensure that the X-ray beam can cover all areas requiring inspection. During the detection process, the X-ray source emits pulsed rays, and the detector receives the ray signals after they penetrate the equipment and converts them into digital images. The image resolution is adjustable from 100μm to 500μm according to the detection requirements. After preliminary noise reduction processing, the acquired image data is transmitted to the image processing unit via gigabit Ethernet. The transmission protocol adopts a dedicated industrial image transmission standard to ensure data integrity and real-time performance.

[0027] The vision inspection unit is used to detect packaging ruptures and pinhole defects. This unit comprises multiple industrial-grade color cameras and a matching lighting system. The cameras utilize high-resolution CMOS sensors with a resolution of 4096×3000 pixels. The cameras are arranged using stereo vision principles, forming a multi-angle observation network around the equipment. Each camera is equipped with a ring-shaped LED illumination device, with brightness and angle remotely adjustable. Camera acquisition is triggered by synchronous signals, ensuring simultaneous image capture by multiple cameras. The acquired images undergo color and geometric correction before being converted to a standard image format. Image data is transmitted via a fiber optic network, with a transmission rate sufficient for real-time processing.

[0028] The implementation of the matrix-based distribution structure is based on equipment area analysis. First, a 3D model of the equipment surface is established, and the monitoring importance weight of each area is determined through finite element analysis. The sensor deployment density is calculated based on the weight distribution, with higher density in important areas and lower density in ordinary areas. Sensor location coordinates are calculated using an optimization algorithm to ensure no blind spots and optimal cost. The installation locations of all sensor nodes are recorded in a configuration file, and the system initializes data acquisition parameters based on this configuration file during runtime. The specific technical implementation steps are as follows: 1. Create a 3D model of the equipment surface. Using 3D modeling software such as SolidWorks and CATIA, a precise 3D model of the equipment's surface is constructed based on its actual size, shape, and structural features. During the modeling process, it is crucial to ensure the model's accuracy and detail to accurately reflect the equipment's true form. For example, for complex mechanical equipment, it is necessary to model the surface shape and relative positional relationships of each component in detail.

[0029] Collect relevant design drawings, measurement data, and other information about the equipment to provide an accurate basis for modeling. Point cloud data of the equipment surface can be obtained using measurement technologies such as laser scanning. Then, the point cloud data can be imported into modeling software for processing and fitting to generate a 3D model.

[0030] 2. Finite element analysis to determine the importance weight of monitoring. Import the completed 3D model into finite element analysis software such as ANSYS or ABAQUS. Based on the equipment's working environment, stress conditions, and operating parameters, set appropriate boundary and load conditions. For example, for equipment subjected to high temperature and high pressure, corresponding temperature and pressure loads need to be set.

[0031] Finite element analysis software is used to simulate and analyze the equipment using multiple physical fields, including mechanical and thermal fields, to obtain the distribution of physical quantities such as stress, strain, and temperature in different regions of the equipment. Based on the distribution of these physical quantities, the monitoring importance weight of each region is determined. For example, regions with concentrated stress or large temperature variations have relatively high monitoring importance weights.

[0032] 3. Calculate sensor deployment density Based on the importance weights of each monitoring area obtained from finite element analysis, calculation rules for sensor deployment density are established. A linear mapping method can be used to map the weight values ​​to the range of sensor deployment density values. For example, the higher the weight value, the greater the corresponding sensor deployment density.

[0033] The calculated sensor deployment density should be adjusted based on the actual conditions of the equipment and monitoring requirements. For example, in some special areas, such as critical parts of the equipment or areas prone to failure, the sensor deployment density needs to be appropriately increased to ensure the accuracy and reliability of monitoring.

[0034] 4. Optimize the algorithm to calculate sensor position coordinates Choose an appropriate optimization algorithm, such as a genetic algorithm or a particle swarm optimization algorithm, to calculate the sensor's position coordinates. These optimization algorithms can find the optimal sensor placement scheme while satisfying the conditions of no blind spots in coverage and optimal cost.

[0035] Define the optimization objective function, which typically includes factors such as sensor coverage and cost. For example, coverage can be represented by the area of ​​the device that the sensor can monitor, and cost can be represented by factors such as the number of sensors and installation costs.

[0036] Set the parameters of the optimization algorithm, such as population size, number of iterations, crossover probability, mutation probability, etc., and perform multiple iterative calculations until the optimal sensor location coordinates are found.

[0037] 5. Record the sensor installation location and initialize the data acquisition parameters. The calculated installation locations of all sensor nodes are recorded in a configuration file, which can be in text, XML, or other formats. The recorded information includes the sensor's number, location coordinates, and type.

[0038] During system operation, the system reads the sensor installation location information from the configuration file and initializes the data acquisition parameters based on this information. For example, it sets parameters such as the sensor's sampling frequency and range to ensure that the sensor can accurately collect monitoring data from various areas of the equipment.

[0039] Therefore, based on the analysis of the equipment area, a three-dimensional model of the equipment surface can be established, and finite element analysis can be used to determine the monitoring importance weight of each area. This allows for the calculation of sensor deployment density and location coordinates, thus achieving a matrix-like sensor layout. Specifically: The 3D model is the foundation of the entire technology, providing geometric information about the device surface and offering an accurate spatial reference for subsequent finite element analysis and sensor placement. Finite element analysis simulates the physical field distribution of the device during actual operation, determining the monitoring importance weights for each area. These weights reflect the importance of each area and provide a basis for sensor deployment. The sensor deployment density is calculated based on these importance weights, enabling denser monitoring of critical areas and improving monitoring accuracy and reliability. Optimization algorithms search for the optimal sensor location coordinates to ensure sensor coverage of all areas of the device while reducing costs. Configuration files record sensor installation location information; during system runtime, data acquisition parameters are initialized according to the configuration files, enabling the sensors to function normally and collect monitoring data from each area of ​​the device.

[0040] Suppose we want to monitor a rectangular device with length L = 10m, width W = 5m, and height H = 3m. The steps are as follows: Create a 3D model: Use SolidWorks software to create a 3D model of the cuboid device with a resolution of 0.01m.

[0041] Finite element analysis determines the importance weight of monitoring: Import the 3D model into ANSYS software and set the boundary conditions as follows: the bottom of the equipment is fixed, and a uniformly distributed vertical load P = 1000 N / m² is applied to the top.

[0042] Mechanical analysis was performed to obtain the stress distribution in different areas of the equipment. Based on the stress magnitude, the equipment surface was divided into three regions: a high-stress region (stress greater than 50 MPa), a medium-stress region (stress between 20 and 50 MPa), and a low-stress region (stress less than 20 MPa).

[0043] The monitoring importance weights for the high-stress region, medium-stress region, and low-stress region were assigned as 0.8, 0.5, and 0.2, respectively.

[0044] Calculate sensor deployment density: Assuming the sensor deployment density ranges from [1, 10] units / m², the sensor deployment density for each region is calculated using a linear mapping method.

[0045] Sensor deployment density in high-stress areas: Items / m², rounded down to 8 items / m².

[0046] Sensor deployment density in medium stress areas: Items / m², rounded to 6 items / m².

[0047] Sensor deployment density in low-stress areas: Items / m², rounded down to 3 items / m².

[0048] The optimized algorithm calculates the sensor position coordinates: We selected a genetic algorithm for optimization calculations, setting the population size to 50, the number of iterations to 100, the crossover probability to 0.8, and the mutation probability to 0.1.

[0049] Define the optimization objective function: Where $C$ is the coverage area of ​​the sensor (range [0, 1]), and $N$ is the number of sensors. and For the weighting coefficients, take respectively , .

[0050] After multiple iterative calculations, the optimal sensor position coordinates were obtained.

[0051] Record the sensor installation location and initialize the data acquisition parameters: Record the calculated sensor position coordinates in a text file, with the following file format: Sensor ID, X coordinate, Y coordinate, Z coordinate, Sensor type 1, 1.0, 2.0, 0.5, Stress sensor 2, 2.0, 3.0, 1.0, Temperature sensor ... When the system is running, it reads the text file and initializes the data acquisition parameters according to the sensor type and location coordinates. For example, it sets the sampling frequency of the stress sensor to 10Hz and the range to 0-100MPa; and the sampling frequency of the temperature sensor to 5Hz and the range to 0-100℃.

[0052] Therefore, by determining the monitoring importance weight of each area through finite element analysis and deploying sensors according to the weight distribution, key areas of the equipment can be monitored, improving the accuracy and reliability of monitoring. Under the premise of ensuring coverage without blind spots, the sensor position coordinates can be calculated through optimization algorithms, allowing for the rational arrangement of the number and location of sensors, reducing sensor usage and installation costs. This ensures that sensors can cover all areas of the equipment, avoiding monitoring blind spots and improving the comprehensiveness of equipment monitoring. During system operation, data acquisition parameters are initialized according to the configuration file, ensuring that sensors accurately collect monitoring data from each area of ​​the equipment, improving data accuracy and reliability.

[0053] The workflow of the infrared temperature sensing unit includes temperature data acquisition, data preprocessing, and data transmission. The acquired temperature data is first compensated for by ambient temperature to eliminate the influence of environmental factors. Then, noise filtering is performed, and a moving average algorithm is used to smooth data fluctuations. The processed temperature data is then stored in association with the device coordinates to form a temperature distribution map.

[0054] The X-ray defect identification unit's execution process includes parameter setting, image acquisition, and image preprocessing. The system automatically sets X-ray parameters, including voltage, current, and exposure time, based on the equipment's material properties. The acquired images undergo dark-field correction and flat-field correction to eliminate inherent detector noise. The corrected images then undergo contrast enhancement processing to improve defect visibility.

[0055] The operation of the visual inspection unit includes illumination control, image acquisition, and image optimization. The system automatically adjusts the illumination brightness according to the ambient light intensity to ensure stable image quality. The acquired images undergo automatic white balance processing to eliminate color cast. Simultaneously, image sharpening processing is performed to enhance the clarity of edge details.

[0056] The data acquisition process of the three units maintains time synchronization, with all data records using a unified timestamp. Time synchronization is achieved through a precision clock protocol, achieving microsecond-level accuracy. Acquired data is uploaded to the data buffer in real time, awaiting processing by the integrated data processing module.

[0057] Maintenance of the dot-matrix distributed structure includes regular calibration and fault diagnosis. The system is equipped with an automatic calibration function to periodically check sensor accuracy deviations. When deviations exceed the allowable range, it automatically performs software compensation or prompts for hardware maintenance. The fault diagnosis function monitors the operating status of each sensor in real time and issues an alarm immediately upon detecting anomalies.

[0058] During data acquisition, the system dynamically adjusts the acquisition parameters based on the equipment's operating status. When the equipment is operating at high speed, the acquisition frequency is automatically increased; when the equipment is operating at low speed, the acquisition frequency is appropriately reduced to conserve resources. This adaptive adjustment mechanism is achieved through real-time monitoring of the equipment's speed signal.

[0059] All acquisition units feature redundancy, with backup sensors deployed in critical areas. When the primary sensor fails, the system automatically switches to the backup sensor, ensuring continuous data acquisition. The switching process requires no manual intervention; the system automatically performs sensor status detection and channel switching.

[0060] Data transmission employs a dual verification mechanism, including cyclic redundancy check and parity check. Data that fails a verification is automatically retransmitted to ensure data integrity. The transmission link status is monitored in real time, and transmission parameters are automatically adjusted when a degradation in transmission quality is detected.

[0061] The multimodal data acquisition module reliably acquires infrared temperature data, X-ray image data, and visual image data from the equipment. Its matrix-based distribution structure ensures comprehensive and uniform data acquisition, providing high-quality raw data for subsequent processing. Coordination between units is achieved through a unified time base and communication protocol, guaranteeing the temporal consistency and spatial correlation of the multimodal data.

[0062] See Example 2 Figure 3 This embodiment relates to the specific operation flow and method of the integrated data processing module and the defect risk assessment module. The integrated data processing module receives raw data from the multimodal data acquisition module, including temperature readings acquired by the infrared temperature sensing unit, X-ray images acquired by the X-ray defect identification unit, and visual images captured by the visual inspection unit. This data is transmitted through a dedicated data interface, and a timestamp synchronization mechanism is used during transmission to keep the data from different modalities aligned in the time dimension. The data conversion unit first performs format unification processing on the input raw data. The infrared temperature data, originally in the form of voltage signals, is converted into temperature values ​​through a calibration curve and arranged into a time series according to the acquisition time order. Each temperature data point carries a precise timestamp and equipment spatial coordinate information. X-ray image data is input in the form of frames, with each frame corresponding to a specific acquisition time. The data conversion unit organizes these image frames in chronological order to generate an X-ray image sequence, while recording the acquisition parameters of each frame, including X-ray intensity, exposure time, and detector gain. The visual image data is also arranged in chronological order to form a visual image sequence, and the acquisition conditions of each frame, such as illumination intensity and camera focal length, are recorded. All time series data use a standardized data format, including header information and a data body. The header information records metadata such as the start time of the sequence, sampling interval, data length, and data type; the data body stores the specific numerical values ​​or image pixel values. This time series analysis format facilitates subsequent time series feature analysis and multimodal data correlation.

[0063] The feature extraction unit extracts defect features from the transformed time-series data. For infrared temperature time series, signal processing techniques are used to analyze temperature change patterns. Statistical features such as mean, variance, and extreme values ​​are calculated by analyzing the statistical distribution of temperature values. Simultaneously, time-domain analysis methods are used to calculate the rate and acceleration characteristics of temperature change. Frequency-domain analysis obtains the spectral characteristics of the temperature signal through transformation, identifying periodic temperature fluctuation patterns. For X-ray image sequences, image processing algorithms analyze image content frame by frame. First, image enhancement is performed to improve the contrast of defect areas; then, edge detection algorithms are used to identify the contour features of packaging damage. Morphological processing is used to extract the geometric features of defects, including area, perimeter, and shape factor; texture analysis algorithms calculate the texture features of defect areas, such as roughness and directionality. For visual image sequences, similar image analysis methods are used, but adjusted for the characteristics of surface defects. Pinhole defect detection uses a high-precision edge detection algorithm to identify small discontinuous areas. Color analysis is used to detect color changes caused by packaging breakage. All extracted feature values ​​are normalized and converted into standardized defect features with zero mean and unit variance. These features are organized according to feature type and device region to form feature vectors for use by subsequent modules.

[0064] Specifically: 1. Signal processing techniques for analyzing temperature change patterns 1) Calculation of statistical characteristics Mean calculation: Sum all temperature values ​​in the infrared temperature time series and divide by the number of temperature values ​​to obtain the mean of the temperature series. The mean reflects the average temperature level within that time period and can be used to determine the overall temperature level. For example, in an industrial production environment, if the average infrared temperature of a piece of equipment is too high, it may indicate that the equipment has an abnormal heating problem.

[0065] Variance calculation: First, calculate the difference between each temperature value and the mean. Square these differences, sum them, and then divide by the number of temperature values ​​to obtain the variance. Variance reflects the degree of dispersion of temperature values ​​relative to the mean. A larger variance indicates greater temperature fluctuations, which may suggest unstable equipment operation or the presence of external interference factors affecting the temperature.

[0066] Extreme value calculation: Identify the maximum and minimum values ​​in the infrared temperature time series. The maximum and minimum values ​​directly reflect the range of temperature variation, which is crucial for determining whether equipment is experiencing extreme overheating or undercooling. For example, in high-temperature processes, if the maximum temperature exceeds the safety threshold, it may lead to a safety accident.

[0067] 2) Time Domain Analysis Temperature change rate calculation: For the temperature values ​​of two adjacent time points in an infrared temperature time series, calculate their difference, then divide by the time interval to obtain the temperature change rate within that time period. The temperature change rate reflects how quickly the temperature rises or falls. For example, during the equipment startup phase, if the temperature change rate is too fast, it may indicate a malfunction in the equipment or an abnormal startup process.

[0068] Temperature change acceleration calculation: Calculate the difference between the rates of temperature change in two adjacent time periods, then divide by the time interval to obtain the temperature change acceleration. Acceleration further describes the changes in the rate of temperature change, helping to detect sudden changes in temperature trends and providing early warnings of potential equipment problems.

[0069] 3) Frequency Domain Analysis Spectral Feature Acquisition: Frequency domain transformation methods such as Fourier transform are used to convert the infrared temperature time series from the time domain to the frequency domain. In the frequency domain, the spectral features of the temperature signal can be obtained, including the amplitude and phase of different frequency components. By analyzing these spectral features, periodic temperature fluctuation patterns can be identified. For example, some devices, due to periodic operating cycles, will produce temperature fluctuations at specific frequencies; these features can be accurately captured through frequency domain analysis.

[0070] 2. Implementation methods for X-ray image sequence feature extraction technology 1) Frame-by-frame analysis of image processing algorithms Contrast Enhancement: X-ray images are processed using methods such as histogram equalization. By adjusting the grayscale distribution of the image, the contrast between defective areas and surrounding normal areas becomes more pronounced. This enhances the identifiability of defects in the image, facilitating subsequent feature extraction. For example, when detecting foreign objects inside food packaging, enhanced contrast makes the foreign objects appear more clearly in the image.

[0071] 2) Edge detection Contour Feature Recognition: Edge detection algorithms such as the Sobel operator and Canny operator are used to process the enhanced X-ray image to detect the edges of the damaged area of ​​the packaging. These edges constitute the contour features of the damaged area. By analyzing the shape and size of the contour, the degree and type of damage can be preliminarily determined.

[0072] 3) Morphological treatment Geometric feature extraction: Morphological operations such as dilation, erosion, opening, and closing are used to process the edge-detected image, removing noise and filling the interior of the damaged area. Then, the geometric features of the defect are calculated, such as area, perimeter, and shape factor. Area reflects the size of the damaged area, perimeter reflects the complexity of the damage, and shape factor describes the shape characteristics of the damage, such as circular or elliptical.

[0073] 4) Texture analysis Texture feature calculation: Texture analysis algorithms such as gray-level co-occurrence matrix are used to calculate the texture features of the defect region, such as roughness and orientation. Roughness reflects the surface roughness of the defect region, while orientation reflects the texture direction of the defect. These texture features can help further distinguish different types of defects.

[0074] 3. Implementation methods for visual image sequence feature extraction technology 1) Pinhole defect detection High-precision edge detection: To address potential pinhole defects in visual images, high-precision edge detection algorithms, such as multi-scale edge detection methods, are used to identify minute discontinuous regions. These discontinuous regions may be the edges of pinhole defects. By accurately detecting these edges, the location and size of the pinhole can be precisely determined.

[0075] 2) Color Change Detection: This involves analyzing color changes in visual images caused by packaging rupture. Color characteristics of different areas, such as RGB and HSV values, can be calculated to compare the color differences between the ruptured and normal areas. If the color difference exceeds a certain threshold, it can be determined that there is a packaging rupture issue.

[0076] 4. Feature normalization and eigenvector organization 1) Feature normalization processing Standardization Transformation: For all extracted feature values, a normalization method is used to transform them into standardized defect features with zero mean and unit variance. Specifically, the mean and standard deviation of each feature are first calculated, then the mean is subtracted from each feature value, and then the result is divided by the standard deviation to obtain the standardized feature value. This eliminates dimensional differences between different features, making the features comparable in subsequent analysis and processing.

[0077] 2) Feature vector organization Organized by type and region: The standardized defect features are organized according to feature type (such as temperature features, geometric features, texture features, etc.) and equipment region to form feature vectors. Each feature vector contains multiple feature information of a specific equipment region. These feature vectors will serve as input to subsequent modules (such as classifiers, prediction models, etc.) for defect identification, classification, and prediction.

[0078] The defect risk assessment module receives standardized defect features output from the integrated data processing module. The risk analysis unit first preprocesses the feature data, including feature selection and dimensionality reduction. It selects a subset of features highly correlated with defect risk and removes redundant features. Then, it establishes a defect risk model that considers the interactions and potential impacts between defects. Defect interactions are modeled through association analysis, analyzing the co-occurrence relationships and conditional probabilities between different defect types. Potential impact assessment is based on the physical characteristics of the defect and the equipment operating environment, considering the severity of the potential consequences. The risk analysis unit outputs a risk score for each defect, which comprehensively considers the defect's severity, development speed, and scope of impact. The risk scores use a standardized scale for easy comparison of risks between different defects. Specifically: An interface interface technology is employed to establish a stable data transmission interface between the defect risk assessment module and the comprehensive data processing module. Widely used interface specifications such as RESTful APIs ensure standardized and compatible data transmission. Message queues (such as Kafka) can be used to cache and asynchronously transmit data, improving system throughput and stability. After the comprehensive data processing module completes the standardization of defect features, it encapsulates the data into a format conforming to the interface specification (such as JSON or XML) and sends it to the defect risk assessment module over the network. The comprehensive data processing module cleans, transforms, and standardizes the raw defect data according to preset rules, giving it a unified format and data type. The defect risk assessment module listens on a specific port through the interface; upon receiving data, it performs format validation and integrity checks to ensure the accuracy of the received standardized defect feature data.

[0079] Feature data preprocessing Filtering methods: Calculate the correlation between each feature and defect risk. Common methods include Pearson correlation coefficient and chi-square test. For example, for numerical features, the Pearson correlation coefficient can be used to measure their linear correlation with defect risk, retaining features with a correlation higher than a set threshold (e.g., 0.5). Alternatively, machine learning algorithms (such as decision trees and logistic regression) can be used as evaluators to iteratively select the optimal feature subset. For instance, the Recursive Feature Elimination (RFE) algorithm gradually removes features that contribute the least to model performance until a preset number of features is reached. Or, feature selection can be performed automatically during model training, such as using Lasso regression, which penalizes feature coefficients during training, causing some unimportant feature coefficients to become 0, thus achieving feature selection. The purpose of feature selection is to identify features highly correlated with defect risk and remove redundant and irrelevant features. Filtering methods select features based on their statistical properties and evaluate the quality of the feature subset based on model performance.

[0080] Dimensionality reduction: The purpose of dimensionality reduction is to reduce the dimensionality of features, thereby reducing data complexity while preserving the main information. PCA transforms high-dimensional data into low-dimensional data by finding the principal components, while LDA incorporates class information, considering the classification performance of the data while reducing dimensionality. The main methods include: Principal Component Analysis (PCA): This method calculates the covariance matrix of the feature data, then performs eigenvalue decomposition on the covariance matrix to obtain eigenvectors and eigenvalues. Eigenvectors with larger eigenvalues ​​are selected as principal components, and the original feature data is projected onto these principal components, achieving data dimensionality reduction.

[0081] Linear Discriminant Analysis (LDA): Based on category information (such as the level of defect risk), a projection direction is found so that data of different categories are separated as much as possible after projection, while data of the same category are clustered as much as possible, thereby achieving dimensionality reduction.

[0082] Steps to establish a defect risk model: Association rule mining: The Apriori algorithm or FP-growth algorithm is used to mine co-occurrence relationships between different defect types. For example, minimum support and minimum confidence thresholds are set to identify frequent itemsets and association rules. If the support and confidence of the rule "Defect A -> Defect B" are both higher than the set thresholds, then a strong co-occurrence relationship is considered to exist between defect A and defect B.

[0083] Conditional probability calculation: Based on historical data, the conditional probabilities between different defect types are statistically analyzed. For example, the probability P(B|A) of defect B occurring given defect A is estimated using a large amount of sample data.

[0084] Association analysis identifies co-occurrence patterns among different defect types by mining frequent itemsets and association rules in the data. Conditional probability, on the other hand, quantifies the likelihood of another defect occurring given the occurrence of one defect, thus modeling the interactions between defects.

[0085] Potential impact assessment Based on the physical characteristics of the defect (such as size, location, and material) and the operating environment of the equipment (such as temperature, pressure, and humidity), a physical model is established to assess the severity of the potential consequences of the defect. For example, for a crack defect in a mechanical part, a fracture mechanics model can be used to calculate the crack propagation rate and the probability of potential part failure. Knowledge and experience from domain experts are collected to establish a knowledge base and reasoning mechanism. When the physical characteristics of the defect and information about the equipment's operating environment are input, the expert system provides an assessment of the severity of the potential consequences of the defect through the reasoning mechanism.

[0086] Physical model-based methods use physical principles and mathematical formulas to describe the development process of defects and their impact on equipment. Expert systems, on the other hand, leverage the knowledge and experience of experts to assess the potential impact of defects through reasoning and judgment.

[0087] The risk score is calculated and output as follows: Weighting: Each factor is assigned a corresponding weight based on the severity, speed of development, and scope of impact of the defect. For example, severity is weighted at 0.5, speed of development at 0.3, and scope of impact at 0.2.

[0088] Standardization: Standardize the scores for each factor so that their values ​​range from 0 to 1. The min-max standardization method can be used to map the raw scores for each factor to the 0-1 interval.

[0089] Comprehensive calculation: Multiply the standardized scores of each factor by their respective weights, and then sum them to obtain the risk score for each defect. For example, if a defect has a severity score of 0.8, a development speed score of 0.6, and an impact range score of 0.7, then the risk score for this defect = 0.8 × 0.5 + 0.6 × 0.3 + 0.7 × 0.2 = 0.72.

[0090] Risk scoring comprehensively considers multiple factors related to defects. Through weighting and standardization, the scores of different factors are unified onto a standardized scale, facilitating risk comparison between different defects. The final risk score can serve as an important basis for subsequent decisions (such as maintenance prioritization and resource allocation).

[0091] The priority determination unit determines the priority order of defect processing based on the output of the risk analysis unit. This unit employs a dynamic sorting algorithm, ranking defects according to their risk scores. Defects with the same risk score are further ranked according to the time they were discovered. For highly correlated defect groups, they are treated as a whole and assigned priority accordingly. The priority order is updated in real time; when new defects are discovered or the risk scores of existing defects change, the priority is recalculated. The priority determination unit also considers the availability of processing resources; when processing resources are limited, the priority allocation strategy is adjusted. The output priority order is represented in list form, with each entry containing a defect identifier, risk score, and processing priority level. This list is transmitted in real time to subsequent modules to guide the allocation of detection resources and the execution of processing actions. Specifically: (a) Obtaining the output results of the risk analysis unit Data reception: Establish a data interface with the risk analysis unit to ensure that detailed information about defects output by the risk analysis unit can be obtained in real time and accurately, including but not limited to the basic description of the defect and risk score.

[0092] Data validation: Validate the format and content of received data, checking its completeness and accuracy. For example, check if the risk score is within a reasonable range and if the defect identifier is unique. If data issues are found, promptly communicate with the risk analysis unit and make corrections.

[0093] (ii) Defect sorting using a dynamic sorting algorithm Initial sorting based on risk score: All defects are sorted from highest to lowest risk score. This can be achieved using sorting functions in programming languages ​​(such as Python), for example, using the sorted() function combined with custom sorting rules.

[0094] Handling defects with the same risk score: For defects with the same risk score, sort them according to the time they were discovered, with the earlier discovered defects listed first. The discovery time can be recorded in the defect information and used as a secondary sorting criterion.

[0095] Dynamic sorting update: When a new defect is discovered, its information is added to the existing defect list, and its risk score is recalculated. The new defect is then inserted into the appropriate position, updating the overall defect priority order.

[0096] Changes in existing defect risk score: When the risk score of an existing defect changes, the defect is removed from its original ranking position, its ranking position is recalculated, and it is inserted into a new appropriate position to ensure the accuracy of the ranking.

[0097] (iii) Handling highly correlated defect groups Defect group identification: By analyzing factors such as defect characteristics, scope of impact, and frequency of occurrence, highly correlated defect groups can be identified. Machine learning algorithms (such as clustering algorithms) can be used to assist in identification, grouping defects with similar characteristics together.

[0098] Overall Priority Allocation: Identified defect groups are treated as a whole and prioritized accordingly. Their position in the priority order is determined based on the group's overall risk score (which can be a weighted average of the risk scores of all defects within the group).

[0099] (iv) Consider the availability of processing resources Resource assessment: Real-time monitoring and processing of resource availability, including human and material resources. Information such as resource usage and remaining quantities can be obtained through the resource management system.

[0100] Adjust priority allocation strategy When resources are sufficient: Determine the priority order of defects according to normal sorting rules.

[0101] When resources are limited: Adjust the priority allocation strategy based on the remaining resources and the urgency of the defects. For example, prioritize defects that have a significant impact on the system and have low processing costs, or temporarily postpone some low-priority defects.

[0102] (v) Output priority list List Format Definition: Define the format of the priority list. Each entry includes a defect identifier, risk score, and processing priority level. The list can be stored and represented using tables or JSON format for easy retrieval and processing by subsequent modules. The priority list should be updated promptly when the priority order changes to ensure the information is always up-to-date.

[0103] Design a data transmission interface with subsequent modules to ensure the priority list is transmitted to them in real time and accurately. Network communication protocols (such as HTTP, TCP / IP, etc.) can be used for data transmission. During data transmission, an error handling mechanism should be implemented to promptly retry or prompt relevant personnel for handling when transmission errors occur, ensuring data integrity and reliability.

[0104] Throughout the implementation process, data flows smoothly between processing units, and the processing algorithms adaptively adjust parameters based on the characteristics of the input data. All processing steps have corresponding exception handling mechanisms; when input data is abnormal or errors occur during processing, the system can automatically take recovery measures to ensure processing continuity. Data processing and risk assessment results are labeled with quality indicators to show their reliability, providing a reference for subsequent modules. The system periodically performs self-checks and calibrations on the processing algorithms to maintain accuracy and consistency.

[0105] Example 3: See Figure 4This embodiment relates to the specific operation flow and method of the detection stage division module and the parameter rejection assessment module. The detection stage division module receives output data from the defect risk assessment module, including standardized defect characteristics and corresponding risk level information. The stage definition unit divides the detection process into stages according to preset time intervals. These time intervals are set based on the overall cycle characteristics of equipment operation, dividing the continuous detection process into several operation stages with clear time boundaries. The duration of each stage is dynamically adjusted according to the equipment operation characteristics and detection requirements, and the start and end points are precisely marked by the system clock. During the stage division process, the system maintains a stage status table, recording the start time, expected duration, and stage type identifier of each stage. When the clock time reaches the stage switching point, the system automatically performs a stage transition operation, updates the current stage status, and notifies the relevant modules.

[0106] The focus allocation unit assigns specific inspection focus areas to each inspection stage based on the risk level data provided by the defect risk assessment module. This unit first obtains the spatial coordinate mapping relationship of the equipment surface, transforming the abstract risk level into specific physical area coordinates. For each inspection stage, the focus allocation unit generates a list of inspection focuses, arranged in priority order of the coordinates of areas requiring focused attention and their corresponding risk values. The spatial proximity principle is considered during area allocation, merging geographically close high-risk areas into the same inspection focus to improve inspection efficiency. Each inspection focus has a weight coefficient, with the weight value proportional to the risk level, guiding the data acquisition module to prioritize resource allocation to high-weight areas.

[0107] The parameter rejection assessment module and the detection stage segmentation module work together, with their data acquisition unit obtaining material property data from the equipment monitoring system in real time. This data includes, but is not limited to, physical parameters such as thermal conductivity, density, elastic modulus, electrical conductivity, and coefficient of thermal expansion. The data acquisition unit connects to the equipment database via a dedicated data interface, periodically querying the latest material data and updating the local material property database. The acquired data undergoes validity verification, removing outliers and data outside the reasonable range to ensure data quality. Material property data is organized according to equipment regions, with each region corresponding to a complete set of material parameters. These parameters are dynamically updated as the equipment's operating status changes.

[0108] The repulsion calculation unit calculates the repulsion index based on the detection focus information provided by the detection stage segmentation module and the material property data obtained by the data acquisition unit. This unit first establishes a correlation model between detection parameters and material properties, considering the influence of different material properties on the detection parameter settings. For each detection focus area, the adaptability index of the detection parameters is calculated based on its material properties. A multi-factor weighted evaluation method is used in the calculation process to comprehensively consider the combined effects of various material properties.

[0109] The rejection index is calculated using the following formula:

[0110] in: This represents the rejection index, with values ​​ranging from 0 to 1. The higher the value, the greater the degree of rejection. This represents the total number of detection parameters considered. It is the first The weighting coefficient of each detection parameter reflects the sensitivity of that parameter to material properties; Indicates the first The actual values ​​of each detection parameter under the current material properties; Indicates the first The optimal values ​​for each detection parameter under the current material properties; and They represent the first The maximum and minimum values ​​that can be set for each detection parameter. Weighting coefficients. Based on statistical analysis of historical testing data, this reflects the degree of influence of different testing parameters on the final testing results. Optimal values ​​are determined. The model was calculated based on a correlation between material properties and testing parameters, and this model was established based on a large amount of experimental data. Actual values... Parameter settings from the current system. Maximum and minimum values. , It is determined by the physical limitations of the testing equipment.

[0111] During the calculation, the system first acquires the material property data of the current detection focus area, and then queries the corresponding parameter optimization model based on the material type. For each detection parameter, the normalized difference between its current setting and the optimal value is calculated. The weighted difference values ​​of all parameters are summed to obtain the final repulsion index. The larger the value of this index, the greater the mismatch between the current detection parameter settings and the material properties, and the greater the parameter adjustments required.

[0112] The repulsion calculation unit performs calculations periodically, triggering a recalculation each time a detection stage changes or material properties change. The calculation results are updated in real time, reflecting the latest equipment status and parameter settings. The calculated repulsion index is stored along with the corresponding detection focus area information for use by subsequent modules. The system also records historical repulsion data, forming a time-series record for analyzing repulsion change trends and the effectiveness of parameter adjustments.

[0113] Throughout the implementation process, the two modules maintain close data interaction and status synchronization. The detection phase division module sends a phase update notification to the parameter repulsion evaluation module each time a phase switches, containing new detection focus information. After completing the repulsion calculation, the parameter repulsion evaluation module returns the result to the detection phase division module for optimizing subsequent phase planning. Data exchange adopts a unified interface specification to ensure the accuracy and timeliness of information transmission. All calculation processes have corresponding exception handling mechanisms; when input data is abnormal or errors occur during calculation, the system can automatically take recovery measures to ensure the continuity of processing.

[0114] Example 4: This example details the operational flow of the adjustment amount determination module and the detection control module. These two modules work together to calculate and adjust the detection parameters based on the current system state, while simultaneously executing corresponding defect handling actions. The following example illustrates their operation: Assume the system is inspecting the packaging integrity of a large industrial piece of equipment. The equipment surface has multiple potential defect areas, and the material properties include different metal alloys and composite materials.

[0115] The adjustment calculation unit in the adjustment determination module receives rejection indexes from the parameter rejection evaluation module and risk level data from the defect risk assessment module. The rejection index quantifies the degree of mismatch between the current detection parameter settings and material properties, while the risk level reflects the severity of the identified defects. The adjustment calculation unit processes this input data using a weighted fusion algorithm to calculate the required adjustment value for each detection parameter. Taking X-ray detection parameters as an example, when the material in the detection focal area changes from aluminum alloy to stainless steel, the rejection index indicates that the current X-ray intensity setting needs adjustment; simultaneously, the risk level indicates the presence of a high-risk defect in this area. The adjustment calculation unit integrates this information to calculate new X-ray intensity values, exposure time parameters, and image contrast settings.

[0116] The magnitude allocation unit receives the adjustment values ​​output by the adjustment calculation unit and allocates them to the corresponding detection parameters. This unit maintains a parameter mapping table, converting abstract adjustment values ​​into control commands for specific devices. Continuing with X-ray detection as an example, the magnitude allocation unit converts the calculated intensity adjustment value into voltage and current setting commands for the X-ray generator, the exposure time adjustment value into timing control signals, and the contrast adjustment value into image processing parameters. These allocation processes take into account the physical limitations of the equipment, ensuring that parameter values ​​are within permissible ranges. The magnitude allocation unit also handles the dependencies between parameters; when one parameter is adjusted, its associated parameters are adjusted synchronously to maintain system consistency. Refer to Table 1 for the system's allocation of adjustment values ​​to the three detection parameters at a given moment.

[0117] Table 1: Data table for the allocation of detection parameter adjustment amounts.

[0118] Parameter type Current value Calculate adjustment amount Value after allocation Applicable Areas Effective date X-ray intensity (kV) 110 +15 125 Area A immediately Infrared sensitivity (°C) 0.5 -0.2 0.3 Area B Next scan cycle Visual exposure (ms) 80 +20 100 Area C immediately The parameter adjustment unit in the detection and control module receives parameter adjustment commands from the value allocation unit and executes the actual parameter modification operations. The parameter adjustment unit connects to each detection unit via the equipment control interface and sends parameter setting commands using a dedicated communication protocol. For X-ray detection units, the parameter adjustment unit sends voltage and current setting commands; for infrared temperature sensing units, it sends sensitivity adjustment commands; and for visual detection units, it sends exposure time modification commands. These adjustment operations have different execution characteristics: some need to take effect immediately, while others can be delayed until the next detection cycle. The parameter adjustment unit monitors the execution status of parameter modifications to ensure successful completion of the adjustment operation. If parameter adjustment fails, the unit will automatically retry or activate a backup adjustment scheme.

[0119] The action execution unit controls the defect processing actuators to perform actual operations based on the priority order provided by the defect risk assessment module. This unit receives a list of defect processing priorities and arranges processing actions according to priority. For high-priority defects, the action execution unit immediately triggers the processing mechanism; for low-priority defects, they can be scheduled for execution in subsequent processing cycles. Processing actions include various types: for detected packaging damage, a marking device may be activated to spray a mark at the defect location; for critical defects, a machine shutdown procedure may be triggered; for minor defects, the defect location may simply be recorded for later inspection. The action execution unit coordinates the work of multiple actuators to avoid conflicts between processing actions. For example, when a robotic arm is marking defects in a certain area, other processing operations should not be performed simultaneously in the same area.

[0120] The adjustment determination module sends the parameter adjustment results to the detection and control module, which then feeds back the execution status to the adjustment determination module. This closed-loop control mechanism allows the system to dynamically adjust parameter settings based on the execution results. The system also records detailed logs of all parameter adjustments and processing actions, including adjustment time, adjustment value, and execution results. These logs are used for subsequent analysis and system optimization.

[0121] When the system detects a new defect or a change in equipment status, the entire adjustment and execution process restarts. The system continuously monitors changes in rejection indicators and risk levels, promptly adjusting parameters and responding with appropriate actions. This dynamic adjustment mechanism enables the detection system to adapt to changing detection environments and maintain optimal detection performance.

[0122] Example 5: This example describes the overall process of the human-machine interaction module and the trimodal equipment defect comprehensive detection method. The human-machine interaction module, as the interface between the system and the user, performs the dual functions of information presentation and parameter configuration. This module interacts with the user through a graphical user interface. The interface design follows the operating habits of industrial control systems, with a clear layout and well-defined functional areas. The data display unit is responsible for real-time visualization of the system's operating status and detection results. This unit receives standardized defect characteristics from the comprehensive data processing module and risk level data from the defect risk assessment module, converting them into a graphical display. The main interface area displays a 3D model of the equipment. The model surface is rendered with color mapping based on real-time data; high-temperature areas are displayed as a red gradient, and defect locations are marked with flashing icons. The right-hand information panel displays detailed data in numerical and graphical form, including temperature change curves, defect size statistics, and risk level distribution maps. All data is refreshed every 200 milliseconds to maintain real-time display. The historical data query function allows users to view detection records for any past time period, supporting filtering by time range, defect type, or risk level. The data export function supports converting selected data into standard format files for subsequent analysis or archiving.

[0123] The parameter input unit provides an interface for users to modify system parameters. This unit features various input controls, including numerical input boxes, sliders, selection lists, and function buttons. Numerical input boxes are used to enter specific parameter values, such as temperature thresholds, exposure times, or sensitivity settings. Input values ​​are range-verified to ensure they comply with equipment limitations. Sliders provide an intuitive way to adjust parameters; users can drag the slider to change the parameter value, with the current value displayed in real time. Selection lists are used to select parameter configurations from predefined options, such as selecting a detection mode or material type. Function buttons are used to perform specific operations, such as saving configurations, restoring defaults, or starting a calibration program. All user input undergoes validity checks; invalid inputs are rejected and error messages are displayed. The parameter modification log records the time, user ID, and modification details of each parameter change, forming a complete operation log.

[0124] The user access control feature differentiates between different operation permission levels. System administrators have full parameter modification permissions, engineers can modify technical parameters, and operators can only view data and perform routine operations. User login requires authentication, and the system displays the corresponding operation interface and function options based on the user's role. The operation guidance feature provides contextual help information; when a user selects a parameter, a detailed explanation and setting suggestions for that parameter are displayed to reduce the probability of operational errors.

[0125] The trimodal integrated equipment defect detection method integrates all functional modules of the system to form a complete detection process. When the method starts, the multimodal data acquisition module simultaneously activates the infrared temperature sensing unit, the X-ray defect identification unit, and the visual inspection unit, and the sensor network deployed according to a dot-matrix distribution structure begins to collect data. The collected raw data is transmitted to the integrated data processing module via a high-speed data bus. The data conversion unit converts the data from different modalities into a unified time-series format, and the feature extraction unit extracts standardized defect features from the time-series data. These features are then passed to the defect risk assessment module, where the risk analysis unit analyzes the interactions and potential impacts between defects, and the priority determination unit determines the processing order based on the analysis results.

[0126] The detection phase division module divides the detection process into multiple phases based on a preset time interval. The phase definition unit manages the phase switching sequence, and the focus allocation unit assigns a detection focus area to each phase. The parameter rejection assessment module's data acquisition unit acquires real-time equipment material characteristic data, and the rejection calculation unit calculates the rejection index based on the detection focus and material data. The adjustment amount determination module's adjustment calculation unit calculates parameter adjustment values ​​based on rejection and risk level, and the value allocation unit allocates the adjustment values ​​to specific detection parameters. The detection control module's parameter adjustment unit executes parameter modification operations, and the action execution unit controls the actuators to handle defects according to priority.

[0127] The human-computer interaction module continuously provides user interaction support. The data display unit updates system status and detection results in real time, while the parameter input unit receives user commands. When a user modifies parameters, the system immediately responds and adjusts the operating parameters of the relevant modules. The method execution process forms a closed-loop control, continuously optimizing system performance based on detection results and user input. All operations and data changes are recorded in the system log, including automatically executed operations and records of manual user intervention. The log file is stored in chronological order and supports retrieval and viewing based on various criteria.

[0128] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A comprehensive equipment defect detection system based on three modes, characterized in that, The application relates to a multi-modal data acquisition module for real-time acquisition of infrared temperature data, X-ray image data and visual image data of equipment; a comprehensive data processing module for processing data acquired by the multi-modal data acquisition module to generate standardized defect features; a defect risk assessment module for assessing the comprehensive risk level of defects based on the standardized defect features generated by the comprehensive data processing module; a detection stage division module for dividing a detection process into multiple stages and determining the detection focus of each stage based on the assessment results of the defect risk assessment module; a parameter exclusion degree assessment module for acquiring equipment material characteristic data and assessing the exclusion degree of detection parameters based on the detection focus of the detection stage division module; an adjustment amount determination module for determining the adjustment amount of detection parameters based on the assessment exclusion degree of the parameter exclusion degree assessment module and the risk level of the defect risk assessment module; and a detection regulation module for regulating detection parameters and performing defect processing actions according to the adjustment amount of the adjustment amount determination module. The multi-modal data acquisition module comprises an infrared temperature sensing unit, an X-ray defect identification unit and a visual detection unit. The infrared temperature sensing unit is used for monitoring temperature abnormalities of equipment. The X-ray defect identification unit is used for identifying packaging damage defects. The visual detection unit is used for detecting packaging rupture and pinhole defects. The infrared temperature sensing unit, the X-ray defect identification unit and the visual detection unit comprise a dot-matrix distribution structure; the dot-matrix distribution structure sets sensor positions based on equipment area analysis to realize uniform data acquisition. The comprehensive data processing module comprises a data conversion unit and a feature extraction unit. The data conversion unit is used for converting original data acquired by the multi-modal data acquisition module into a time series analysis format.

2. The three-modality based integrated inspection system for device defects according to claim 1, wherein, The feature extraction unit is used for extracting defect features from the time series analysis format converted by the data conversion unit. The defect risk assessment module comprises a risk analysis unit and a priority determination unit. The risk analysis unit is used for analyzing the interaction and potential influence between defects. The priority determination unit is used for determining the priority order of defect processing based on the analysis results of the risk analysis unit. The detection stage division module comprises a stage definition unit and a focus allocation unit.

3. The three-modality based integrated inspection system for defects of devices according to claim 1, wherein, The stage definition unit is used for dividing detection stages according to preset time intervals. The focus allocation unit is used for allocating a detection focus area to each stage based on the risk level of the defect risk assessment module. The parameter exclusion degree assessment module comprises a data acquisition unit and an exclusion degree calculation unit.

4. The three-modality based integrated inspection system for device defects according to claim 1, wherein, The data acquisition unit is used for acquiring equipment material characteristic data in real time. The exclusion degree calculation unit is used for calculating the exclusion degree index of detection parameters based on the detection focus of the detection stage division module and the material characteristic data acquired by the data acquisition unit. The adjustment amount determination module comprises an adjustment calculation unit and a value allocation unit.

5. The three-modality based integrated inspection system for device defects according to claim 1, wherein, The adjustment calculation unit is used for calculating an adjustment value based on the exclusion degree index of the parameter exclusion degree assessment module and the risk level of the defect risk assessment module. The adjustment calculation unit is used for calculating an adjustment value based on the exclusion degree index of the parameter exclusion degree assessment module and the risk level of the defect risk assessment module. ​ 6. The three-modality based integrated inspection system for device defects according to claim 1, wherein, ​ ​ ​ 7. The three-modality based integrated inspection system for device defects according to claim 1, wherein, ​ ​ The value distribution unit is configured to distribute the adjustment value calculated by the adjustment calculation unit to the corresponding detection parameter.

8. The three-modality based integrated inspection system for device defects according to claim 1, wherein, The detection regulation module comprises a parameter adjustment unit and an action execution unit. The parameter adjustment unit is configured to modify the detection parameter setting according to the adjustment amount determined by the adjustment amount determination module. The action execution unit is configured to control the defect processing executor based on the priority order of the defect risk assessment module.

9. The three-modality based integrated inspection system for device defects according to claim 1, wherein, Further comprising: A human-computer interaction module configured to display and record system data and allow a user to change system parameters; The human-computer interaction module comprises a data display unit and a parameter input unit; The data display unit is configured to visually display the standardized defect features of the comprehensive data processing module and the risk level of the defect risk assessment module in real time; and the parameter input unit is configured to receive the parameter change instruction input by the user.

10. A three-modality based comprehensive inspection method for equipment defects, characterized in that, All modules and method processes of the device defect comprehensive detection system based on the three modalities according to any one of claims 1 to 9 are included.

Images