Intelligent inspection method and system based on AI multi-source fusion

Abstract

The invention relates to the technical field of intelligent inspection, and discloses an intelligent inspection method and system based on AI multi-source fusion, and the method comprises the steps: collecting multi-mode perception data when an expert carries out the touch inspection operation; analyzing the multi-modal perception data, and constructing a biomechanical model and a tactile knowledge mapping model; establishing a standard tactile operation model and an acoustic feature mapping relation database; converting the standard tactile operation model into animation guidance and tactile feedback in an augmented reality interface; actual operation data of the inspectors are collected and compared with the standard touch operation model, and operation improvement suggestions are generated; fusing the tactile inspection result, the acoustic detection result and the visual inspection result to generate a comprehensive evaluation report; according to the invention, digital inheritance and standardized application of the tactile skills are realized, and the technical problem that the tactile skills are difficult to transmit in traditional inspection is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection, and more specifically, it relates to an intelligent inspection method and system based on AI multi-source fusion. Background Art

[0002] During the inspection process of high-precision equipment such as airplanes, inspectors need to simultaneously pay attention to the status of equipment components and refer to technical documents, and detect the status of components through tactile operations such as "knocking, touching, and dialing". These tactile inspection operations highly rely on the personal experience of inspectors and are crucial for detecting certain hidden defects (such as internal delamination and structural looseness).

[0003] However, the existing technologies have the following problems: Tactile inspection skills are highly personalized and implicit, and are difficult to be effectively transmitted through traditional documents or oral means; there is a lack of technical means to digitalize and parameterize the tactile experience of experts; existing AR-assisted systems ignore the importance of tactile inspection during patrol and lack an assessment of the correctness of inspectors' tactile operations; different materials and component structures require different tactile inspection skills, increasing the complexity of skill expression and learning; inspectors need to frequently refer to work cards and technical manuals, resulting in interrupted workflows and distracted attention.

[0004] The above problems lead to difficult-to-guarantee inspection quality, poor consistency of detection results, and serious loss of experience after experts retire, restricting the progress of equipment detection technology. Summary of the Invention

[0005] The present invention provides an intelligent inspection method and system based on AI multi-source fusion to solve the technical problems in related technologies such as the difficulty in transmitting tactile inspection skills, lack of objective assessment, and loss of experience.

[0006] The present invention provides an intelligent inspection method based on AI multi-source fusion, including the following steps: Collect multi-modal perception data when an expert performs a tactile inspection operation, where the multi-modal perception data includes tactile parameters, acoustic signals, and action trajectories; Analyze the multi-modal perception data and construct a biomechanical model and a tactile knowledge mapping model; Based on the biomechanical model and the tactile knowledge mapping model, establish a standard tactile operation model and an acoustic feature mapping relationship database; Convert the standard tactile operation model into an animation guidance and tactile feedback in an augmented reality interface; Collect the actual operation data of the inspector and compare it with the standard tactile operation model to generate operation improvement suggestions; Based on the operation improvement suggestions and the actual operation data of the inspector, fuse the tactile inspection results, acoustic detection results, and visual inspection results to generate a comprehensive assessment report.

[0007] In a preferred embodiment, the steps of collecting multimodal perception data during the tactile inspection operation by the expert include: Collecting the contact force curve, motion trajectory, and vibration response data during the expert's tactile inspection through an intelligent glove equipped with a pressure sensor, an acceleration sensor, and a position sensor; Collecting knocking sound waves and structural vibration signals through a directional microphone array; Recording the overall operation posture and motion trajectory data through a motion capture system.

[0008] In a preferred embodiment, the steps of analyzing the multimodal perception data and constructing a biomechanical model and a tactile knowledge mapping model include: Applying a biomechanical analysis algorithm to analyze the biomechanical characteristics of the expert's tactile actions and constructing a biomechanical mathematical model including joint angle changes, muscle torque distribution, and energy efficiency; Applying a feature extraction algorithm to analyze the differences in the expert's tactile feedback for components in different states and establishing a state feedback mapping model; Performing statistical analysis on the operation data of multiple experts, extracting the kinematic characteristics of best practices, and forming a standardized tactile inspection parameter library.

[0009] In a preferred embodiment, the steps of establishing a standard tactile operation model and an acoustic feature mapping relationship database based on the biomechanical model and the tactile knowledge mapping model include: Applying a multivariate time series analysis algorithm to analyze the expert's tactile inspection actions, extracting key parameters, and establishing a standard operation model; Using an acoustic signal processing algorithm to analyze the knocking sound wave data, extracting spectral features, time-domain features, and time-frequency features, and constructing a mapping relationship database of defect type acoustic features; Applying a classification algorithm to construct an acoustic feature discrimination model to achieve an objective judgment of knocking detection.

[0010] In a preferred embodiment, the steps of converting the standard tactile operation model into an animation guidance and tactile feedback in an augmented reality interface include: Applying a three-dimensional visualization algorithm to convert the skill features into an animation guidance and quantitative parameters in the augmented reality interface; Constructing a context-aware system based on component recognition, retrieving the corresponding operation guidelines from the standard operation model library according to the recognized components and the current task requirements; Constructing a tactile feedback system to provide tactile feedback through the micro vibration elements in the intelligent glove to guide correct operations.

[0011] In a preferred embodiment, the steps of collecting the actual operation data of the inspector and comparing it with the standard tactile operation model to generate operation improvement suggestions include: Adopt a real-time sensing data analysis algorithm, and use the inertial sensor and pressure sensor integrated in the intelligent glove to collect the actual operation data of the inspector; Based on the similarity calculation algorithm, compare the collected actual operation data with the standard operation model and calculate the operation similarity; Apply a personalized learning model to generate customized skill adjustment suggestions based on the biomechanical characteristics and skill mastery of the inspector.

[0012] In a preferred embodiment, the steps of fusing the tactile inspection results, acoustic detection results and visual inspection results to generate a comprehensive evaluation report include: Apply a multi-modal data fusion algorithm to fuse the tactile operation results with the visual inspection results and acoustic detection results; Construct a tactile knowledge base system to collect and analyze the tactile operation data of different inspectors; Apply an incremental learning algorithm to continuously update and optimize the standard operation model and decision rules based on the continuously accumulated operation data and feedback.

[0013] In a preferred embodiment, the multi-modal data fusion algorithm includes feature-level fusion and decision-level fusion. Among them, feature-level fusion normalizes the feature vectors of different modalities and then connects them to form a joint feature representation, and decision-level fusion calculates the credibility and uncertainty of the detection results of each modality based on the Dempster-Shafer evidence theory.

[0014] In a preferred embodiment, the tactile knowledge base system includes: A data collection module for continuously collecting and storing the tactile operation data of different inspectors; An analysis and optimization module for analyzing the operation data using statistical analysis algorithms and refining best practices; A knowledge representation module for converting the refined best practices into a structured knowledge representation form; A knowledge application module for using the experience in the knowledge base for training new personnel and assisting in decision-making.

[0015] In a preferred embodiment, an intelligent inspection system based on AI multi-source fusion is used to execute an intelligent inspection method based on AI multi-source fusion, including: A multi-modal perception data collection module for collecting multi-modal perception data when an expert performs a tactile inspection operation. The multi-modal perception data includes tactile parameters, acoustic signals and action trajectories; The data analysis and processing module is used to analyze multi-modal perception data and construct a biomechanical model and a tactile knowledge mapping model; The model establishment module, based on the biomechanical model and the tactile knowledge mapping model, establishes a standard tactile operation model and an acoustic feature mapping relationship database; The interaction feedback module is used to convert the standard tactile operation model into an animation guide and tactile feedback in the augmented reality interface; The operation evaluation module is used to collect the actual operation data of the inspector and compare it with the standard tactile operation model to generate operation improvement suggestions; The result fusion module is used to fuse the tactile inspection results, acoustic detection results and visual inspection results based on the operation improvement suggestions and the actual operation data of the inspector to generate a comprehensive evaluation report.

[0016] The beneficial effects of the present invention are as follows: It converts the previous implicit and difficult-to-transmit tactile inspection skills into explicit and digitizable parameter models, realizes the objective quantification and standardization of tactile experience, improves the inspection consistency and the detection accuracy rate; Through the biomechanical model and tactile knowledge extraction, an accurate mapping relationship between the component state and tactile feedback is established, turning subjective operations such as knocking detection into objective evaluations based on data; With the help of standard tactile operation modeling and AR technology assistance, "hand-to" guidance is realized, shortening the time for newbies to master skills and significantly reducing the skill learning threshold; The objective judgment system based on acoustic signal processing technology reduces the influence of environmental noise on knocking detection, improves the signal-to-noise ratio in the standard airport noise environment, and significantly improves the detection reliability; By establishing a tactile knowledge base system, the precipitation and inheritance of organizational-level tactile skill knowledge are realized, the problem of experience loss after experts retire is solved, and a sustainable skill inheritance solution is provided for enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of an intelligent inspection method based on AI multi-source fusion of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Now, the subject matter described herein will be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the protection scope of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute or add various processes or components as needed. Additionally, the features described in some examples can also be combined in other examples.

[0019] In at least one embodiment of the present invention, an intelligent inspection method based on AI multi-source fusion is disclosed. As Figure 1 shown, it includes the following steps: Step 1: Collect multi-modal perception data when an expert performs a tactile inspection operation. The multi-modal perception data includes tactile parameters, acoustic signals, and action trajectories; Specifically, it includes the following steps: Step 1.1: An intelligent glove equipped with pressure sensors, acceleration sensors, and position sensors is used to capture detailed parameters of the inspector's hand movements; Step 1.2: A directional microphone array is used to collect knocking sound waves and structural vibration signals; Step 1.3: A motion capture system is used to record the overall operation postures and action trajectories.

[0020] The sensor configuration of the intelligent glove can be adjusted according to different inspection scenarios: In scenarios that require precise force control (such as knocking detection), a high-density sensing network composed of 16 high-precision piezoresistive pressure sensors and 8 low-latency acceleration sensors can be adopted; In scenarios that require tracking complex movements (such as touch detection), a hybrid sensing network composed of flexible fiber optic sensors and elastic resistive strain sensors can be adopted to reduce the impact of sensor rigidity on the operation; In harsh environments (such as high-temperature and humid environments), a waterproof and oil-proof packaging structure and anti-interference sensor types can be adopted, such as sealed piezoelectric sensors and temperature-compensated acceleration sensors.

[0021] In some embodiments, the directional microphone array can adopt the following optional configurations: High-density circular array: A circular array composed of 16 omnidirectional microphones, which applies beamforming algorithms to achieve noise suppression and signal enhancement; Distributed array: Arrange 6 to 8 microphones around the inspection area, and apply acoustic positioning algorithms to determine the knocking position, which is suitable for the inspection of large components; Integrated contact microphone: Directly integrate the acoustic sensor on the intelligent glove, and collect knocking sound waves through bone conduction to reduce the impact of environmental noise.

[0022] According to another embodiment of the present application, the system can also selectively integrate the following sensing modules: Thermal imaging sensor: Used to capture the temperature changes of components during the inspection process to assist in identifying the thermal response characteristics during the friction process; Near-infrared spectroscopy sensor: Used to detect the surface and shallow structure characteristics of materials to provide material information complementary to tactile inspection; Visual tracking module: Consisting of a high-frame-rate camera and a depth sensor, it provides visual recording of the operation process and a three-dimensional spatial reference.

[0023] The complete motion parameters of an expert performing a tactile examination are collected through this system to form a multi-modal perception data set containing the following: Contact force curve , where represents the magnitude of the acting force, represents the time variable; Motion trajectory , where represents the position coordinates in three-dimensional space; Vibration response , where represents the amplitude, represents the frequency, represents the time variable; Acoustic characteristics , where represents the intensity of the sound signal, represents the frequency, represents the time variable.

[0024] Step 2: Analyze the multi-modal perception data and construct a biomechanical model and a tactile knowledge mapping model; Specifically, it includes the following steps: Step 2.1: Based on the multi-modal perception data, apply biomechanical analysis algorithms to analyze the biomechanical characteristics of the expert's tactile actions, including parameters such as joint angle changes, muscle torque distribution, and energy efficiency, and construct a biomechanical mathematical model containing these characteristics : ; Among them, represents the biomechanical mathematical model; represents the function of joint angle change over time, describing the angle changes of the examiner's wrist, elbow, knuckles, etc. during the tactile examination; represents the function of muscle torque change over time, reflecting the magnitude and distribution of the muscle torque generated by the examiner during operations such as tapping and rubbing; represents the function of energy efficiency change over time, quantifying the ratio between the energy consumption of the examiner's actions and the effective detection force, and reflecting the economy and sustainability of the operation. These three functions together constitute the biomechanical mathematical model describing the expert's tactile examination skills.

[0025] ; Among them, represents the th joint at time The angular value reflects the degree of joint bending or rotation; Indicates the importance weight of the joint in the tactile examination. The larger the value, the more significant the impact of the joint on the tactile examination result; Represents the total number of joints monitored, that is, the number of all hand joints participating in the tactile examination action; Represents the function of the joint angle changing with time. It is a weighted index that comprehensively considers the angle changes of all joints and reflects the overall hand posture. By combining the angle changes of multiple joints through weighting, the dynamic characteristics of the examiner's hand posture can be comprehensively characterized, providing a quantitative basis for the digitization and standardization of tactile examination skills.

[0026] ; Among them, Represents the function of the muscle torque changing with time, reflecting the magnitude of the torque generated by the examiner during the tactile examination; Represents at time the force generated by the muscle, describing the magnitude of the acting force exerted by the examiner; Represents the length of the force arm, which refers to the perpendicular distance from the acting point of the force to the axis of rotation; Is the integration variable, representing the time range from the start of the examination (0) to the current moment (t); Represents the cross - product operation, calculating the torque vector generated by the force and the force arm; the integral symbol Represents the cumulative calculation of the torque from the start to the moment of the entire examination process. By integrating the torque over the entire examination process, the precise control ability of the force exerted by the examiner can be quantified, reflecting the force control skills and stability of the expert in the tactile examination.

[0027] ; Among them, Represents the function of the energy efficiency changing with time, reflecting the energy utilization efficiency of the examiner's actions; Represents at time the effective detection force generated, referring to the magnitude of the effective force actually used for detection; Represents the th muscle group's energy consumption at time , quantifying the biological energy consumed by muscle activity; Represents the summation operation of the energy consumption of all muscle groups participating in the action; Represents the total number of muscle groups participating in the action, referring to the number of all muscle groups activated during the tactile examination; Is the index variable of the muscle group, with a value range from 1 to The higher this ratio is, the higher the operation efficiency of the inspector, the lower the fatigue level, and the better the sustainability, reflecting the energy optimization ability of experts in tactile inspection.

[0028] These three functions together constitute a biomechanical mathematical model for describing experts' tactile inspection skills. By precisely quantifying these parameters, experts' implicit tactile skills can be transformed into teachable explicit knowledge.

[0029] The implementation of the above biomechanical analysis algorithm is as follows: Apply principal component analysis to the collected time-series data of hand joint angles to extract key motion features and reduce the dimension of action representation; Apply the inverse dynamics algorithm to calculate the torque distribution of each joint during action execution. This algorithm is based on Newton-Euler equations and considers joint angular acceleration and external loads; Adopt an energy consumption estimation model to calculate the energy efficiency of different operation modes, which comprehensively considers factors such as muscle activity, joint friction, and gravitational potential energy; Integrate the above features through Bayesian optimization method to construct an optimal biomechanical model.

[0030] In the scenario of aircraft skin tapping detection, this algorithm can extract the unique wrist-forearm coordination pattern during experts' tapping, identify the best tapping angle (relative to the aircraft surface) and the ideal tapping force range, effectively solving the problem that tapping skills are difficult to describe verbally.

[0031] The biomechanical analysis algorithm can also adopt the following implementation path: In some embodiments, a deep neural network can be used to replace the traditional dynamics model, and directly predict joint torque and muscle activity from sensing data through end-to-end learning. This method can reduce the dependence on precise human parameters and improve the generalization ability of the model among inspectors with different body types; In other embodiments, a multi-scale analysis method can be adopted to model macroscopic actions (such as overall arm movement) and microscopic actions (such as fingertip fine actions) separately, and then integrate them through a hierarchical structure. This method is more suitable for analyzing complex multi-stage inspection operations.

[0032] Step 2.2: Apply a feature extraction algorithm to analyze the differences in tactile feedback of experts for components in different states, and establish a state feedback mapping model : ; Among them, is the component state vector, which includes different state categories such as normal and abnormal, representing the health status of the detected component; represents the mapping from the tactile feedback feature vector to the component state vector The mapping function; is the tactile feedback feature vector, including multi-dimensional features such as force, vibration, sound, etc., represents the th feature component; is the weight coefficient of each feature, indicating the importance of the th feature in the state judgment; is the feature dimension, that is, the length of the feature vector, representing the total number of features used for state judgment. This formula means that the multi-dimensional tactile feedback features are mapped to the component state representation by weighted summation.

[0033] The specific implementation of the above feature extraction algorithm is as follows: Apply time-frequency analysis methods to the collected tactile feedback data, including short-time Fourier transform and wavelet transform, to extract the frequency domain features of the vibration signal; Adopt statistical feature extraction methods to calculate the time domain features such as the peak value, peak-to-peak distance, rise time, and fall time of the force curve; Combine acoustic signals and mechanical signals, and apply cross-correlation analysis to extract cross-modal features; Determine the weight coefficients of each feature through a feature importance evaluation method based on multi-class support vector machines .

[0034] In the scenario of aircraft composite component delamination detection, this algorithm can identify the subtle differences between normal components and delaminated components in the tapping feedback.

[0035] According to another embodiment of the present application, the feature extraction algorithm can adopt the following optional implementation methods: Optionally, an autoencoder neural network can be used for unsupervised feature learning. This method can automatically discover the potential structure in the data and is particularly suitable for processing historical inspection data lacking clear labels; In an environment with high noise, a feature extraction method based on the attention mechanism can be adopted. By learning to focus on the key time-frequency regions in the signal, the ability to extract weak defect features can be improved; For multi-modal data, a cross-modal deep representation learning method can be selected to map tactile signals and acoustic signals to a shared feature space, enhance the complementarity between different modalities, and improve the feature expression ability.

[0036] In terms of the construction of the state feedback mapping model: A mapping model based on fuzzy logic can be adopted. This model represents expert experience through fuzzy rules and can better handle the uncertain relationship between feedback features and state judgment; A probabilistic graphical model (such as a Bayesian network) can be adopted to explicitly model the dependence relationship and causal structure between features and improve the interpretability of the model; In the case of complex detection tasks, an ensemble learning method can be adopted to combine the prediction results of multiple basic models to improve the accuracy and robustness of state judgment.

[0037] Step 2.3, by statistically analyzing the operation data of multiple experts, this embodiment extracts the kinematic characteristics of "best practices" to form a standardized haptic inspection parameter library, which contains the optimal operation parameters for different components and different inspection types.

[0038] Step 3, based on the biomechanical model and the haptic knowledge mapping model, establish a standard haptic operation model and an acoustic feature mapping relationship database; Specifically, it includes the following steps: Step 3.1, based on the biomechanical model and haptic knowledge, apply the multivariate time series analysis algorithm to analyze the haptic inspection actions of experts, extract key parameters, and establish a standard operation model : ; Among them, represents the standard operation model; is the contact position, indicating the exact spatial coordinates where the inspector's finger or tool contacts the component to be inspected; is the force, indicating the magnitude of the pressure exerted by the inspector on the component to be inspected; is the direction, indicating the movement direction of the haptic operation or the direction of the force action, which can be represented by a three-dimensional vector; is the duration, indicating the time length from the start to the end of the haptic operation; represents the operation sequence, that is, a series of ordered combinations of basic haptic actions, such as standardized operation processes like "tap - press - slide"; represents the position parameter range, defining the spatial boundary and the optimal contact area of effective haptic inspection; represents the force parameter range, specifying the minimum effective force and the maximum safe force required for different inspection types; represents the time parameter range, defining the shortest effective duration and the longest recommended time for various haptic operations.

[0039] The specific implementation method of the above-mentioned multivariate time series analysis algorithm is as follows: Apply the dynamic time warping algorithm to align the time of the same operation performed by different experts and different times, eliminating the influence of the difference in action execution speed; Adopt the hidden Markov model to analyze the action sequence, decompose the continuous operation into a discrete state sequence, and identify the key stages of the operation; Apply the Gaussian mixture model to each key stage to establish the probability distribution of operation parameters and determine the reasonable range of parameters. Analyze the parameter selection strategy under different operating conditions through the decision tree algorithm to form a conditional decision model for different components and detection targets.

[0040] In the inspection scenario of aircraft engine pylons, the algorithm successfully models the expert's loosening detection operation into three key stages: initial positioning, applying a test force, and vibration evaluation.

[0041] Step 3.2: Use the acoustic signal processing algorithm to analyze the knocking sound wave data, extract spectral features, time-domain features, and time-frequency features, and construct a mapping relationship database of acoustic features of defect types. : ; Among them, represents the mapping relationship database of acoustic features of defect types; represents the th type of defect, such as different structural anomaly types like delamination, crack, looseness, etc.; represents the acoustic feature vector corresponding to this defect type, including multi-dimensional parameters such as frequency-domain features (such as main frequency, spectral distribution), time-domain features (such as amplitude, decay time), and time-frequency features (such as energy distribution); represents the total number of defect types, that is, the number of different types of defects that the system can identify; is the index of the defect type, an integer from 1 to . This database establishes the mapping relationship between defect types and their unique acoustic features, providing a data basis for subsequent automatic defect identification.

[0042] The specific implementation method of the above acoustic signal processing algorithm is as follows: Apply the adaptive filtering algorithm to eliminate background noise from the collected acoustic signal and improve the signal-to-noise ratio; Use the fast Fourier transform to calculate the signal power spectral density and extract frequency-domain features; Apply the envelope analysis method to extract the attenuation characteristics of the sound and calculate the attenuation time constant of different materials; apply the Mel-Frequency Cepstral Coefficients (MFCC) analysis method to extract the spectral shape features of the sound; Conduct time-frequency joint analysis through two-dimensional wavelet transform to capture transient features and the variation law of frequency with time.

[0043] In the inspection scenario of aircraft honeycomb structures, in this embodiment, by analyzing the percussion acoustic characteristics of honeycomb structures in different states, an acoustic feature database including four states of normal, debonded, water - ingress, and crushed is established. For example, experiments show that the main resonance frequency of a normal honeycomb structure is usually in the range of 1100 to 1300 Hz, while the main resonance frequency of a debonded structure drops to 800 to 1000 Hz, and the water - ingress structure shows obvious high - frequency attenuation characteristics. These quantitative acoustic features provide a basis for objective judgment.

[0044] Step 3.3, apply a classification algorithm to construct an acoustic feature discrimination model. This model can map the newly collected acoustic features to the corresponding defect types based on the constructed mapping relationship database, so as to achieve objective judgment of percussion detection. Therefore, this embodiment effectively solves the technical problem of relying on the subjective judgment of inspectors.

[0045] Among them, the specific implementation method of the acoustic feature discrimination model is as follows: Apply principal component analysis to the extracted acoustic feature vectors for dimensionality reduction, retaining more than 90% of the information volume; Adopt an ensemble learning model composed of a support vector machine and a random forest for defect classification and discrimination; Analyze the classification results through a confusion matrix, and calculate the recognition accuracy and false - alarm rate of various defects; Introduce a classification probability output mechanism. When the discrimination confidence is lower than the threshold, the system will prompt the inspector to conduct additional verification.

[0046] Step 4, convert the standard tactile operation model into an animation guide and tactile feedback in the augmented reality interface; Specifically, it includes the following steps: Step 4.1, apply a three - dimensional visualization algorithm to convert the extracted skill features into an animation guide and quantitative parameters in the AR interface.

[0047] This algorithm converts the abstract tactile parameter model into intuitive visual guidance elements: ; Among them, represents the set of visualization guidance elements, including visual elements such as animation guides, force indicators, operation trajectories, etc.; represents the conversion function, which is responsible for converting tactile parameters and biomechanical characteristics into visual representations; represents the standard operation model, including the contact position , force , direction and duration and other parameters; Represents a biomechanical model that includes biomechanical parameters such as hand joint angles, muscle activity patterns, and kinematic characteristics. This formula describes how to convert an abstract tactile operation model and biomechanical characteristics into intuitive and visible guiding elements in the AR environment.

[0048] The specific implementation of the above three-dimensional visualization algorithm is as follows: Apply skeletal animation technology to construct a 3D model of hand operations. This model contains 15 joint points and can accurately simulate various inspection actions; Based on the biomechanical model parameters, apply the forward kinematic algorithm to calculate the motion trajectories of the joints; Adopt real-time rendering technology to convert the motion trajectories into smooth AR animations, and at the same time add visual guiding elements such as direction and force; Apply color mapping technology to encode quantitative parameters such as force and speed into intuitive visual effects; Through transparency and layer control technology, ensure that the visualization elements do not block the actual vision of the inspectors.

[0049] In the inspection scenario of aircraft hydraulic pipeline joints, this algorithm converts the expert's "rotation - pulling" detection operation into an animated guidance in the AR interface. The system displays the ideal hand posture, rotation angle, and pulling force range, and indicates the detection key points through color changes (for example, a pulling force of 8 to 10N is the best range for detecting the connection quality of the joint). This kind of visual guidance enables novice inspectors to accurately imitate the tactile skills of experts and improves the inspection accuracy.

[0050] The three-dimensional visualization algorithm can adopt the following different implementation methods: In some implementation methods, a hierarchical visualization strategy can be adopted to dynamically adjust the complexity and detail level of the visualization content according to the skill level of the inspectors. For beginners, display the complete action guidance; for intermediate inspectors, only display the key parameters and operation points; for advanced inspectors, only provide warning information when the operation deviates from the standard; In some other implementation methods, a physics-based visual effect can be adopted to intuitively present the force and contact feedback through physical effects such as elastic deformation and fluid dynamics. For example, represent the knocking force by the flow speed of virtual liquid and represent the pressure distribution by surface deformation; Spatial audio technology can also be adopted to enhance the visual feedback, mapping different tactile parameters into sound cues with spatial positioning to assist inspectors in receiving parameter feedback when it is not convenient to view the display.

[0051] Step 4.2, construct a context-aware system based on component recognition, which integrates the following functional modules: Component recognition module: Use image recognition algorithms to identify the aircraft components being inspected currently; Task matching module: Retrieve the corresponding operation guide from the standard operation model library according to the identified components and the current task requirements; Visual rendering module: Convert the operation guide into an animated demonstration and parameter indication in the AR interface.

[0052] Among them, the specific implementation method of the context-aware system is as follows: The component recognition module adopts a pre-trained convolutional neural network model, which is trained based on an image dataset of 1500 common aircraft components; The task matching module adopts an ontology-based semantic matching algorithm to establish a mapping between the identified components and the inspection tasks and operation model library in the maintenance manual; The visual rendering module adopts spatial anchoring technology to accurately register AR elements to the corresponding positions in the real world, and the positioning accuracy is better than 10mm.

[0053] In actual applications, when the inspector wears AR glasses to observe the aircraft flap hinge, the system can automatically identify the component type, and according to the inspection requirements in the maintenance manual, superimpose the corresponding tactile inspection operation guide in the AR interface. At the same time, the system will also display the service history and the last inspection result of the hinge, providing complete context information.

[0054] According to another embodiment of the present application, the context-aware system can adopt the following optional interaction modes: Optionally adopt a multimodal interaction method, allowing the inspector to control the AR interface through voice commands, gestures or eye tracking to achieve system control while keeping both hands in an operating state; In a collaborative work scenario, a shared AR space can be adopted to enable multiple inspectors to see the same AR guidance content at the same time, promoting teamwork and on-site training; In a complex inspection process, a progressive guidance mode can be adopted to automatically advance to the next step according to the currently completed inspection steps, reducing the cognitive burden of the inspector.

[0055] Step 4.3, construct a tactile feedback system, which combines AR visual guidance with tactile feedback; When the inspector's operation deviates from the standard operation, provide tactile feedback through the micro vibration elements in the intelligent glove to guide the correct operation; Present the processing results of the acoustical signals collected in real time in the AR interface in the form of spectrograms and numerical indicators intuitively to assist the inspector in making judgments.

[0056] The specific implementation method of the tactile feedback system is as follows: The intelligent glove adopts a distributed vibration feedback architecture, and integrates 20 micro piezoelectric vibration elements at the key positions of the palm and five fingers; Each vibration element can be independently controlled in terms of amplitude, frequency, and pulse pattern, forming a rich tactile coding system; The system adopts a closed-loop feedback control algorithm to dynamically adjust the feedback intensity according to the deviation between the actual operation of the inspector and the standard model; Implement the vibration mode mapping algorithm in parallel to convert different types of operation errors into different vibration modes. For example, excessive force is indicated by gradually increasing vibrations, and deviation in direction is indicated by directional vibration sequences.

[0057] In the scenario of aircraft landing gear inspection, when the inspector performs the shock absorber looseness detection, if the applied force is insufficient, the intelligent glove will generate gradually increasing vibrations at the index finger and thumb positions to indicate the need to increase pressure; if the inspection direction deviates, directional vibrations will be generated on the palm side to guide adjustment in the correct direction.

[0058] According to another embodiment of the present application, the tactile feedback system can adopt the following optional implementation methods: In some embodiments, electrostimulation tactile feedback technology can be used to replace vibration feedback. By precisely stimulating the nerves in different areas of the hand with low current, richer tactile sensations such as pressure, texture, and light touch can be generated, improving the information density of the feedback; A flexible mechanism driven by shape memory alloy (SMA) can be used to provide physical restraint at the joint positions of the glove to guide the hand posture of the inspector, which is particularly effective in inspection tasks that require precise angle control; In other embodiments, tactile feedback can be extended to the arm and trunk areas, and vibration elements distributed on the work clothes can be used to provide full-body posture guidance, which is suitable for inspection tasks that require overall body coordination.

[0059] In terms of tactile coding strategies: A psychophysics-based tactile coding model can be adopted to optimize the feedback mode according to the characteristics of human tactile perception. For example, multiple feedbacks staggered in time can be designed by utilizing the tactile masking effect and adaptation effect; The feedback intensity can be adjusted using a progressive learning curve, gradually weakening the feedback signal as the inspector's skills improve, to achieve a smooth transition from "strong guidance" to "light hint"; For inspectors with special needs, personalized tactile coding schemes can be provided. For example, the feedback intensity can be enhanced for inspectors with lower tactile sensitivity, and the feedback position can be adjusted for inspectors with uneven hand sensitivity distribution.

[0060] Therefore, through the above technical means, this embodiment realizes the visual expression and interactive guidance of tactile skills, effectively solving the problem of difficult transmission of tactile skills.

[0061] Step 5: Collect the actual operation data of the inspector and compare it with the standard tactile operation model to generate operation improvement suggestions; Specifically, it includes the following steps: Step 5.1: Adopt a real-time sensing data analysis algorithm, and use the inertial sensors and pressure sensors integrated in the intelligent glove to collect the actual operation data of the inspector, including parameters such as operation force, frequency, direction, and trajectory; The specific implementation method of the real-time sensing data analysis algorithm is as follows: Apply the Kalman filter algorithm to perform real-time preprocessing on the sensor raw data to eliminate noise and jitter; Adopt the sliding window Fourier transform to perform frequency domain analysis on the sensing data to extract the rhythm characteristics of the operation; Apply the trajectory reconstruction algorithm to fuse the data of multiple inertial sensors and accurately reconstruct the 3D motion trajectory of the hand; Adopt the feature extraction algorithm to extract key operation parameters from the continuous data stream, such as peak force, duration, and trajectory deviation.

[0062] In the aircraft control surface degree of freedom inspection scenario, this algorithm can accurately capture the force change process applied by the inspector on the control surface, record the motion damping characteristics of the control surface, and automatically extract key measurement data. The system sampling rate reaches 200Hz, ensuring that minute force changes and instantaneous response characteristics are captured.

[0063] Step 5.2: Based on the similarity calculation algorithm, compare the collected actual operation data with the standard operation model and calculate the operation similarity : ; Among them, represents the operation similarity; represents the user's actual operation parameter vector, including actual detection operation parameters such as force, frequency, and angle; represents the standard operation parameter vector, including the ideal operation parameters in the standard model; represents the similarity function of the th parameter, which is used to calculate the matching degree between the user's operation and the standard operation in this parameter; represents the weight coefficient of the th parameter, reflecting the importance of this parameter in the overall evaluation; represents the total number of parameters, that is, the number of operation feature dimensions considered. For different detection tasks, the weights of each parameter will be dynamically adjusted. For example, the force parameter has a higher weight in knock detection, while the trajectory parameter has a higher weight in friction detection.

[0064] The specific implementation method of the similarity calculation algorithm is as follows: Design dedicated similarity measurement functions for different types of parameters. For example, use the Gaussian similarity function for force parameters, the Fréchet distance for trajectory parameters, and the cosine similarity for frequency parameters. Apply an adaptive weighting mechanism to dynamically adjust the weights of each parameter according to different inspection types. For example, in knock detection, the weights of force and frequency are relatively high, while in friction detection, the weights of trajectory and pressure distribution are relatively high. Use the dynamic time warping algorithm to solve the problem of operation timing differences, ensuring that the same operations performed at different speeds can be correctly matched. Apply a multi-level evaluation strategy to calculate the similarity at three levels: overall operation, key stages, and detailed actions, forming a comprehensive evaluation.

[0065] In the scenario of aircraft fuel pipeline connection inspection, this algorithm can evaluate the matching degree between the tightening-undo inspection operation performed by the inspector and the standard model. The system not only evaluates the overall operation similarity but also points out specific deviations, such as "the tightening torque is less than 15%" or "the undo angle is too large by 30°", providing precise improvement guidance for the inspector.

[0066] Step 5.3, apply a personalized learning model to generate customized skill adjustment suggestions based on the inspector's biomechanical characteristics and skill mastery: Analyze the inspector's operation preferences and biomechanical limitations; Calculate the adjustable range of the standard operation model while maintaining the detection effect; Generate a skill adjustment plan that suits personal characteristics to improve the skill mastery efficiency and operation comfort.

[0067] Among them, the specific implementation method of the personalized learning model is as follows: Build a database of inspectors' biomechanical characteristics, including parameters such as hand size, strength range, and movement habits; Apply the reinforcement learning algorithm to gradually establish a personalized operation model through multiple operation attempts and feedback; Use the constraint optimization algorithm to calculate the adjustable range of operation parameters suitable for personal characteristics while meeting the detection quality requirements; Apply the decision tree algorithm to generate hierarchical skill adjustment suggestions for progressive guidance from simple adjustments to complex optimizations.

[0068] In the scenario of aircraft skin knock detection, the system finds that a certain inspector has limited wrist flexibility and it is difficult to maintain stability in the traditional two-finger knock posture.

[0069] It can be seen that through the above technical means, this embodiment realizes the objective evaluation and personalized guidance of tactile operation quality, effectively solving the contradiction between tactile skill standardization and personalization.

[0070] Step 6. Based on the operation improvement suggestions and the actual operation data of the inspectors, integrate the tactile inspection results, acoustic detection results, and visual inspection results to generate a comprehensive evaluation report.

[0071] Specifically, it includes the following steps: Step 6.1. Apply the multi-modal data fusion algorithm to fuse the tactile operation results and visual inspection results to generate a comprehensive evaluation report:

[0072] Among them, represents the comprehensive evaluation result, which is the final detection and evaluation result obtained after multi-modal data fusion; represents the fusion function, which is an algorithm function that integrates data of different modalities into a single result; represents the tactile inspection result, which is the analysis result of the detection data obtained through tactile sensing devices such as intelligent gloves; represents the visual inspection result, which is the analysis result of the images obtained through visual sensing devices such as cameras; represents the acoustic inspection result, which is the analysis result of the sound signals obtained through acoustic sensing devices such as microphones. These parameters together constitute the mathematical expression of multi-modal fusion decision-making, realizing the comprehensive utilization of information from different perception channels.

[0073] The specific implementation method of the multi-modal data fusion algorithm is as follows: Apply the feature-level fusion method to normalize and connect the feature vectors of different modalities to form a joint feature representation; Adopt the decision-level fusion method based on Dempster-Shafer evidence theory to calculate the credibility and uncertainty of the detection results of each modality; Apply the weighted voting mechanism to dynamically adjust the weights of each modality in the fusion decision according to the historical accuracy of each modality under different detection conditions; Adopt the conflict detection and resolution strategy. When there are significant differences in the results of different modalities, the system will mark the high-confidence conflict points and recommend additional verification.

[0074] In the inspection scenario of the honeycomb structure of aircraft composite materials, this algorithm fuses the results of tactile tapping detection, acoustic analysis, and visual image detection. When all three modalities indicate an abnormality in a certain area, the system gives a high-confidence defect report; when the acoustic and tactile results indicate an abnormality while the visual is normal (such as internal debonding without surface deformation), the system gives a medium-confidence abnormality warning based on the weight of evidence; when only a single modality indicates an abnormality, the system will recommend additional verification.

[0075] The multi-modal data fusion algorithm can adopt the following different implementation strategies: In some embodiments, an end-to-end fusion method based on deep learning, such as a multi-modal attention network, can be adopted. This method automatically learns the correlation and complementarity between modalities through an attention mechanism and does not require manual design of fusion rules; For inspection processes with strong temporal characteristics, a temporal perception fusion strategy can be selected, such as a Long Short-Term Memory (LSTM) network or a Temporal Convolutional Network (TCN), to handle the co-variation of different modalities in the time dimension; In scenarios with high data uncertainty, a fusion method based on fuzzy set theory can be adopted. This method can better express and handle the ambiguity and uncertainty of detection results and provide a more reasonable comprehensive evaluation.

[0076] In terms of fusion decision support: A multi-level fusion architecture can be adopted to integrate data at different abstraction levels. For example, raw sensing data is fused at a low level, extracted features are fused at a middle level, and decision results of each modality are fused at a high level; Background knowledge constraints can be introduced to incorporate prior knowledge such as component historical maintenance records, material properties, and service conditions into the fusion process to improve the context relevance of decisions; For particularly complex or high-risk inspection tasks, a human-machine collaborative fusion mode can be adopted. The system provides preliminary fusion results and uncertainty analysis, and human experts make the final decision.

[0077] Step 6.2: Construct a tactile knowledge base system, which consists of the following components: Data acquisition module: Continuously collect and store tactile operation data of different inspectors; Analysis and optimization module: Apply statistical analysis algorithms to analyze operation data and refine best practices; Knowledge representation module: Convert the refined best practices into a structured knowledge representation form; Knowledge application module: Use the experience in the knowledge base for training new personnel and assisting in decision-making.

[0078] The specific implementation method of the tactile knowledge base system is as follows: The data acquisition module adopts a distributed data collection architecture to collect and preliminarily process operation data through edge computing devices, and then encrypt and transmit it to the central database; The analysis and optimization module applies clustering algorithms to group and classify the collected operation data to identify the characteristic differences between efficient and inefficient operation modes; The knowledge representation module adopts an ontology modeling method to construct a multi-level relationship network containing component operation characteristic results to achieve the formal representation of complex tactile knowledge; The knowledge application module applies the case-based reasoning method to retrieve similar cases from the knowledge base according to the current inspection scenario, and provides targeted guidance and suggestions.

[0079] In practical applications, when a newly recruited inspector starts to learn the task of inspecting the loose installation of fuel pumps, the system extracts relevant expert operation models from the knowledge base, including the tactile feedback characteristics in typical loose cases, common misjudgment cases and their correction methods. As the system continues to run, when it is found that an inspector has developed a more efficient detection technique (such as faster detection speed and no reduction in accuracy), this technique will be automatically recognized, verified and added to the knowledge base for other inspectors to learn, realizing the iterative optimization of organizational-level skills.

[0080] Step 6.3, apply the incremental learning algorithm, and continuously update and optimize the standard operation model and decision rules based on the continuously accumulated operation data and feedback, so as to realize the structured inheritance and iterative optimization of tactile experience. Therefore, this embodiment effectively solves the technical problem of the loss of expert experience.

[0081] The specific implementation method of the incremental learning algorithm is as follows: Apply an online learning framework so that the model can process continuously arriving new data without completely retraining; Adopt a historical data weighting mechanism based on the forgetting curve to reduce the influence weight of old data and enhance the learning effect of new data; Apply a change detection algorithm to monitor the model performance. When a performance decline or a new pattern is identified, trigger the model update process; Adopt a sample caching strategy based on importance sampling to retain representative historical data samples and balance the model stability and adaptability.

[0082] Application example of this embodiment: This embodiment has been applied to the detection of composite material components in a large aircraft maintenance enterprise. The specific scenario is the detection of delamination in the composite material wing skin of an aircraft. This detection task highly depends on the knocking detection experience and tactile skills of inspectors.

[0083] Application scenario description: Internal delamination defects in the composite material wing skin are difficult to directly observe visually. Traditionally, experienced inspectors need to detect them through specific knocking methods. This enterprise faces the following key problems: The core detection expert is about to retire, and it is difficult to teach his knocking detection skills to novices through conventional methods; The detection quality highly depends on personal experience, and the consistency of detection results of different inspectors is poor; The airport environment has serious noise interference, which affects the accuracy of knocking sound judgment.

[0084] Implementation process example: Implementation of multi-modal perception data acquisition:

[0085] In the application implementation, first, 3 of the most senior composite skin inspection experts in the enterprise were selected. They wore intelligent gloves equipped with 16 pressure sensors and 8 acceleration sensors and performed inspection operations on the wing skin templates containing preset delamination defects.

[0086] During the data acquisition process, four inspection scenarios were specifically designed: Knock detection in the normal area (without defects); Knock detection in the slightly delaminated area (delamination area < 10 cm²); Knock detection in the moderately delaminated area (delamination area 10 to 30 cm²); Knock detection in the severely delaminated area (delamination area > 30 cm²).

[0087] For each inspection scenario of each expert, 50 groups of data samples were collected respectively, totaling 600 groups of multi-modal data, including: The knock force curve recorded by the intelligent glove, with a sampling rate of 500 Hz; The acoustic signal collected by the directional microphone array, with a sampling rate of 44.1 kHz; The hand movement trajectory recorded by the motion capture system, with a sampling accuracy of millimeter level.

[0088] Analysis found that senior experts showed obvious regular characteristics during the knocking process: the knocking action was centered on the wrist, with an incident angle of about 70°, the single knock force was maintained within the range of 4.5 to 5.2 N, and the knocking frequency remained at an average of 1.8 times per second. There was an obvious correlation between these parameters and the defect type.

[0089] Implementation of tactile skill AR visualization presentation:

[0090] Based on the collected multi-modal data, the system constructed a "tactile operation model for aircraft skin knock detection" and presented it to the inspectors through customized AR glasses. The system has the following key functions in actual applications: Automatic component recognition and operation guidance: When the inspector wears AR glasses to observe the wing skin, the system automatically recognizes the current part and superimposes a semi-transparent animation guidance in the AR field of view, showing the correct knocking method, angle and frequency. The animation guidance uses a 3D hand model, highlighting the knocking point, knocking direction and force, and at the same time showing the standard knock force curve on the side.

[0091] Real-time Tactile Feedback Correction: When the inspector performs a tapping operation, the micro-vibration elements in the intelligent glove provide real-time tactile feedback. When the tapping angle of the inspector deviates from the standard angle by more than 10°, a slight vibration warning is provided on the back of the glove; when the tapping force exceeds the ideal range, rhythmic vibrations are provided at the fingertip positions to guide the adjustment.

[0092] Enhanced Presentation of Acoustic Results: The system processes the acoustic signals generated by the inspector's current tapping in real time, synchronously displays the spectrogram in the AR interface and superimposes the typical spectrogram reference lines of the normal / abnormal areas, making the subtle acoustic differences that are difficult to directly distinguish audible become visual. The system also marks the energy differences in the key frequency bands (such as the layering characteristic frequency band from 800 to 1200 Hz).

[0093] In practical applications, when a newly recruited inspector was inspecting the outer panel of the 787 wing, the system first identified that this area belonged to the "CFRP skin-stringer connection area" and immediately called the corresponding standard operation model. A virtual hand appeared in the AR interface, demonstrating the ideal tapping trajectory in this area (detecting once every 80 mm along the stringer direction).

[0094] When the inspector tried to tap, the system detected that the tapping force was insufficient (only 3.2 N), immediately generated vibration feedback through the glove at the index finger and thumb positions, and at the same time, the force indicator bar in the AR interface turned yellow, indicating that the force needed to be increased. After the inspector adjusted accordingly, a small internal delamination was successfully detected, which had been missed by other novice inspectors many times before.

[0095] Verification of Technical Effects: Six months after the application of this embodiment in the delamination detection of a composite material wing skin, a systematic evaluation of the key technical effects was carried out, and the results are as follows: Digital Standardization Effect of Tactile Skills:

[0096] Through the tracking evaluation of the skill mastery of 10 novice inspectors, the following data was obtained: Under the traditional training method (master-apprentice oral instruction), it took an average of 47 days for novices to reach the basic qualified level (detection accuracy rate > 80%); After adopting this embodiment, it only took an average of 23 days for novices to reach the same level, and the skill mastery time was shortened by 51%; The consistency of inspection operations (evaluating the consistency of the inspection results of 10 inspectors on the same set of samples) was improved from 68% to 91%; The standard deviation of the operation action parameters of the inspectors decreased: the standard deviation of the tapping angle decreased from ±15° to ±4°, and the standard deviation of the tapping force decreased from ±1.8 N to ±0.6 N.

[0097] After the departure of the retired experts, the new team of inspectors trained with this system still maintained a high level of detection ability, effectively solving the problem of the interruption of knowledge inheritance.

[0098] Improvement effect of the percussion detection accuracy:

[0099] By conducting blind tests on samples with preset defects of different types and degrees, the detection accuracy was evaluated: For slight delamination defects, the detection rate of the traditional detection method was 63%, which increased to 89% after adopting this implementation method; For moderate delamination defects, the accuracy rate of the traditional detection method was 76%, which increased to 94% after adopting this implementation method; In the 75dB airport background noise environment, the accuracy rate of the traditional detection method dropped to 58%, while it could still be maintained at 87% after adopting this implementation method; The false alarm rate (misjudging normal areas as defects) decreased from 12% to 4.5%.

[0100] During the actual maintenance process, within 6 months of applying this implementation method, a total of 28 early delamination defects that were not detected by previous conventional inspections were detected, preventing potential safety hazards. At the same time, the inspection efficiency increased by 38%, and the inspection time for each aircraft was reduced from 3.2 hours to 2.0 hours on average.

[0101] The above describes the embodiments of the present invention. However, these embodiments are not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative and not restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of this embodiment.

Claims

1. An intelligent inspection method based on AI multi-source fusion, characterized in that, Including the following steps: Collect multimodal perception data when an expert performs a tactile inspection operation, where the multimodal perception data includes tactile parameters, acoustic signals, and action trajectories; Analyze the multimodal perception data to construct a biomechanical model and a tactile knowledge mapping model; Based on the biomechanical model and the tactile knowledge mapping model, establish a standard tactile operation model and an acoustic feature mapping relationship database; Convert the standard tactile operation model into an animation guidance and tactile feedback in an augmented reality interface; Collect the actual operation data of the inspector and compare it with the standard tactile operation model to generate operation improvement suggestions; Based on the operation improvement suggestions and the actual operation data of the inspector, integrate the tactile inspection results, acoustic detection results, and visual inspection results to generate a comprehensive evaluation report.

2. The intelligent inspection method based on AI multi-source fusion according to claim 1, wherein The step of collecting multimodal perception data when an expert performs a tactile inspection operation includes: Through an intelligent glove equipped with a pressure sensor, an acceleration sensor, and a position sensor, collect the contact force curve, action trajectory, and vibration response data when an expert performs a tactile inspection; Through a directional microphone array, collect knocking sound waves and structural vibration signals; Through a motion capture system, record the overall operation posture and action trajectory data.

3. An intelligent inspection method based on AI multi-source fusion according to claim 1, characterized in that, The step of analyzing the multimodal perception data to construct a biomechanical model and a tactile knowledge mapping model includes: Apply a biomechanical analysis algorithm to analyze the biomechanical characteristics of an expert's tactile actions, and construct a biomechanical mathematical model including joint angle changes, muscle torque distribution, and energy efficiency; Apply a feature extraction algorithm to analyze the differences in tactile feedback of an expert for components in different states, and establish a state feedback mapping model; Conduct statistical analysis on the operation data of multiple experts, extract the kinematic characteristics of best practices, and form a standardized tactile inspection parameter library.

4. An intelligent inspection method based on AI multi-source fusion according to claim 1, characterized in that, The step of establishing a standard tactile operation model and an acoustic feature mapping relationship database based on the biomechanical model and the tactile knowledge mapping model includes: Apply a multivariate time series analysis algorithm to analyze an expert's tactile inspection actions, extract key parameters, and establish a standard operation model; Use an acoustic signal processing algorithm to analyze the knocking sound wave data, extract spectral features, time-domain features, and time-frequency features, and construct a mapping relationship database of defect type acoustic features; Apply a classification algorithm to construct an acoustic feature discrimination model to achieve an objective judgment of knocking detection.

5. The intelligent inspection method based on AI multi-source fusion according to claim 1, characterized in that, The step of converting the standard tactile operation model into an animation guidance and tactile feedback in an augmented reality interface includes: Apply a 3D visualization algorithm to convert the skill features into an animation guidance and quantitative parameters in an augmented reality interface; Construct a context-aware system based on component recognition, and retrieve the corresponding operation guidelines from the standard operation model library according to the recognized components and the current task requirements; Construct a tactile feedback system to provide tactile feedback through the micro vibration elements in the intelligent glove to guide correct operations.

6. The intelligent inspection method based on AI multi-source fusion according to claim 1 is characterized in that, The step of collecting the actual operation data of the inspector and comparing it with the standard tactile operation model to generate operation improvement suggestions includes: Adopt a real-time sensing data analysis algorithm, and use the inertial sensor and pressure sensor integrated in the intelligent glove to collect the actual operation data of the inspector; Based on the similarity calculation algorithm, compare the collected actual operation data with the standard operation model to calculate the operation similarity; Apply the personalized learning model to generate customized skill adjustment suggestions based on the biomechanical characteristics and skill mastery of the inspectors.

7. An intelligent inspection method based on AI multi-source fusion according to claim 1, characterized in that The steps of fusing the tactile inspection results, acoustic detection results and visual inspection results to generate a comprehensive evaluation report include: Apply the multimodal data fusion algorithm to fuse the tactile operation results with the visual inspection results and acoustic detection results; Construct a tactile knowledge base system to collect and analyze the tactile operation data of different inspectors; Apply the incremental learning algorithm to continuously update and optimize the standard operation model and decision rules based on the continuously accumulated operation data and feedback.

8. The intelligent inspection method based on AI multi-source fusion according to claim 7, wherein The multimodal data fusion algorithm includes feature-level fusion and decision-level fusion. Among them, feature-level fusion normalizes the feature vectors of different modalities and then connects them to form a joint feature representation, and decision-level fusion calculates the credibility and uncertainty of the detection results of each modality based on the Dempster-Shafer evidence theory.

9. An intelligent inspection method based on AI multi-source fusion according to claim 7, characterized in that, The tactile knowledge base system includes: A data acquisition module for continuously collecting and storing the tactile operation data of different inspectors; An analysis and optimization module for analyzing the operation data using statistical analysis algorithms to extract best practices; A knowledge representation module for converting the extracted best practices into a structured knowledge representation form; A knowledge application module for using the experience in the knowledge base for training new personnel and assisting in decision-making.

10. An intelligent inspection system based on AI multi-source fusion, which is used to execute an intelligent inspection method based on AI multi-source fusion according to any one of claims 1-9, characterized in that, Including: A multimodal perception data acquisition module for acquiring multimodal perception data when an expert performs a tactile inspection operation. The multimodal perception data includes tactile parameters, acoustic signals and action trajectories; A data analysis and processing module for analyzing the multimodal perception data to construct a biomechanical model and a tactile knowledge mapping model; A model establishment module for establishing a standard tactile operation model and an acoustic feature mapping relationship database based on the biomechanical model and the tactile knowledge mapping model; An interaction feedback module for converting the standard tactile operation model into an animation guidance and tactile feedback in the augmented reality interface; An operation evaluation module for collecting the actual operation data of the inspector and comparing it with the standard tactile operation model to generate operation improvement suggestions; A result fusion module for fusing the tactile inspection results, acoustic detection results and visual inspection results based on the operation improvement suggestions and the actual operation data of the inspector to generate a comprehensive evaluation report.

Images