Embodied intelligent AI inspection method based on digital twinning

Abstract

The application relates to the technical field of intelligent robot inspection, and discloses a somatic intelligent AI inspection method based on digital twinning, wherein the somatic intelligent AI inspection method based on digital twinning comprises the following steps: a dynamic attention multi-layer feature extraction network is used to analyze multispectral images collected by a robot-mounted sensor to generate a real-time distribution diagram of industrial facility abnormal coverage; a space-time sequence contrast learning model is used to analyze historical abnormal coverage data to generate an abnormal coverage trend prediction; a physically constrained neural differential equation model is used to analyze an abnormal coverage evolution process to generate an accurate abnormal evolution prediction result; a multi-objective optimization algorithm is used to generate an optimal inspection path; a digital twinning system is dynamically updated, and an industrial facility maintenance optimization suggestion is generated; and through the dynamic attention multi-layer feature extraction network and a light condition self-adaptive enhancement algorithm, the detection difficulty in a strong light and high reflection environment is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot inspection technology, and more specifically, to an embodied intelligent AI inspection method based on digital twins. Background Technology

[0002] With the development of industrial automation and intelligence, intelligent operation and maintenance of large-scale facilities and infrastructure has become a key link in improving operational efficiency and reducing maintenance costs. In complex environments, target facilities often face unique challenges such as abnormal surface changes and complex, ever-changing environmental conditions. The abnormal state of a facility's surface is a dynamic process, influenced by various factors such as the environment, terrain, and geographical location, severely impacting the facility's normal operation. Simultaneously, complex lighting and high-reflectivity environments interfere with the detection accuracy of traditional robotic inspection systems, while variable terrain conditions further increase the difficulty of inspection tasks.

[0003] Existing robotic inspection technologies suffer from the following main problems: First, traditional image recognition technology still has room for improvement in its accuracy of detecting surface anomalies under complex lighting conditions. Current image recognition algorithms need further enhancement in their ability to distinguish between areas of strong light reflection and abnormal areas. In certain complex environments, the false alarm rate remains high, affecting the reliability of maintenance decisions. Second, mainstream detection methods primarily focus on static state detection, with limited ability to predict the temporal evolution of abnormal states, leading to relatively passive maintenance strategies. Existing technologies mainly acquire current anomaly information, lacking sufficient accuracy in predicting future anomaly trends, making it difficult for maintenance teams to develop precise maintenance plans in advance. Third, current prediction models still have shortcomings in terms of physical process constraints, posing challenges to accurately describing the dynamic evolution of abnormal states under sparse data conditions, especially in predicting rare environmental conditions. Most data-driven prediction models suffer from limited accuracy when data is scarce or when faced with new environmental conditions. Finally, although intelligent path planning technology has made progress, there are still efficiency issues in optimizing inspection paths in specific complex environments. Existing technologies do not comprehensively consider environmental factors, abnormal states of target objects, and prediction trends in path planning, making it difficult to achieve accurate coverage of high-risk areas within a limited time, resulting in less than ideal efficiency in the allocation of inspection resources.

[0004] In summary, there is an urgent need for a robotic twin inspection system that can adapt to complex environments, has predictive capabilities, intelligent path optimization, and supports closed-loop optimization, in order to improve facility operation and maintenance efficiency and economic benefits. Summary of the Invention

[0005] This invention provides an embodied intelligent AI inspection method based on digital twins, which solves the technical problems of traditional robot inspection systems in related technologies, such as low accuracy of abnormal coverage detection under complex lighting conditions, inability of static coverage detection to predict the accumulation trend of abnormal states, limited generalization ability of abnormal prediction models, and low efficiency of inspection path planning.

[0006] This invention provides an embodied intelligent AI inspection method based on digital twins, comprising:

[0007] By using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors on a robot, a real-time distribution map of abnormal coverage of industrial facilities is generated.

[0008] Based on the real-time distribution map of abnormal coverage of industrial facilities, a spatiotemporal sequence comparison learning model is used to analyze historical abnormal coverage data and generate an abnormal coverage trend prediction.

[0009] Based on the prediction of anomalous coverage trends, the evolution process of anomalous coverage is analyzed using a physical constraint neural differential equation model, generating accurate anomalous evolution prediction results.

[0010] Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, the optimal inspection path is generated using a multi-objective optimization algorithm.

[0011] Based on multispectral images, real-time distribution maps of abnormal coverage of industrial facilities, prediction results of abnormal evolution, and real-time data during the inspection process, the digital twin system is dynamically updated, and suggestions for optimizing the maintenance of industrial facilities are generated.

[0012] Furthermore, the step of using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors on a robot and generating a real-time distribution map of abnormal coverage of industrial facilities includes:

[0013] The robot uses a multispectral camera and other sensors to collect images of the facility's surface, including the visible and near-infrared bands.

[0014] An adaptive enhancement algorithm based on illumination conditions is applied to preprocess multispectral images to reduce the interference of complex illumination conditions on detection accuracy;

[0015] The preprocessed image is processed by a dynamic attention multi-layer feature extraction network, and the feature map weights are dynamically adjusted so that the model pays more attention to real abnormal areas and ignores areas with lighting interference.

[0016] Features extracted from different convolutional layers are fused through a feature pyramid network to capture anomalous features at different granularities.

[0017] Anomaly coverage of each pixel is calculated using a fully connected layer and a regression layer to generate anomaly coverage distribution map of industrial facilities.

[0018] Furthermore, the step of using a spatiotemporal sequence contrastive learning model to analyze historical abnormal coverage data and generate anomaly coverage trend predictions includes:

[0019] Collect abnormal coverage images of the same industrial facility at different time points, and combine them with terrain data to construct a time-series dataset;

[0020] By using a spatiotemporal feature extraction network, the temporal evolution pattern of anomaly accumulation is captured;

[0021] By applying a contrastive learning framework, the model learns the feature differences under different anomaly accumulation rates;

[0022] By combining terrain data and historical accumulation patterns, an abnormal accumulation rate model is established.

[0023] Based on the accumulation rate model, predict the future distribution of anomaly coverage.

[0024] Furthermore, the step of analyzing the anomalous coverage evolution process using a physically constrained neural differential equation model to generate accurate anomalous evolution prediction results includes:

[0025] The evolution of anomalous coverage can be expressed as a spatiotemporal partial differential equation:

[0026] ;

[0027] in Indicates position ,time Abnormal coverage; The partial derivative of coverage with respect to time represents the rate of change of coverage over time. Indicates time Topographical and environmental conditions; The spatial gradient representing coverage, , Represents the partial derivative operator; This represents the set of parameters to be learned. Represents the dynamic function to be learned, describing the physical and data-driven process of anomalous evolution;

[0028] Constructing a hybrid structure that combines prior physical knowledge with neural networks:

[0029] ;

[0030] in Represents the overall dynamic function; Represents physical priors, based on fluid dynamics equations; Represents the residual term learned by the neural network;

[0031] Construct a physical consistency constraint loss function to ensure that the model predictions conform to physical laws;

[0032] An adaptive numerical solver is constructed to efficiently solve neural differential equations using a variable step-size algorithm with controllable error thresholds.

[0033] Build an end-to-end training framework to jointly optimize physical parameters and neural network parameters.

[0034] Furthermore, the step of generating the optimal inspection path using a multi-objective optimization algorithm includes:

[0035] Based on the status information and anomaly prediction results of industrial facilities in the digital twin system, the inspection priority of industrial facilities is calculated.

[0036] By taking real-time terrain data into account, a robot energy consumption prediction model under the influence of terrain is constructed.

[0037] Based on facility priority, energy consumption prediction, path accessibility and safety factors, a multi-objective optimization problem is established to generate the optimal inspection path;

[0038] During the inspection process, the inspection route is dynamically adjusted based on real-time environmental changes and the status of industrial facilities.

[0039] The optimized inspection path is converted into robot motion control commands to enable autonomous movement and execution of inspection tasks.

[0040] Furthermore, the step of dynamically updating the digital twin system and generating maintenance optimization suggestions for industrial facilities includes:

[0041] Data such as the distribution map of abnormal coverage of industrial facilities obtained by robot inspections are transmitted back to the digital twin system in real time;

[0042] Based on the processed data, update the status information of industrial facilities in the digital twin system;

[0043] Based on the newly acquired measured data, the parameters of the anomaly prediction model are corrected;

[0044] Develop industrial facility maintenance strategies based on abnormal coverage distribution and predicted trends;

[0045] Based on the optimization results, maintenance operation suggestions are generated, including maintenance schedule, regional priority division and resource allocation scheme.

[0046] Furthermore, the adaptive enhancement algorithm for lighting conditions is performed through the following steps:

[0047] Estimate the light intensity and angle in the image:

[0048] ;

[0049] in Represents pixels Estimated light intensity at the location; Represents pixels Visible light band image intensity at that location; Represents pixels Near-infrared image intensity at the location; This represents the illumination estimation function, which is used to fuse visible light and near-infrared information and output the illumination intensity.

[0050] Calculate the adaptive enhancement parameters:

[0051] ;

[0052] in Represents pixels Adaptive enhancement parameters at the location; This represents a parameter mapping function that dynamically adjusts the enhancement coefficient based on the local illumination intensity. Represents pixels Estimated light intensity at the location;

[0053] Application adaptive enhancement:

[0054] ;

[0055] in Indicates the enhanced image in Pixel value at; Indicates the original image in Pixel value at; Represents pixels Adaptive enhancement parameters at the location.

[0056] Furthermore, the core of the dynamic attention multi-layer feature extraction network is the calculation of the attention weight matrix:

[0057] Attention weight calculation:

[0058] ;

[0059] in Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the attention mapping function, which maps the feature map to a weight distribution. This represents the sigmoid activation function;

[0060] Weighted sum of feature maps and attention weights:

[0061] ;

[0062] in This represents the weighted feature map; Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the element-wise multiplication operator.

[0063] Furthermore, the contrastive learning framework enables the model to learn feature differences under different anomaly accumulation rates; the loss function for contrastive learning is defined as:

[0064] ;

[0065] in This represents the contrastive learning loss function; This represents the feature representation of the current sample. Indicates and Paired positive sample feature representation, This represents the feature representation of other samples in the batch; Represents the cosine similarity function; It represents temperature parameters and controls the smoothness of the distribution; Indicates the sample index. , This represents the original number of samples within a batch, while This represents the total number of samples after considering data augmentation; Indicates an indicator function, when Time to take Otherwise ; Represents the exponentiation operator; This represents the natural logarithm operator.

[0066] The present invention provides a computer storage medium, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-described embodied intelligent AI inspection method based on digital twins.

[0067] The beneficial effects of this invention are as follows: by employing a dynamic attention multi-layer feature extraction network and an adaptive enhancement algorithm for lighting conditions, it effectively solves the detection challenges in strong light and high reflectivity environments. Detection accuracy is improved under strong lighting conditions, in areas of drastic lighting changes, and in abnormal boundary regions, providing a more reliable data foundation for maintenance decisions.

[0068] By fusing spatiotemporal sequence contrastive learning and physical constraint neural differential equation model, the system can predict the accumulation trend of abnormal states in advance, improving the prediction accuracy and providing sufficient response time for operation and maintenance decisions.

[0069] By combining physical prior knowledge with neural networks, the hybrid structure enables the model to maintain reliable prediction performance in data-sparse and rare terrain environments, and reduces prediction error when facing unseen terrain conditions.

[0070] Based on multi-objective optimization path planning that considers industrial facility priority and terrain influence, the system reduces travel distance, improves coverage efficiency, increases battery utilization efficiency, increases the coverage area per inspection, and improves inspection work efficiency.

[0071] By accurately predicting anomalies and prioritizing them, the maintenance costs of industrial facilities can be reduced, the average annual power generation can be increased, and the return on investment can be improved.

[0072] The dynamic updates and model parameter corrections of the digital twin system form a closed-loop optimization mechanism, which improves the accuracy of the system model every month. After long-term operation, the system maintains a stable high accuracy level, demonstrating its self-learning and continuous optimization capabilities. Attached Figure Description

[0073] Figure 1 This is a flowchart of an embodied intelligent AI inspection method based on digital twins in this invention;

[0074] Figure 2 This is a flowchart of step 1 in this invention;

[0075] Figure 3 This is a flowchart of step 2 in this invention;

[0076] Figure 4 This is a flowchart of step 3 in this invention;

[0077] Figure 5 This is a flowchart of step 4 in this invention;

[0078] Figure 6 This is a flowchart of step 5 in this invention. Detailed Implementation

[0079] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0080] At least one embodiment of the present invention discloses an embodied intelligent AI inspection method based on digital twins, such as... Figures 1 to 6 As shown, it includes:

[0081] Step 1: Utilize a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors mounted on the robot, and generate a real-time distribution map of abnormal coverage of industrial facilities.

[0082] This step utilizes a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors mounted on a robot, generating a real-time distribution map of abnormal coverage of industrial facilities. It should be noted that this specifically includes:

[0083] Step 1.1: Acquire multispectral image data;

[0084] The robot uses a multispectral camera and other sensors to acquire images of the facility's surface, including visible and near-infrared bands, forming multi-channel image data of the industrial facility. The acquired image data includes spectral information on abnormal coverage areas, reflective areas, and normal areas of the industrial facility's surface.

[0085] Abnormal coverage areas on the surface of industrial facilities refer to various abnormal areas appearing on the surface of industrial facilities, including:

[0086] Areas covered by surface accumulations: Surface coverage caused by natural environmental factors such as sand and dust and debris; Surface pollution caused by production environmental factors such as industrial dust and oil.

[0087] Material deterioration areas: Surface corrosion and oxidation areas; material aging and weathering areas; material discoloration areas affected by environmental factors;

[0088] Areas with structural anomalies: surface cracks and damage; deformation and dents; areas with loose or missing connectors;

[0089] Functional anomaly areas: areas with abnormal temperature; areas with abnormal humidity; areas with electrical insulation failure;

[0090] These anomalous coverage areas will exhibit different spectral characteristics than normal areas in multispectral images.

[0091] Step 1.2: Apply the illumination condition adaptive enhancement algorithm;

[0092] An adaptive illumination enhancement algorithm is applied to preprocess multispectral images to reduce the interference of complex illumination conditions on detection accuracy. This algorithm proceeds through the following steps:

[0093] First, estimate the light intensity and angle in the image:

[0094] ;

[0095] in Represents pixels Estimated light intensity at the location; Represents pixels Visible light band image intensity at that location; Represents pixels Near-infrared image intensity at the location; This represents the illumination estimation function, which is used to fuse visible light and near-infrared information to output illumination intensity.

[0096] Then, calculate the adaptive enhancement parameters:

[0097] ;

[0098] in Represents pixels Adaptive enhancement parameters at the location; This represents a parameter mapping function that dynamically adjusts the enhancement coefficient based on the local illumination intensity. Represents pixels The estimated light intensity at that location.

[0099] Finally, apply adaptive enhancement:

[0100] ;

[0101] in Indicates the enhanced image in Pixel value at; Indicates the original image in Pixel value at; Represents pixels Adaptive enhancement parameters at the location.

[0102] Step 1.3: Process the image using a dynamic attention multi-layer feature extraction network;

[0103] The preprocessed image is processed by a dynamic attention multi-layer feature extraction network. This network dynamically adjusts the feature map weights, making the model focus more on real-world anomaly regions while ignoring regions affected by lighting interference. The core of the dynamic attention model is the calculation of the attention weight matrix:

[0104] ;

[0105] in Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the attention mapping function, which maps the feature map to a weight distribution. This represents the sigmoid activation function.

[0106] The weighted result of the feature map and attention weights is as follows:

[0107] ;

[0108] in This represents the weighted feature map; Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the element-wise multiplication operator.

[0109] Step 1.4: Perform multi-scale feature fusion;

[0110] Features extracted from different convolutional layers are fused using a feature pyramid network to capture anomalous features at different granularities:

[0111] ;

[0112] in Indicates the first The feature pyramid output of the layer; This indicates an upsampling operation, which upsamples the high-level feature map to the current layer resolution; Indicates the first The feature pyramid output of the layer; Indicates the first Lateral connectivity features of layers; This indicates a convolution operation used for feature fusion.

[0113] Step 1.5: Calculate the anomaly coverage distribution map;

[0114] Finally, the abnormal coverage of each pixel is calculated using fully connected layers and regression layers to generate an abnormal coverage distribution map of industrial facilities:

[0115] ;

[0116] in Represents pixels Anomaly coverage at [location], value range , This indicates no abnormalities. Indicates complete abnormal coverage; The fused feature map is represented in The eigenvector at that location; This represents the coverage mapping function, which is typically a fully connected layer with regression activation.

[0117] Step 2: Based on the real-time distribution map of abnormal coverage of industrial facilities, analyze historical abnormal coverage data using a spatiotemporal sequence comparison learning model to generate anomaly coverage trend prediction.

[0118] According to one embodiment of this application, this step utilizes a spatiotemporal sequence contrastive learning model to analyze historical abnormal coverage data and generate anomaly coverage trend predictions. It should be understood that this specifically includes:

[0119] Step 2.1, construct the time series dataset;

[0120] Anomaly coverage images of the same industrial facility at different time points were collected and combined with topographic data to construct a time-series dataset. Each time-series sample contains:

[0121] ;

[0122] in Indicates the first A time-series sample set of an industrial facility; , , They represent the first The industrial facility in the first , , Abnormal overlay images at various time points; , , They represent the first The industrial facility in the first , , Terrain data corresponding to each time point; , , They represent the first The industrial facility in the first , , A timestamp at a specific point in time; Indicates the timing length.

[0123] Step 2.2, extract spatiotemporal features;

[0124] A spatiotemporal feature extraction network is used to capture the temporal evolution patterns of anomaly accumulation. This network combines temporal and spatial convolutional modules to extract joint spatiotemporal features.

[0125] Spatial feature extraction:

[0126] ;

[0127] in Indicates time Spatial characteristics; Represents a multi-layer feature extraction network in spatial representation; Indicates time The abnormal coverage distribution.

[0128] Temporal feature extraction:

[0129] ;

[0130] in Indicates position The temporal characteristics of the location; Represents a temporal convolutional network; Indicates position Time series data at the location; middle The symbol represents a tensor slice, which extracts the sequence data of a spatial location across all time steps.

[0131] Spatiotemporal joint characteristics:

[0132] ;

[0133] in Represents spatiotemporal joint features (fused feature vectors); This indicates the feature concatenation operation, which involves spatial features. With time characteristics The features are concatenated along the feature dimension to form a new joint feature vector; Representing spatial characteristics; Indicates time characteristics.

[0134] Step 2.3: Train the contrastive learning model;

[0135] A contrastive learning framework is applied to enable the model to learn feature differences under different anomaly accumulation rates. The loss function for contrastive learning is defined as:

[0136] ;

[0137] in This represents the contrastive learning loss function; This represents the feature representation of the current sample. Indicates and Paired positive sample feature representation, This represents the feature representation of other samples in the batch; Represents the cosine similarity function; It represents temperature parameters and controls the smoothness of the distribution; Indicates the sample index. , This represents the original number of samples within a batch, while This represents the total number of samples after considering data augmentation; Indicates an indicator function, when Time to take Otherwise ; Represents the exponentiation operator; This represents the natural logarithm operator.

[0138] Step 2.4: Establish an anomaly accumulation rate model;

[0139] By combining terrain data and historical accumulation patterns, an anomaly accumulation rate model is established:

[0140] ;

[0141] in Indicates the location ,time and terrain and environmental conditions The abnormal accumulation rate (the rate of change in coverage per unit area per unit time). Indicates position ,time The spatiotemporal characteristics of the location; Indicates time Topographic environmental conditions (vector); This represents the rate mapping function.

[0142] Step 2.5, predict abnormal coverage trends;

[0143] Based on the accumulation rate model, predict future anomaly coverage:

[0144] ;

[0145] in Indicates the predicted future time In position Abnormal coverage at the location; Indicates the current time exist Abnormal coverage at the location; Indicates time ,Location Topographical and environmental conditions The abnormal accumulation rate below; Indicates time In the interval The integral over the period represents the cumulative abnormal change. This indicates the prediction time step.

[0146] This formula can be solved approximately using numerical integration.

[0147] Step 3: Based on the anomaly coverage trend prediction, the anomaly coverage evolution process is analyzed using a physical constraint neural differential equation model to generate accurate anomaly evolution prediction results.

[0148] According to another embodiment of this application, this step utilizes a physically constrained neural differential equation model to analyze the anomalous coverage evolution process and generate accurate anomalous evolution prediction results. Furthermore, it specifically includes:

[0149] Step 3.1, construct the anomaly coverage evolution equation;

[0150] The evolution of anomalous coverage can be expressed as a spatiotemporal partial differential equation:

[0151] ;

[0152] in Indicates position ,time Abnormal coverage; The partial derivative of coverage with respect to time represents the rate of change of coverage over time. Indicates time Topographical and environmental conditions; The spatial gradient representing coverage, , Represents the partial derivative operator; This represents the set of parameters to be learned. This represents the dynamic function to be learned, describing the physical and data-driven process of anomalous evolution.

[0153] Step 3.2: Construct a hybrid structure of prior physical knowledge and neural network;

[0154] According to one embodiment of this application, a hybrid structure is constructed that combines prior physical knowledge with neural networks:

[0155] ;

[0156] in Represents the overall dynamic function; Represents physical priors, based on fluid dynamics equations; This represents the residual term learned by the neural network.

[0157] The physical terms are as follows:

[0158] ;

[0159] in Representing physical priors, Represents the convection term. For divergence operators, This is the slope gradient vector. For coverage; This represents the diffusion coefficient, indicating the anomalous diffusion capacity. Represents the Laplace operator. , indicating spatial diffusion; This represents the source of the anomaly, indicating the process of generating or eliminating the anomaly.

[0160] The neural network term is:

[0161] ;

[0162] in Represents the residual term learned by the neural network; The parameter is Neural networks; Indicates the abnormal coverage at the current location and time; Indicates topographical and environmental conditions; The spatial gradient represents the abnormal coverage.

[0163] This neural network is used to learn complex nonlinear residual dynamics that physical models fail to capture, thereby improving the overall model's prediction accuracy.

[0164] Step 3.3: Construct the physical consistency constraint loss function;

[0165] To ensure that the model predictions conform to physical laws, according to one embodiment of this application, a physical consistency constraint loss function is constructed:

[0166] ;

[0167] in Indicates the loss of physical consistency; Let represent the square norm, and let represent the sum of squared errors at all points in space. This is the slope gradient vector. For coverage; This represents the diffusion coefficient, indicating the anomalous diffusion capacity. Represents the Laplace operator; This represents the source of the anomaly, indicating the process of generating or eliminating the anomaly.

[0168] The total loss function includes data fitting loss and physical consistency loss:

[0169] ;

[0170] in Represents the total loss function; This represents the data fitting loss; This represents the weighting factor, which controls the weight of physical losses in the total losses.

[0171] Step 3.4: Construct an adaptive numerical solver;

[0172] According to one embodiment of this application, an adaptive numerical solver is constructed to efficiently solve neural differential equations using a variable step-size algorithm with controllable error thresholds.

[0173] The variable step size algorithm dynamically adjusts the integration step size based on local error estimation:

[0174] ;

[0175] in Indicates the first The time step size; Indicates the first The time step size; Indicates the first The local error estimate of the step; Indicates the error tolerance threshold (preset constant); Indicates the order of the numerical integration method; express The root of the power.

[0176] when When, reduce the step size and recalculate; when At this time, increase the step size to improve efficiency.

[0177] Step 3.5, end-to-end training and prediction;

[0178] According to another embodiment of this application, an end-to-end training framework is constructed to jointly optimize physical parameters and neural network parameters:

[0179] For each training sample, the prediction result is obtained by solving the initial value problem:

[0180] ;

[0181] in , , They represent The anomaly coverage at any given time, the terrain and environmental conditions, and the spatial gradient of the anomaly coverage; Indicates the predicted future moment Abnormal coverage; Indicates the current time Abnormal coverage; Represents the dynamic function; Represents the integral variable, and represents time; Indicates to The integral; Indicates in Integral over an interval.

[0182] Calculate the error between the predicted result and the actual coverage, and then backpropagate to update the parameters:

[0183] ;

[0184] in Represents model parameters (including neural network weights and physical parameters); This represents the learning rate and controls the step size of parameter updates. Indicates to The gradient of the total loss function; This represents the assignment operator, indicating parameter updates.

[0185] Once the model is trained, it can predict the evolution of abnormal coverage at any future time. Especially under conditions of sparse data and rare terrain, it can still maintain reliable prediction performance, providing a scientific basis for subsequent maintenance decisions.

[0186] Step 4: Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, the optimal inspection path is generated using a multi-objective optimization algorithm.

[0187] According to a preferred embodiment of this application, this step utilizes a multi-objective optimization algorithm to combine industrial facility status information and anomaly prediction results from the digital twin system to generate an optimal inspection path. It can be understood that this specifically includes:

[0188] Step 4.1, calculate the priority of industrial facilities;

[0189] Based on the status information and anomaly prediction results of industrial facilities in the digital twin system, the inspection priority of industrial facilities is calculated:

[0190] ;

[0191] in Indicates industrial facilities Inspection priority (numerical value, the larger the value, the higher the priority); , , These represent the weights of the current anomaly coverage, the predicted anomaly coverage growth rate, and the historical failure frequency, respectively. Indicates industrial facilities Current anomaly coverage; Indicates industrial facilities The projected abnormal coverage growth rate; Indicates industrial facilities Historical failure frequency.

[0192] Step 4.2: Construct an energy consumption prediction model under the influence of terrain;

[0193] According to one embodiment of this application, a robot energy consumption prediction model under the influence of terrain is constructed by taking into account real-time terrain data:

[0194] ;

[0195] in Indicates the terrain and environmental conditions Below, from position Exercise Energy consumption; Indicates from the ideal flat terrain condition arrive Basic energy consumption; The terrain influence function is defined as follows:

[0196] ;

[0197] in Indicates the uphill / downhill influence coefficient; Represents the surface roughness coefficient; Represents the slope gradient vector; Indicates from arrive The displacement vector; This represents the magnitude of the displacement vector.

[0198] Step 4.3: Establish and solve the multi-objective optimization problem;

[0199] According to another embodiment of this application, a multi-objective optimization problem is established based on facility priority, energy consumption prediction, path accessibility, and safety factors to generate the optimal inspection path:

[0200] Objective function:

[0201] ;

[0202] in Indicates the robot's inspection path; This represents the priority target weight coefficient; This represents the weighting coefficient for energy consumption targets; This indicates all industrial facilities in the path. Summation; Indicates industrial facilities Priority; This represents all adjacent node pairs in the path. Summation; Indicates from arrive Energy consumption; This indicates maximization.

[0203] The constraints include battery energy constraints, motion time constraints, and safe distance constraints:

[0204] Battery energy constraints: ,in To maximize the robot's available energy;

[0205] Exercise time constraints: , For exercise time, The maximum permissible exercise time;

[0206] Safety distance constraints: , Path and obstacles The minimum distance, This is the safety threshold.

[0207] The optimization employs an improved differential evolution algorithm, with the following specific steps:

[0208] Initialize the candidate path population:

[0209] ;

[0210] in , , They respectively represent Article 1, Article 2 to Article 3. Candidate paths, The population size is the total number of candidate paths.

[0211] Each candidate path is subjected to mutation and crossover operations to generate a new path;

[0212] Evaluate the fitness (i.e., objective function value) of the new path and retain the best paths;

[0213] Repeat step 2.3 until convergence or the maximum number of iterations is reached.

[0214] Step 4.4: Implement dynamic path adjustment;

[0215] According to a preferred embodiment of this application, during the inspection process, the inspection path is dynamically adjusted based on real-time acquired environmental changes and industrial facility status:

[0216] ;

[0217] in This indicates the adjusted inspection path; Indicates the original inspection path; This indicates the amount of change in topographical and environmental conditions; This indicates the change in the priority of industrial facilities; This represents a path adjustment function that re-plans the path based on changes in the input.

[0218] If environmental changes lead to increased safety risks or changes in priorities, subsequent path segments will be recalculated.

[0219] Step 4.5: Achieve autonomous motion control for the robot;

[0220] According to another embodiment of this application, the optimized inspection path is converted into robot motion control commands to achieve autonomous movement and execution of inspection tasks:

[0221] ;

[0222] in Indicates time Robot control signals; Indicates time Robot's current location; Indicates time Target position (coordinate vector); Indicates time Topographical and environmental conditions; This represents the motion controller function, which performs motion compensation in conjunction with terrain effects.

[0223] This control system can effectively cope with the challenges posed by the complex terrain and environmental conditions in area B, ensuring that the robot moves accurately along the planned path and completes the image acquisition task.

[0224] Step 5: Based on multispectral images, real-time distribution maps of abnormal coverage of industrial facilities, prediction results of abnormal evolution, and real-time data during the inspection process, dynamically update the digital twin system and generate suggestions for optimizing the maintenance of industrial facilities.

[0225] According to another embodiment of this application, this step utilizes real-time data acquired during inspections to update the digital twin system and generate maintenance optimization suggestions for industrial facilities. It should be noted that this specifically includes:

[0226] Step 5.1, Real-time data transmission and processing;

[0227] According to one embodiment of this application, data such as the distribution map of abnormal coverage of industrial facilities obtained by robot inspection are transmitted back to the digital twin system in real time:

[0228] ;

[0229] in This represents the dataset that is transmitted back in real time. Indicates industrial facilities In time Observational anomaly coverage; Indicates industrial facilities In time Multispectral image data; Indicates time Measured terrain and environmental parameters.

[0230] Perform quality checks and preprocessing on the returned data, removing outliers and filling in missing values:

[0231] ;

[0232] in This represents the processed dataset; This represents data quality control functions, including outlier detection and missing value imputation. This represents the dataset that is transmitted back in real time.

[0233] Step 5.2, update the state of the digital twin model;

[0234] According to another embodiment of this application, the status information of industrial facilities in the digital twin system is updated based on the processed data:

[0235] ;

[0236] in Indicates industrial facilities The new state; Indicates industrial facilities The historical state; This represents the state update function, which integrates historical and new observation data.

[0237] The update operation employs a Bayesian fusion method, comprehensively considering both historical data and new observations:

[0238] ;

[0239] in Represents given observation data Industrial facilities The posterior probability of a state; Let the likelihood function be denoted as , representing the likelihood function in state . The following observations The probability of; Represents the prior probability of a state; This indicates a direct proportional relationship, and this represents the unnormalized Bayes formula.

[0240] Step 5.3, calibrate the parameters of the anomaly prediction model;

[0241] According to another embodiment of this application, the parameters of the anomaly prediction model are corrected based on newly acquired measured data:

[0242] ;

[0243] in Indicates the corrected model parameters; Indicates the model parameters before correction; Indicates the learning rate parameter; This indicates that the loss function is applied to the parameters. The gradient; This represents the loss function between predicted and observed values.

[0244] Parameter calibration employs an online learning approach to maintain the model's adaptability to environmental changes.

[0245] ;

[0246] in This represents the loss function between predicted and observed values. This indicates the anomaly coverage predicted by the model; Indicates the actual observed anomaly coverage; The square norm represents the sum of squared errors across all pixels; This indicates a parameter regularization term (such as L2 regularization) to prevent overfitting; This represents the regularization coefficient, which is used to weigh the loss term.

[0247] Step 5.4, optimize maintenance strategy;

[0248] According to one embodiment of this application, an industrial facility maintenance strategy is formulated based on abnormal coverage distribution and predicted trends:

[0249] Define the maintenance revenue function:

[0250] ;

[0251] in Indicates time Maintenance of industrial facilities The benefits; Indicates maintenance of post-industrial facilities Expected increase in power generation; Indicates electricity price; Indicates maintenance of industrial facilities The cost.

[0252] Calculation of expected increase in power generation:

[0253] ;

[0254] in Indicates maintenance of post-industrial facilities Expected increase in power generation; Indicates industrial facilities The rated power generation capacity; Indicates industrial facilities Conversion efficiency; Indicates anomaly coverage The relationship function between power generation efficiency and efficiency.

[0255] The optimal maintenance time and region are determined by solving the following optimization problem:

[0256] ;

[0257] in Represents decision variables, Indicates maintenance of industrial facilities , This indicates that maintenance will not be performed. This represents the summation over all industrial facilities; Indicates time Maintenance of industrial facilities The benefits.

[0258] Constraints include resource constraints and time constraints, specifically:

[0259] Resource constraints:

[0260] ;

[0261] in This represents the summation over all industrial facilities. Represent decision variables; To maintain the budget; Indicates maintenance of industrial facilities The cost;

[0262] Time constraints:

[0263] ;

[0264] in To maintain industrial facilities Time required Total available time Represents decision variables.

[0265] Step 5.5: Generate maintenance operation suggestions;

[0266] According to a preferred embodiment of this application, maintenance operation suggestions are generated based on the optimization results, including:

[0267] Maintenance schedule:

[0268] ;

[0269] in Indicates the maintenance schedule. , , They represent the 1st, 2nd, and 3rd respectively. The recommended maintenance numbers for each industrial facility and their corresponding recommended maintenance times are listed below. This represents the total number of industrial facilities recommended for maintenance.

[0270] Regional priority allocation:

[0271] ;

[0272] in This represents a list of region priority levels. , , They represent the 1st, 2nd, and 3rd respectively. Each region and its corresponding maintenance priority Indicates the first A region number or name, Indicates the region Maintenance priority (numerical value, the larger the value, the higher the priority). This represents the total number of regions.

[0273] Resource allocation plan:

[0274] ;

[0275] in Indicates the resource allocation plan. , , They represent the 1st, 2nd, and 3rd respectively. Each region and the quantity or type of resources allocated to it. Indicates allocation to a region Resources This represents the total number of regions.

[0276] The system presents maintenance recommendations in a visual format on the digital twin platform for operations and maintenance personnel to use as a reference for decision-making. It also records maintenance actions and their effects, providing data support for subsequent model optimization. Furthermore, this method shifts operations and maintenance decision-making from experience-driven to data-driven, improving resource utilization efficiency.

[0277] A computer storage medium includes a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, it is used to implement the above-described embodied intelligent AI inspection method based on digital twins.

[0278] Here, the present invention provides an implementation example:

[0279] According to embodiments of this application, the AI- and digital twin-based robotic twin inspection method has been successfully applied in Park A of Region B. Park A is located on the edge of the desert in Region B, experiencing more than 85 days of sandstorms annually, resulting in severe accumulation of abnormalities on equipment surfaces, which conventional inspection methods struggle to address. It should be noted that the application example is as follows:

[0280] Campus A covers an area of ​​approximately 2.5 square kilometers and contains a large amount of critical infrastructure distributed across 5 areas. The main challenges facing Campus A are:

[0281] Strong light interference: High local light intensity and strong reflection from the equipment surface can lead to a false alarm rate of up to 30% for conventional image recognition systems.

[0282] Uneven dust distribution: Sand dunes, topography, and obstructions cause large differences in abnormal accumulation rates in different areas of the park;

[0283] Complex terrain conditions: Sandy and gravelly surfaces cause high energy consumption and short battery life for robot movement, and conventional path planning is difficult to cover the entire field;

[0284] Passive inspection strategy: Lacking accurate change prediction models, it is difficult to grasp the timing of inspections, resulting in waste of resources.

[0285] Before applying this system, a dataset was constructed by collecting 3,000 multispectral images under different lighting conditions and environmental factors, and labeling various defects and states as the training set. It should be understood that in actual deployment:

[0286] The KUKA youBot mobile robot is equipped with a multi-sensor fusion platform, including a high-resolution RGB camera, a thermal imaging camera, a LiDAR, and a multispectral sensor, to acquire images in five bands (blue, green, red, red-edge, and near-infrared).

[0287] Actual measurements showed that under strong light conditions in Park A, near-infrared and red-edge band information can effectively distinguish different surface conditions and reflective areas;

[0288] The adaptive enhancement algorithm for illumination conditions dynamically adjusts parameters based on the day's illumination angle (obtained from a weather station) during processing. It employs a weighted combination method of visible and near-infrared bands, with a visible band weight of 0.3 and a near-infrared band weight of 0.7. Under local terrain conditions, a higher weight for the near-infrared band provides more stable results.

[0289] In actual operation, the attention mechanism of the dynamic attention network pays special attention to abnormal areas on the device surface, which improves the accuracy of state estimation at the boundary.

[0290] According to embodiments of this application, in campus A, the application of the physical constraint neural differential equation model mainly focuses on the following aspects:

[0291] Physical parameter settings: Based on local historical topographic data and environmental factor variation characteristics, initial physical parameters are set as follows:

[0292] The diffusion coefficient is 0.015 (determined based on the distribution of local environmental factors).

[0293] The settlement rate is 0.008 (under clear weather).

[0294] The topographic influence factor is dynamically adjusted according to different ground conditions.

[0295] Model training and validation: Two months of historical data were collected, with 80% used for training and 20% for validation. Physical consistency constraints improved the model's prediction accuracy by 18% in rare terrain conditions (such as after a sandstorm) compared to a purely data-driven model.

[0296] Evolutionary Prediction Case: 48 hours before a sandstorm warning, the system predicted the evolutionary trends of environmental changes in various areas of the park after the sandstorm, accurately identifying three risk "hotspots." These areas did indeed show significant changes in their condition after the sandstorm, validating the accuracy of the prediction.

[0297] According to one embodiment of this application, in a practical application within Park A:

[0298] Inspection priority calculation: The system sets the weight to the current anomaly coverage weight. The weight of the predicted abnormal coverage growth rate is 0.4. 0.5, historical fault frequency weight The value is 0.1, emphasizing predictive maintenance.

[0299] Terrain adaptation energy consumption model: Field measurements showed that under typical local terrain conditions, energy consumption for walking on sand increased by about 35%, and the terrain influence coefficient was obtained by fitting the measurement data.

[0300] Path optimization case study: In a standard inspection task, the system generated a path for the inspection period from 8:00 AM to 12:00 PM, with the following terrain conditions: sandy road surface, temperature 24℃. Compared to the traditional "serpentine" scanning path:

[0301] It covered the same number of inspection points but reduced the walking distance by 22%;

[0302] Battery efficiency improved by 26%;

[0303] The focus was on the southeastern region, where forecasts are expected to change rapidly.

[0304] Digital Twin Update: After the inspection is completed, the system sends the processing results of 85,000 collected images back to the digital twin platform, updates the facility status, and recommends maintenance times and area priorities.

[0305] High-priority areas (abnormal status or predicted significant changes within 3 days): Repair is recommended within 1 day;

[0306] Medium priority area (minor abnormality): Repair recommended within 3 days;

[0307] Low-priority areas (normal status and slow changes): It is recommended to reassess after 7 days;

[0308] Based on actual application data, this implementation method achieved the following technical results after being applied in Park A for 6 months:

[0309] The comparison data of detection accuracy are shown in Table 1:

[0310] Table 1: Comparison of Detection Accuracy Data

[0311]

[0312] Table 2 shows a comparison of system performance and maintenance effectiveness:

[0313] Table 2: Comparison Results of System Performance and Maintenance Effectiveness

[0314]

[0315] It should be noted that this implementation method demonstrated excellent performance, particularly in dealing with extreme terrain conditions. During the six-month application period, the system predicted high-risk areas 2-4 days in advance during three sandstorm warnings. The operations and maintenance team made targeted protective and maintenance preparations, resulting in a 61.9% reduction in equipment losses after sandstorms compared to the same period in previous years, significantly improving the park's resilience under extreme terrain conditions.

[0316] Furthermore, the system demonstrates significant self-optimization capabilities. Initially, the detection accuracy was 87%, and after six months of continuous learning and parameter calibration, the accuracy steadily increased to 94%, remaining stable under varying seasonal terrain conditions, thus proving the system's adaptability and sustainability.

[0317] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A digital twin-based embodied intelligent AI inspection method, characterized in that, include: By using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors on a robot, a real-time distribution map of abnormal coverage of industrial facilities is generated. The step of using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors on a robot and generating a real-time distribution map of abnormal coverage of industrial facilities includes: The robot uses a multispectral camera and other sensors to collect images of the facility's surface, including the visible and near-infrared bands. An adaptive enhancement algorithm based on illumination conditions is applied to preprocess multispectral images to reduce the interference of complex illumination conditions on detection accuracy; The preprocessed image is processed by a dynamic attention multi-layer feature extraction network, and the feature map weights are dynamically adjusted so that the model pays more attention to real abnormal areas and ignores areas with lighting interference. Features extracted from different convolutional layers are fused through a feature pyramid network to capture anomalous features at different granularities. Anomaly coverage of each pixel is calculated using a fully connected layer and a regression layer to generate anomaly coverage distribution map of industrial facilities. Based on the real-time distribution map of abnormal coverage of industrial facilities, a spatiotemporal sequence comparison learning model is used to analyze historical abnormal coverage data and generate an abnormal coverage trend prediction. Based on the prediction of anomalous coverage trends, the evolution process of anomalous coverage is analyzed using a physical constraint neural differential equation model, generating accurate anomalous evolution prediction results. Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, the optimal inspection path is generated using a multi-objective optimization algorithm. Based on multispectral images, real-time distribution maps of abnormal coverage of industrial facilities, prediction results of abnormal evolution, and real-time data during the inspection process, the digital twin system is dynamically updated, and suggestions for optimizing the maintenance of industrial facilities are generated.

2. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The steps of analyzing historical abnormal coverage data using a spatiotemporal sequence contrastive learning model to generate anomaly coverage trend predictions include: Collect abnormal coverage images of the same industrial facility at different time points, and combine them with terrain data to construct a time-series dataset; By using a spatiotemporal feature extraction network, the temporal evolution pattern of anomaly accumulation is captured; By applying a contrastive learning framework, the model learns the feature differences under different anomaly accumulation rates; By combining terrain data and historical accumulation patterns, an abnormal accumulation rate model is established. Based on the accumulation rate model, predict the future distribution of anomaly coverage.

3. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The steps for analyzing the anomalous coverage evolution process using a physically constrained neural differential equation model to generate accurate anomalous evolution prediction results include: The evolution of anomalous coverage can be expressed as a spatiotemporal partial differential equation: ； in Indicates position ,time Abnormal coverage; The partial derivative of coverage with respect to time represents the rate of change of coverage over time. Indicates time Topographical and environmental conditions; The spatial gradient representing coverage, , Represents the partial derivative operator; This represents the set of parameters to be learned. Represents the dynamic function to be learned, describing the physical and data-driven process of anomalous evolution; Constructing a hybrid structure that combines prior physical knowledge with neural networks: ； in Represents the overall dynamic function; Represents physical priors, based on fluid dynamics equations; Represents the residual term learned by the neural network; Construct a physical consistency constraint loss function to ensure that the model predictions conform to physical laws; An adaptive numerical solver is constructed to efficiently solve neural differential equations using a variable step-size algorithm with controllable error thresholds. Build an end-to-end training framework to jointly optimize physical parameters and neural network parameters.

4. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The steps for generating the optimal inspection path using a multi-objective optimization algorithm include: Based on the status information and anomaly prediction results of industrial facilities in the digital twin system, the inspection priority of industrial facilities is calculated. By taking real-time terrain data into account, a robot energy consumption prediction model under the influence of terrain is constructed. Based on facility priority, energy consumption prediction, path accessibility and safety factors, a multi-objective optimization problem is established to generate the optimal inspection path; During the inspection process, the inspection route is dynamically adjusted based on real-time environmental changes and the status of industrial facilities. The optimized inspection path is converted into robot motion control commands to enable autonomous movement and execution of inspection tasks.

5. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The steps of dynamically updating the digital twin system and generating maintenance optimization recommendations for industrial facilities include: The abnormal coverage distribution map of industrial facilities obtained by the robot inspection is transmitted back to the digital twin system in real time; Based on the processed data, update the status information of industrial facilities in the digital twin system; Based on the newly acquired measured data, the parameters of the anomaly prediction model are corrected; Develop industrial facility maintenance strategies based on abnormal coverage distribution and predicted trends; Based on the optimization results, maintenance operation suggestions are generated, including maintenance schedule, regional priority division and resource allocation scheme.

6. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The adaptive enhancement algorithm for lighting conditions is performed through the following steps: Estimate the light intensity and angle in the image: ； in Represents pixels Estimated light intensity at the location; Represents pixels Visible light band image intensity at that location; Represents pixels Near-infrared image intensity at the location; This represents the illumination estimation function, which is used to fuse visible light and near-infrared information and output the illumination intensity. Calculate the adaptive enhancement parameters: ； in Represents pixels Adaptive enhancement parameters at the location; This represents a parameter mapping function that dynamically adjusts the enhancement coefficient based on the local illumination intensity. Represents pixels Estimated light intensity at the location; Application adaptive enhancement: ； in Indicates the enhanced image in Pixel value at; Indicates the original image in Pixel value at; Represents pixels Adaptive enhancement parameters at the location.

7. The embodied intelligent AI inspection method based on digital twins according to claim 1, characterized in that, The core of the dynamic attention multi-layer feature extraction network is the calculation of the attention weight matrix: Attention weight calculation: ； in Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the attention mapping function, which maps the feature map to a weight distribution. This represents the sigmoid activation function; Weighted sum of feature maps and attention weights: ； in This represents the weighted feature map; Represents the attention weight matrix; This represents the feature map extracted by the convolutional layer; This represents the element-wise multiplication operator.

8. The embodied intelligent AI inspection method based on digital twins according to claim 2, characterized in that, The contrastive learning framework enables the model to learn feature differences under different anomaly accumulation rates; the loss function of contrastive learning is defined as: ； in This represents the contrastive learning loss function; This represents the feature representation of the current sample. Indicates and Paired positive sample feature representation, This represents the feature representation of other samples in the batch; Represents the cosine similarity function; It represents temperature parameters and controls the smoothness of the distribution; Indicates the sample index. , This represents the original number of samples within a batch, while This represents the total number of samples after considering data augmentation; Indicates an indicator function, when Time to take Otherwise ; Represents the exponentiation operator; This represents the natural logarithm operator.

9. A computer storage medium, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the embodied intelligent AI inspection method based on digital twins as described in any one of claims 1-8.

Images