Digital twinning-based intelligent AI inspection method with body

Abstract

The invention relates to the technical field of intelligent robot inspection, and discloses a digital twinning-based intelligent AI inspection method, which comprises the following steps of: analyzing a multispectral image acquired by a sensor carried by a robot by utilizing a dynamic attention multilayer feature extraction network; generating an industrial facility abnormal coverage degree real-time distribution map; analyzing historical abnormal coverage data by using a space-time sequence contrast learning model, and generating abnormal coverage trend prediction; analyzing an abnormal coverage evolution process by using a physical constraint neural differential equation model, and generating an accurate abnormal evolution prediction result; generating an optimal inspection path by using a multi-objective optimization algorithm; dynamically updating the digital twin system, and generating an industrial facility maintenance optimization suggestion; according to the invention, through the dynamic attention multilayer feature extraction network and the illumination condition adaptive enhancement algorithm, the detection problem in the strong light and high reflection environment is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent robot inspection technology, and more specifically, to an embodied intelligent AI inspection method based on digital twins. Background Art

[0002] With the advancement of industrial automation and intelligence, intelligent O&M of large-scale facilities and infrastructure has become a key component in improving operational efficiency and reducing O&M costs. In complex environments, target facilities often face unique challenges, including abnormal surface conditions and complex and changing environmental conditions. Abnormal surface conditions are a dynamic process, influenced by multiple factors, including the environment, terrain, and geographic location, severely impacting the normal operation of the facility. Furthermore, complex lighting and highly reflective environments interfere with the detection accuracy of traditional robotic inspection systems, while variable terrain and environmental conditions further complicate the inspection task.

[0003] Existing robotic inspection technology has the following major problems: First, the accuracy of traditional image recognition technology in detecting surface anomalies of target objects under complex lighting conditions still needs to be improved. The existing image recognition algorithm's ability to distinguish between strong light reflection areas and abnormal areas needs to be further enhanced. In certain special and complex environments, the false alarm rate is still high, affecting the reliability of maintenance decisions. Second, mainstream detection methods mainly focus on static state detection, and their ability to predict the time evolution trend of abnormal states is limited, resulting in relatively passive maintenance strategies. Existing technologies mainly obtain abnormal state information at the current moment, and the prediction accuracy of future abnormal change trends is insufficient, making it difficult for operation and maintenance teams to formulate accurate maintenance plans in advance. Third, current prediction models still have shortcomings in terms of physical process constraints. Accurately describing the dynamic evolution of abnormal states in the case of sparse data is challenging, especially in rare environmental conditions. The prediction ability needs to be improved. The accuracy of most data-driven prediction models is limited when data is scarce or when facing new environmental conditions. Finally, although intelligent path planning technology has made progress, there are still efficiency issues in optimizing inspection paths in specific complex environments. Existing technologies do not comprehensively consider environmental factors, abnormal states of targets, and predicted trends in path planning, making it difficult to achieve accurate coverage of high-risk areas within a limited time, resulting in less than ideal efficiency in inspection resource allocation.

[0004] To sum up, there is an urgent need for a robot twin inspection system that can adapt to complex environments, has predictive capabilities, intelligent path optimization, and supports closed-loop optimization to improve facility operation and maintenance efficiency and economic benefits. Summary of the Invention

[0005] The present invention provides an embodied intelligent AI inspection method based on digital twins, which solves the technical problems in related technologies such as low accuracy of abnormal coverage detection under complex lighting conditions, inability of static coverage detection to predict the accumulation trend of abnormal states, limited generalization ability of abnormal prediction models, and inefficient inspection path planning of traditional robot inspection systems.

[0006] The present invention provides an embodied intelligent AI inspection method based on digital twins, comprising: A dynamic attention multi-layer feature extraction network is used to analyze multispectral images collected by sensors onboard robots to generate a real-time distribution map of abnormal coverage of industrial facilities. Based on the real-time distribution map of abnormal coverage of industrial facilities, the spatiotemporal sequence comparative learning model is used to analyze historical abnormal coverage data and generate abnormal coverage trend forecasts; Based on the prediction of abnormal coverage trend, the physical constraint neural differential equation model is used to analyze the abnormal coverage evolution process and generate accurate abnormal evolution prediction results; Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, a multi-objective optimization algorithm is used to generate the optimal inspection path; Based on multispectral images, real-time distribution maps of industrial facility anomaly coverage, anomaly evolution prediction results and real-time data during inspection execution, the digital twin system is dynamically updated and optimization suggestions for industrial facility maintenance are generated.

[0007] Furthermore, the step of using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors mounted on the robot to generate a real-time distribution map of abnormal coverage of industrial facilities includes: Use multispectral cameras and other sensors onboard the robot to collect images of the facility's surface, including visible and near-infrared wavelengths; Applying the adaptive illumination enhancement algorithm to pre-process the multispectral image to reduce the interference of complex illumination conditions on detection accuracy; The pre-processed image is processed by a dynamic attention multi-layer feature extraction network, and the feature map weights are dynamically adjusted to make the model pay more attention to the real abnormal areas and ignore the light interference areas; The features extracted by different convolutional layers are fused through the feature pyramid network to capture abnormal features of different granularities; The abnormal coverage of each pixel is calculated through the fully connected layer and regression layer to generate the abnormal coverage distribution map of industrial facilities.

[0008] Furthermore, the step of analyzing historical abnormal coverage data using the spatiotemporal sequence contrast learning model to generate abnormal coverage trend predictions includes: Collect abnormal coverage images of the same industrial facility at different time points, combine them with terrain data, and construct a time series dataset; Capturing the temporal evolution pattern of anomaly accumulation through spatiotemporal feature extraction network; Applying a contrastive learning framework, the model learns the feature differences under different anomaly accumulation rates; Combining topographic data with historical accumulation patterns to develop anomaly accumulation rate models; Based on the accumulation rate model, the future anomaly coverage distribution is predicted.

[0009] Furthermore, the step of analyzing the abnormal coverage evolution process using the physical constraint neural differential equation model to generate accurate abnormal evolution prediction results includes: The anomaly cover evolution is expressed as a spatiotemporal partial differential equation: ; in Indicates location ,time Abnormal coverage of represents the partial derivative of coverage with respect to time, indicating the rate of change of coverage over time; Indicates time Topographic and environmental conditions; represents the spatial gradient of coverage, , represents the partial derivative operator; represents the set of parameters to be learned; Represents the dynamic function to be learned, describing the physical and data-driven process of anomaly evolution; Build hybrid structures that combine physical prior knowledge with neural networks: ; in represents the total kinetic function; represents the physical prior term, based on the fluid dynamics equation; Represents the residual term of neural network learning; Construct a physical consistency constraint loss function to ensure that the model predictions conform to physical laws; Build an adaptive numerical solver to efficiently solve neural differential equations using a variable step-size algorithm with controllable error thresholds; Build an end-to-end training framework to jointly optimize physical parameters and neural network parameters.

[0010] Furthermore, the step of generating the optimal inspection path using a multi-objective optimization algorithm includes: Calculate the inspection priority of industrial facilities based on the status information and abnormality prediction results of industrial facilities in the digital twin system; By considering real-time terrain data, a robot energy consumption prediction model under the influence of terrain is constructed; Based on facility priority, energy consumption prediction, path accessibility, and safety factors, a multi-objective optimization problem is established to generate the optimal inspection path; During the inspection process, the inspection route is dynamically adjusted based on real-time environmental changes and industrial facility status; The optimized inspection path is converted into robot motion control instructions to achieve autonomous movement and perform inspection tasks.

[0011] Furthermore, the steps of dynamically updating the digital twin system and generating industrial facility maintenance optimization suggestions include: Transmit data such as abnormal coverage distribution maps of industrial facilities obtained by robot inspections to the digital twin system in real time; Based on the processed data, the status information of industrial facilities in the digital twin system is updated; Correct the anomaly prediction model parameters based on the newly acquired measured data; Formulate maintenance strategies for industrial facilities based on abnormal coverage distribution and predicted trends; Based on the optimization results, maintenance action recommendations are generated, including maintenance schedule, area prioritization, and resource allocation plans.

[0012] Furthermore, the illumination condition adaptive enhancement algorithm is performed by the following steps: Estimate lighting intensity and angle in an image: ; in Represents pixel points Estimated light intensity at ; Represents pixel points The visible light band image intensity at ; Represents pixel points The near-infrared band image intensity at ; Represents the illumination estimation function, which is used to fuse visible light and near-infrared information and output light intensity; Calculate the adaptive enhancement parameters: ; in Represents pixel points Adaptive enhancement parameters at ; Represents a parameter mapping function that dynamically adjusts the enhancement coefficient according to the local light intensity; Represents pixel points Estimated light intensity at ; Apply adaptive enhancements: ; in The enhanced image is The pixel value at ; Indicates that the original image is The pixel value at ; Represents pixel points Adaptive enhancement parameters at .

[0013] Furthermore, the core of the dynamic attention multi-layer feature extraction network is the calculation of the attention weight matrix: Attention weight calculation: ; in represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; represents the attention mapping function, which maps the feature map to the weight distribution; Represents the sigmoid activation function; Weighting of feature maps and attention weights: ; in Represents the weighted feature map; represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; Represents the element-wise multiplication operator.

[0014] Furthermore, the contrastive learning framework enables the model to learn the feature differences under different anomaly accumulation rates; the loss function of contrastive learning is defined as: ; in represents the contrastive learning loss function; represents the feature representation of the current sample, Represents Paired positive sample feature representation, Represents the feature representation of other samples in the batch; represents the cosine similarity function; represents the temperature parameter, which controls the smoothness of the distribution; represents the sample index, , represents the original number of samples in the batch, and Indicates the total number of samples after considering data enhancement; represents the indicator function, when Time , otherwise ; Represents the exponential operator; Represents the natural logarithm operator.

[0015] The present invention provides a computer storage medium comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned embodied intelligent AI inspection method based on digital twins.

[0016] The beneficial effect of this invention lies in: through a dynamic attention multi-layer feature extraction network and an adaptive illumination enhancement algorithm, it effectively solves the detection problem in strong light and highly reflective environments. Detection accuracy is improved in strong light conditions, areas with drastic lighting changes, and areas with abnormal boundaries, providing a more reliable data foundation for maintenance decisions.

[0017] By integrating spatiotemporal sequence contrast learning with a physically constrained neural differential equation model, the system can predict the accumulation trend of abnormal conditions in advance, improving prediction accuracy and providing sufficient response time for operation and maintenance decisions. By combining physical prior knowledge with a neural network, the model maintains reliable prediction performance even in data-sparse and rare terrain environments, and reduces prediction errors when faced with unseen terrain conditions. Based on multi-objective optimized path planning based on industrial facility priorities and terrain influences, the system reduces mileage, improves coverage efficiency, and increases battery usage efficiency. The coverage area of ​​a single inspection is increased, thereby improving inspection efficiency. Through accurate anomaly prediction and prioritization, the maintenance cost of industrial facilities is reduced, the average annual power generation is increased, and the return on investment is improved; The dynamic update of the digital twin system and the correction of model parameters form a closed-loop optimization mechanism. The accuracy of the system model will increase every month, and after long-term operation, it will maintain a stable high accuracy level, proving the system's self-learning and continuous optimization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of an embodied intelligent AI inspection method based on digital twins in the present invention; Figure 2 is a flow chart of step 1 in the present invention; Figure 3 It is a flow chart of step 2 in the present invention; Figure 4 It is a flow chart of step 3 in the present invention; Figure 5 It is a flow chart of step 4 in the present invention; Figure 6 It is a flow chart of step 5 in the present invention. DETAILED DESCRIPTION

[0019] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0020] At least one embodiment of the present invention discloses an embodied intelligent AI inspection method based on digital twins, such as Figures 1 to 6 Shown, including: Step 1: Use a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by the robot's onboard sensors to generate a real-time distribution map of abnormal coverage of industrial facilities; This step uses a dynamic attention multi-layer feature extraction network to analyze the multispectral images collected by the robot's onboard sensors to generate a real-time distribution map of abnormal coverage of industrial facilities. It should be noted that the specific steps include: Step 1.1, collecting multispectral image data; The robot uses a multispectral camera and other sensors to capture surface images of facilities, including visible and near-infrared wavelengths, generating multi-channel image data of the industrial facility. The captured image data includes spectral information on abnormally covered areas, reflective areas, and normal areas on the surface of the industrial facility.

[0021] The abnormal coverage areas on the surface of industrial facilities refer to various abnormal state areas on the surface of industrial facilities, including: Surface accumulation coverage area: surface coverage caused by natural environmental factors such as sand, dust, and debris; surface pollution caused by production environmental factors such as industrial dust and oil pollution; Material deterioration areas: surface corrosion and oxidation areas; material aging and weathering areas; material discoloration areas affected by environmental factors; Structural anomalies: surface cracks and damage; deformations and depressions; areas with loose or missing connectors; Functional abnormality areas: abnormal temperature hotspot areas; abnormal humidity areas; electrical insulation failure areas; These abnormal coverage areas will show spectral characteristics different from normal areas in multispectral images.

[0022] Step 1.2, apply the illumination condition adaptive enhancement algorithm; The illumination condition adaptive enhancement algorithm is applied to preprocess the multispectral image to reduce the interference of complex illumination conditions on detection accuracy. The algorithm is carried out through the following steps: First, estimate the lighting intensity and angle in the image: ; in Represents pixel points Estimated light intensity at ; Represents pixel points The visible light band image intensity at ; Represents pixel points The near-infrared band image intensity at ; Represents the illumination estimation function, which is used to fuse visible light and near-infrared information and output light intensity.

[0023] Then, the adaptive enhancement parameters are calculated: ; in Represents pixel points Adaptive enhancement parameters at ; Represents a parameter mapping function that dynamically adjusts the enhancement coefficient according to the local light intensity; Represents pixel points Estimated light intensity at .

[0024] Finally, apply adaptive augmentation: ; in The enhanced image is The pixel value at ; Indicates that the original image is The pixel value at ; Represents pixel points Adaptive enhancement parameters at .

[0025] Step 1.3, process the image through a dynamic attention multi-layer feature extraction network; The pre-processed image is processed by a dynamic attention multi-layer feature extraction network, which dynamically adjusts the feature map weights so that the model pays more attention to the real abnormal areas and ignores the light interference areas. The core of the dynamic attention model is the calculation of the attention weight matrix: ; in represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; represents the attention mapping function, which maps the feature map to the weight distribution; Represents the sigmoid activation function.

[0026] The weighted result of the feature map and attention weight is: ; in Represents the weighted feature map; represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; Represents the element-wise multiplication operator.

[0027] Step 1.4, perform multi-scale feature fusion; The features extracted by different convolutional layers are fused through the feature pyramid network to capture abnormal features of different granularities: ; in Indicates the Feature pyramid output of the layer; Represents the upsampling operation, which upsamples the high-level feature map to the current layer resolution; Indicates the Feature pyramid output of the layer; Indicates the lateral connectivity characteristics of the layers; Represents the convolution operation, which is used for feature fusion.

[0028] Step 1.5, calculate the abnormal coverage distribution map; Finally, the anomaly coverage of each pixel is calculated through the fully connected layer and the regression layer to generate the anomaly coverage distribution map of industrial facilities: ; in Represents pixel points The abnormal coverage at the , Indicates no abnormality. Indicates complete exception coverage; Indicates that the fused feature map is The eigenvector at ; Represents the coverage mapping function, usually a fully connected layer plus regression activation.

[0029] Step 2: Based on the real-time distribution map of abnormal coverage of industrial facilities, the historical abnormal coverage data is analyzed using the spatiotemporal sequence comparative learning model to generate an abnormal coverage trend forecast; According to one embodiment of the present application, this step uses a spatiotemporal sequence contrast learning model to analyze historical abnormal coverage data and generate abnormal coverage trend predictions. It should be understood that this specifically includes: Step 2.1, construct a time series dataset; Collect abnormal coverage images of the same industrial facility at different time points, combine them with terrain data, and construct a time series dataset. Each time series sample contains: ; in Indicates the A time series sample set of industrial facilities; 、 、 Respectively represent Industrial facilities in 、 、 Abnormal coverage image at each time point; 、 、 Respectively represent Industrial facilities in 、 、 Topographic data corresponding to each time point; 、 、 Respectively represent Industrial facilities in 、 、 The timestamp of a time point; Indicates the length of the time series.

[0030] Step 2.2, extract spatiotemporal features; The temporal evolution pattern of anomaly accumulation is captured through the spatiotemporal feature extraction network. The network combines the temporal convolution module and the spatial convolution module to extract spatiotemporal joint features: Spatial feature extraction: ; in Indicates time spatial characteristics; Representation space multi-layer feature extraction network; Indicates time Abnormal coverage distribution.

[0031] Temporal feature extraction: ; in Indicates location Temporal characteristics of the location; represents a temporal convolutional network; Indicates location Time series data at; middle Represents the tensor slice symbol, which means taking out the sequence data of the spatial position at all time steps.

[0032] Spatiotemporal joint features: ; in Represents the spatiotemporal joint features (the fused feature vector); Represents the feature splicing operation, that is, spatial features and time characteristics Concatenate on the feature dimension to form a new joint feature vector; Represents spatial features; Represents time characteristics.

[0033] Step 2.3, training the contrastive learning model; The contrastive learning framework is applied to enable the model to learn the feature differences under different anomaly accumulation rates. The loss function of contrastive learning is defined as: ; in represents the contrastive learning loss function; represents the feature representation of the current sample, Represents Paired positive sample feature representation, Represents the feature representation of other samples in the batch; represents the cosine similarity function; represents the temperature parameter, which controls the smoothness of the distribution; represents the sample index, , represents the original number of samples in the batch, and Indicates the total number of samples after considering data enhancement; represents the indicator function, when Time , otherwise ; Represents the exponential operator; Represents the natural logarithm operator.

[0034] Step 2.4, establish an abnormal accumulation rate model; Combining topographic data and historical accumulation patterns, a model of abnormal accumulation rates was established: ; in Indicates the location ,time and terrain and environmental conditions Abnormal accumulation rate under (rate of change of coverage per unit area per unit time); Indicates location ,time The spatiotemporal characteristics of the location; Indicates time Terrain and environmental conditions (vector); Represents the rate mapping function.

[0035] Step 2.5, predict abnormal coverage trend; Based on the accumulation rate model, predict future anomaly coverage: ; in Represents the predicted future time In position Abnormal coverage at ; Indicates the current time exist Abnormal coverage at ; Indicates time ,Location , topographic and environmental conditions Abnormal accumulation rate under ; Indicates time In the interval The integral on represents the cumulative abnormal change; Represents the forecast time step.

[0036] This formula can be approximately solved by numerical integration method.

[0037] Step 3: Based on the abnormal coverage trend prediction, the physical constraint neural differential equation model is used to analyze the abnormal coverage evolution process and generate accurate abnormal evolution prediction results; According to another embodiment of the present application, this step uses a physical constraint neural differential equation model to analyze the abnormal coverage evolution process and generate accurate abnormal evolution prediction results. In addition, it specifically includes: Step 3.1, construct the anomaly coverage evolution equation; The anomaly cover evolution is expressed as a spatiotemporal partial differential equation: ; in Indicates location ,time Abnormal coverage of represents the partial derivative of coverage with respect to time, indicating the rate of change of coverage over time; Indicates time Topographic and environmental conditions; represents the spatial gradient of coverage, , represents the partial derivative operator; represents the set of parameters to be learned; Represents the dynamic function to be learned, describing the physical and data-driven process of anomaly evolution.

[0038] Step 3.2, construct a hybrid structure of physical prior knowledge and neural network; According to one embodiment of the present application, a hybrid structure is constructed to combine physical prior knowledge with a neural network: ; in represents the total kinetic function; represents the physical prior term, based on the fluid dynamics equation; Represents the residual term learned by the neural network.

[0039] The physical items are: ; in represents the physical prior term, represents the convection term, is the divergence operator, is the slope gradient vector, is coverage; represents the diffusion coefficient, which indicates the abnormal diffusion capacity; represents the Laplace operator, , indicating spatial diffusion; It represents the abnormal source item, indicating the process of generating or eliminating the abnormality.

[0040] The neural network terms are: ; in Represents the residual term of neural network learning; Indicates that the parameter is neural networks; Indicates the anomaly coverage at the current location and time; Indicates terrain and environmental conditions; Represents the spatial gradient of anomaly coverage.

[0041] The neural network is used to learn complex nonlinear residual dynamics that are not captured by the physical model, thereby improving the prediction accuracy of the overall model.

[0042] Step 3.3, construct the physical consistency constraint loss function; To ensure that the model predictions conform to physical laws, according to one embodiment of the present application, a physical consistency constraint loss function is constructed: ; in Indicates loss of physical consistency; represents the square norm, which represents the sum of squared errors of all spatial points; is the slope gradient vector, is coverage; represents the diffusion coefficient, which indicates the abnormal diffusion capacity; represents the Laplace operator; It represents the abnormal source item, indicating the process of generating or eliminating the abnormality.

[0043] The total loss function includes data fitting loss and physical consistency loss: ; in represents the total loss function; represents the data fitting loss; Represents the trade-off coefficient, which controls the weight of physical loss in the total loss.

[0044] Step 3.4, construct an adaptive numerical solver; According to one embodiment of the present application, an adaptive numerical solver is constructed to efficiently solve neural differential equations using a variable step size algorithm with a controllable error threshold: The variable step size algorithm dynamically adjusts the integration step size based on the local error estimate: ; in Indicates the The time step of the step; Indicates the The time step of the step; Indicates the The local error estimate of the step; Indicates the error tolerance threshold (preset constant); Indicates the order of the numerical integration method; express power root.

[0045] when When , reduce the step size and recalculate; when When , increase the step size to improve efficiency.

[0046] Step 3.5, end-to-end training and prediction; According to another embodiment of the present application, an end-to-end training framework is constructed to jointly optimize physical parameters and neural network parameters: For each training sample, the prediction result is obtained by solving the initial value problem: ; in 、 、 Respectively The abnormal coverage at the time, the terrain and environmental conditions, and the spatial gradient of abnormal coverage; Represents the predicted future moment Abnormal coverage of Indicates the current time Abnormal coverage of represents the dynamics function; represents the integral variable, represents time; Express 's points; Indicates The integral of the interval.

[0047] Calculate the error between the predicted result and the actual coverage, and backpropagate to update the parameters: ; in Represents model parameters (including neural network weights and physical parameters); Represents the learning rate, which controls the parameter update step; Express The total loss function gradient of Represents the assignment operator, indicating parameter update.

[0048] After the model training is completed, the abnormal coverage evolution at any future moment can be predicted. Especially in data-sparse and rare terrain environments, it can still maintain reliable prediction performance and provide a scientific basis for subsequent maintenance decisions.

[0049] Step 4: Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, a multi-objective optimization algorithm is used to generate the optimal inspection path; According to a preferred embodiment of the present application, this step utilizes a multi-objective optimization algorithm to combine the industrial facility status information and abnormality prediction results in the digital twin system to generate the optimal inspection path. It can be understood that it specifically includes: Step 4.1, calculate the priority of industrial facilities; Calculate the inspection priority of industrial facilities based on the status information and abnormality prediction results of industrial facilities in the digital twin system: ; in Indicates industrial facilities Inspection priority (numeric value, the larger the value, the higher the priority); 、 、 They represent the weights of the current anomaly coverage, the predicted anomaly coverage growth rate, and the historical fault frequency respectively; Indicates industrial facilities Current anomaly coverage; Indicates industrial facilities Projected abnormal coverage growth rate; Indicates industrial facilities The historical failure frequency.

[0050] Step 4.2, construct an energy consumption prediction model under the influence of terrain; According to one embodiment of the present application, a robot energy consumption prediction model under the influence of terrain is constructed by considering real-time terrain data: ; in Indicates terrain and environmental conditions Next, from position Exercise to energy consumption; Indicates ideal flat ground conditions arrive Basic energy consumption; represents the terrain influence function, which is defined as: ; in Indicates the upslope / downslope influence coefficient; Indicates the ground roughness coefficient; represents the slope gradient vector; Indicates from arrive The displacement vector of Indicates the modulus of the displacement vector.

[0051] Step 4.3, establish and solve the multi-objective optimization problem; According to another embodiment of the present application, a multi-objective optimization problem is established based on facility priority, energy consumption prediction, path accessibility, and safety factors to generate the optimal inspection path: Objective function: ; in Indicates the robot inspection path; Indicates the priority target weight coefficient; Indicates the energy consumption target weight coefficient; Indicates that all industrial facilities in the path sum; Indicates industrial facilities priority; Represents all adjacent node pairs in the path sum; Indicates from arrive energy consumption; Indicates maximization.

[0052] Constraints include battery energy constraints, motion time constraints, and safety distance constraints: Battery energy constraints: ,in is the maximum available energy of the robot; Movement time constraints: , For exercise time, is the maximum allowed exercise time; Safety distance constraints: , Path and obstacles The minimum distance, is the safety threshold.

[0053] The optimization adopts the improved differential evolution algorithm, the specific steps are: Initialize the candidate path population: ; in 、 、 Respectively represent Article 1, Article 2 to Article 3 candidate paths, is the population size, i.e. the total number of candidate paths; Perform mutation and crossover operations on each candidate path to generate a new path; Evaluate the fitness of the new path (i.e., the objective function value) and retain the excellent path; Repeat step 2.3 until convergence or the maximum number of iterations is reached.

[0054] Step 4.4, implement dynamic path adjustment; According to a preferred embodiment of the present application, during the inspection process, the inspection path is dynamically adjusted according to the real-time acquired environmental changes and industrial facility status: ; in Indicates the adjusted inspection path; Indicates the original inspection path; Indicates the amount of change in terrain and environmental conditions; Indicates the amount of change in the priority of industrial facilities; Represents a path adjustment function that replans the path based on input changes.

[0055] If environmental changes lead to increased security risks or changes in priorities, subsequent path segments are recalculated.

[0056] Step 4.5, realize autonomous motion control of the robot; According to another embodiment of the present application, the optimized inspection path is converted into robot motion control instructions to achieve autonomous movement and perform inspection tasks: ; in Indicates time The robot control signal; Indicates time The robot's current position; Indicates time Target position (coordinate vector); Indicates time Topographic and environmental conditions; Represents a motion controller function that performs motion compensation incorporating terrain effects.

[0057] The control system can effectively cope with the challenges brought by the complex terrain and environmental conditions in area B, ensuring that the robot moves accurately according to the planned path and completes the image acquisition task.

[0058] Step 5: Based on the multispectral imagery, the real-time distribution map of industrial facility anomaly coverage, the anomaly evolution prediction results, and the real-time data during the inspection execution process, the digital twin system is dynamically updated and optimization recommendations for industrial facility maintenance are generated; According to another embodiment of the present application, this step uses the real-time data obtained from the inspection to update the digital twin system and generate industrial facility maintenance optimization suggestions. It should be noted that the specific steps include: Step 5.1, real-time data transmission and processing; According to one embodiment of the present application, data such as the abnormal coverage distribution map of industrial facilities obtained by robot inspection is transmitted back to the digital twin system in real time: ; in Indicates the data set that is returned in real time; Indicates industrial facilities In time Observation anomaly coverage; Indicates industrial facilities In time Multispectral image data; Indicates time Measured terrain environment parameters.

[0059] Perform quality inspection and preprocessing on the returned data, remove outliers and fill in missing values: ; in represents the processed dataset; Represents data quality control functions, including outlier detection, missing value filling, etc.; Indicates the dataset that is returned in real time.

[0060] Step 5.2, update the digital twin model status; According to another embodiment of the present application, the status information of the industrial facilities in the digital twin system is updated based on the processed data: ; in Indicates industrial facilities The new state; Indicates industrial facilities the historical status of Represents the state update function, fusing historical and new observation data.

[0061] The update operation uses a Bayesian fusion method, taking into account the historical state and new observation data: ; in Represents a given observation data Lower industrial facilities the posterior probability of the state; Represents the likelihood function, which means that in state Observed probability; represents the state prior probability; Represents a proportional relationship and represents the unnormalized Bayesian formula.

[0062] Step 5.3, calibrate the anomaly prediction model parameters; According to another embodiment of the present application, the anomaly prediction model parameters are corrected based on the newly acquired measured data: ; in represents the model parameters after correction; represents the model parameters before correction; represents the parameter learning rate; Represents the loss function for parameters gradient; Represents the loss function between predicted values ​​and observed values.

[0063] Parameter correction uses online learning to keep the model adaptable to environmental changes: ; in represents the loss function between the predicted value and the observed value, represents the anomaly coverage predicted by the model; represents the abnormal coverage of actual observations; represents the square norm, which represents the sum of square errors of all pixels; Represents parameter regularization terms (such as L2 regularization) to prevent overfitting; Represents the regularization coefficient, which weighs the loss term.

[0064] Step 5.4, optimize maintenance strategy; According to one embodiment of the present application, an industrial facility maintenance strategy is formulated based on abnormal coverage distribution and predicted trends: Define the maintenance benefit function: ; in Indicates time Maintaining industrial facilities income; Indicates maintenance of post-industrial facilities expected increase in electricity generation; Indicates the electricity price; Indicates maintenance of industrial facilities cost.

[0065] Calculation of expected power generation increase: ; in Indicates maintenance of post-industrial facilities expected increase in electricity generation; Indicates industrial facilities Rated generating capacity; Indicates industrial facilities conversion efficiency; Indicates abnormal coverage The relationship function with power generation efficiency.

[0066] The optimal maintenance time and area are determined by solving the following optimization problem: ; in represents the decision variable, Indicates maintenance of industrial facilities , Indicates no maintenance; represents the sum of all industrial facilities; Indicates time Maintaining industrial facilities of income.

[0067] Constraints include resource constraints and time constraints, specifically: Resource constraints: ; in represents the sum of all industrial facilities, represents the decision variable; budget for maintenance; Indicates maintenance of industrial facilities Cost; Time constraints: ; in To maintain industrial facilities Time required, is the total available time, represents the decision variable.

[0068] Step 5.5, generating maintenance action recommendations; According to a preferred embodiment of the present application, a maintenance operation suggestion is generated based on the optimization result, including: Maintenance schedule: ; in Indicates the maintenance schedule, 、 、 Represents the first, second, and The industrial facility number recommended for maintenance and its corresponding recommended maintenance time, The total number of industrial facilities recommended for maintenance.

[0069] Regional prioritization: ; in Represents a regional priority list, 、 、 Represents the first, second, and areas and their corresponding maintenance priorities, Indicates the area number or name, Indicates area Maintenance priority (numeric value, the larger the higher the priority), The total number of regions.

[0070] Resource allocation plan: ; in represents the resource allocation plan, 、 、 Represents the first, second, and regions and the amount or type of resources allocated to them, Indicates the area assigned to resources, The total number of regions.

[0071] The system visualizes maintenance recommendations on the digital twin platform for operators to use as a reference for decision-making. It also records maintenance actions and their effects, providing data support for subsequent model optimization. Furthermore, this approach shifts operational decisions from being experience-driven to being data-driven, improving resource utilization efficiency.

[0072] A computer storage medium includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-mentioned embodied intelligent AI inspection method based on digital twins.

[0073] Here, the present invention provides an implementation example: According to the embodiments of this application, the robot twin inspection method based on AI and digital twins has been successfully applied in Park A in Region B. Park A is located on the edge of the desert in Region B, with an average annual dust storm of more than 85 days. The accumulation of abnormalities on the equipment surface is serious, and conventional inspection methods are difficult to deal with. It should be noted that the application examples are as follows: Park A covers an area of ​​approximately 2.5 square kilometers and contains a large amount of critical infrastructure distributed across five areas. The main challenges facing Park A are: Strong light interference: The local light intensity is high and the device surface reflects strongly, causing the conventional image recognition system to have a false alarm rate of up to 30%; Uneven dust distribution: Sand dunes, terrain, and obstructions cause large variations in abnormal accumulation rates in different areas of the park; Complex terrain conditions: Sandy and gravel roads cause the robot to consume high energy and have short battery life, and conventional path planning makes it difficult to cover the entire terrain. Passive inspection strategy: Lack of accurate change prediction model makes it difficult to grasp the inspection timing, resulting in waste of resources.

[0074] Before applying the system, a dataset was constructed. 3,000 multispectral images under different lighting conditions and environmental factors were collected, and various defects and conditions were annotated as training sets. It should be understood that in actual deployment: Use a KUKA youBot mobile robot equipped with a multi-sensor fusion platform, including a high-resolution RGB camera, a thermal imaging camera, a lidar, and a multispectral sensor to collect images in five bands (blue, green, red, red edge, and near-infrared); The actual measurement found that under strong light conditions in Park A, the near-infrared band and red-edge band information can effectively distinguish different surface states and reflective areas; The adaptive illumination enhancement algorithm dynamically adjusts parameters based on the day's sunlight angle (obtained from the weather station) during processing. It uses a weighted combination of visible and near-infrared bands, with a weight of 0.3 for the visible light band and 0.7 for the near-infrared band. Under local terrain conditions, a higher weight for the near-infrared band provides more stable results.

[0075] In actual operation, the dynamic attention network pays special attention to abnormal areas on the surface of the device, improving the accuracy of state estimation at the boundaries.

[0076] According to the embodiments of the present application, in Park A, the application of the physical constraint neural differential equation model mainly focuses on the following aspects: Physical parameter setting: Based on local historical terrain data and the changing characteristics of environmental factors, set the initial physical parameters: The diffusion coefficient is 0.015 (determined based on the distribution of local environmental factors); The sedimentation rate is 0.008 (in clear weather); The terrain impact factor is dynamically adjusted according to different ground conditions.

[0077] Model training and validation: Two months of historical data were collected, with 80% used for training and 20% for validation. Physical consistency constraints improved the model's prediction accuracy by 18% in rare terrain conditions (such as after a sandstorm) compared to a purely data-driven model.

[0078] Evolutionary prediction example: 48 hours before a sandstorm warning, the system predicted the evolutionary trends of environmental changes in various areas of the park following the sandstorm, accurately identifying three risk "hotspots." These areas did indeed experience significant changes in their conditions after the sandstorm, validating the accuracy of the prediction.

[0079] According to one embodiment of the present application, in actual application of Park A: Inspection priority calculation: The system sets the weight to the current abnormal coverage weight The predicted abnormal coverage growth rate weight is 0.4. 0.5, historical fault frequency weight It is 0.1, focusing on predictive maintenance.

[0080] Terrain adaptation energy consumption model: Actual measurements found that under local typical terrain conditions, energy consumption for walking on sand increased by about 35%. The terrain influence coefficient was obtained by fitting the measured data.

[0081] Path optimization example: During a standard inspection task, the system generated a path for the inspection period from 8:00 AM to 12:00 PM on a sandy road surface with a temperature of 24°C. Compared with the traditional "snake" scanning path, the following were achieved: Covering the same number of inspection points but reducing walking distance by 22%; Battery efficiency increased by 26%; Special attention was paid to the southeastern region where faster changes were predicted.

[0082] Digital Twin Update: After the inspection is complete, the system processes the 85,000 images collected and transmits them back to the digital twin platform, updating the facility status and recommending maintenance times and regional priorities. High-priority areas (abnormal status or predicted to change significantly within 3 days): maintenance is recommended within 1 day; Medium priority area (minor abnormality): It is recommended to repair within 3 days; Low-priority areas (normal status and slow changes): It is recommended to reassess after 7 days; According to actual application data, this implementation method has achieved technical results after 6 months of application in Park A: The comparison data of detection accuracy is shown in Table 1: Table 1: Comparison of detection accuracy data

[0083] The comparison of system performance and maintenance effect is shown in Table 2: Table 2: Comparison of system performance and maintenance effect

[0084] It's worth noting that this implementation demonstrates exceptional performance, particularly in addressing extreme terrain conditions. During its six-month application period, the system issued three sandstorm warnings, identifying high-risk areas 2-4 days in advance. The operations team implemented targeted protective and maintenance measures, resulting in a 61.9% reduction in equipment losses following sandstorms compared to the same period in history. This significantly enhances the park's resilience in extreme terrain conditions.

[0085] Furthermore, the system demonstrates significant self-optimization. Initially, its detection accuracy was 87%. After six months of continuous learning and parameter correction, this accuracy steadily increased to 94%. This accuracy remained stable across terrain and environmental conditions in different seasons, demonstrating the system's adaptability and sustainability.

[0086] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. An embodied intelligent AI inspection method based on digital twins, characterized in that: include: A dynamic attention multi-layer feature extraction network is used to analyze multispectral images collected by sensors onboard robots to generate a real-time distribution map of abnormal coverage of industrial facilities. Based on the real-time distribution map of abnormal coverage of industrial facilities, the historical abnormal coverage data is analyzed using the spatiotemporal sequence comparative learning model to generate abnormal coverage trend forecasts; Based on the prediction of abnormal coverage trend, the physical constraint neural differential equation model is used to analyze the abnormal coverage evolution process and generate accurate abnormal evolution prediction results; Based on the abnormal evolution prediction results and combined with the industrial facility status information in the digital twin system, a multi-objective optimization algorithm is used to generate the optimal inspection path; Based on multispectral images, real-time distribution maps of industrial facility anomaly coverage, anomaly evolution prediction results and real-time data during inspection execution, the digital twin system is dynamically updated and optimization suggestions for industrial facility maintenance are generated.

2. The embodied intelligent AI inspection method based on digital twins according to claim 1 is characterized in that: The step of using a dynamic attention multi-layer feature extraction network to analyze multispectral images collected by sensors carried by the robot to generate a real-time distribution map of abnormal coverage of industrial facilities includes: Use multispectral cameras and other sensors onboard the robot to collect images of the facility's surface, including visible and near-infrared wavelengths; Applying the adaptive enhancement algorithm for illumination conditions to pre-process multispectral images and reduce the interference of complex illumination conditions on detection accuracy; The pre-processed image is processed by a dynamic attention multi-layer feature extraction network, and the feature map weights are dynamically adjusted to make the model pay more attention to the real abnormal areas and ignore the light interference areas; The features extracted by different convolutional layers are fused through the feature pyramid network to capture abnormal features of different granularities; The abnormal coverage of each pixel is calculated through the fully connected layer and regression layer to generate the abnormal coverage distribution map of industrial facilities.

3. The embodied intelligent AI inspection method based on digital twins according to claim 1 is characterized in that: The step of analyzing historical abnormal coverage data using a spatiotemporal sequence contrast learning model to generate an abnormal coverage trend forecast includes: Collect abnormal coverage images of the same industrial facility at different time points, combine them with terrain data, and construct a time series dataset; Capturing the temporal evolution pattern of anomaly accumulation through spatiotemporal feature extraction network; Applying a contrastive learning framework, the model learns the feature differences under different anomaly accumulation rates; Combining topographic data with historical accumulation patterns to develop anomaly accumulation rate models; Based on the accumulation rate model, the future anomaly coverage distribution is predicted.

4. The embodied intelligent AI inspection method based on digital twins according to claim 1 is characterized in that: The steps of analyzing the abnormal coverage evolution process using the physical constraint neural differential equation model to generate accurate abnormal evolution prediction results include: The anomaly cover evolution is expressed as a spatiotemporal partial differential equation: ； in Indicates location ,time Abnormal coverage of represents the partial derivative of coverage with respect to time, indicating the rate of change of coverage over time; Indicates time Topographic and environmental conditions; represents the spatial gradient of coverage, , represents the partial derivative operator; represents the set of parameters to be learned; Represents the dynamic function to be learned, describing the physical and data-driven process of anomaly evolution; Build hybrid structures that combine physical prior knowledge with neural networks: ； in represents the total kinetic function; represents the physical prior term, based on the fluid dynamics equation; Represents the residual term of neural network learning; Construct a physical consistency constraint loss function to ensure that the model predictions conform to physical laws; Build an adaptive numerical solver to efficiently solve neural differential equations using a variable step-size algorithm with controllable error thresholds; Build an end-to-end training framework to jointly optimize physical parameters and neural network parameters.

5. The embodied intelligent AI inspection method based on digital twins according to claim 1 is characterized in that: The step of generating the optimal inspection path by using a multi-objective optimization algorithm includes: Calculate the inspection priority of industrial facilities based on the status information and abnormality prediction results of industrial facilities in the digital twin system; By considering real-time terrain data, a robot energy consumption prediction model under the influence of terrain is constructed; Based on facility priority, energy consumption prediction, path accessibility, and safety factors, a multi-objective optimization problem is established to generate the optimal inspection path; During the inspection process, the inspection route is dynamically adjusted based on real-time environmental changes and industrial facility status; The optimized inspection path is converted into robot motion control instructions to achieve autonomous movement and perform inspection tasks.

6. The embodied intelligent AI inspection method based on digital twins according to claim 1 is characterized in that: The steps of dynamically updating the digital twin system and generating industrial facility maintenance optimization suggestions include: The abnormal coverage distribution map of industrial facilities obtained by robot inspection is transmitted back to the digital twin system in real time; Based on the processed data, the status information of industrial facilities in the digital twin system is updated; Correct the anomaly prediction model parameters based on the newly acquired measured data; Formulate maintenance strategies for industrial facilities based on abnormal coverage distribution and predicted trends; Based on the optimization results, maintenance action recommendations are generated, including maintenance schedule, area prioritization, and resource allocation plans.

7. The embodied intelligent AI inspection method based on digital twins according to claim 2 is characterized in that: The illumination condition adaptive enhancement algorithm is performed by the following steps: Estimate lighting intensity and angle in an image: ； in Represents pixel points Estimated light intensity at ; Represents pixel points The visible light band image intensity at ; Represents pixel points The near-infrared band image intensity at ; Represents the illumination estimation function, which is used to fuse visible light and near-infrared information and output light intensity; Calculate the adaptive enhancement parameters: ； in Represents pixel points Adaptive enhancement parameters at ; Represents a parameter mapping function that dynamically adjusts the enhancement coefficient according to the local light intensity; Represents pixel points Estimated light intensity at ; Apply adaptive enhancements: ； in The enhanced image is The pixel value at ; Indicates that the original image is The pixel value at ; Represents pixel points Adaptive enhancement parameters at .

8. The embodied intelligent AI inspection method based on digital twins according to claim 2 is characterized in that: The core of the dynamic attention multi-layer feature extraction network is the calculation of the attention weight matrix: Attention weight calculation: ； in represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; represents the attention mapping function, which maps the feature map to the weight distribution; Represents the sigmoid activation function; Weighting of feature maps and attention weights: ； in Represents the weighted feature map; represents the attention weight matrix; Represents the feature map extracted by the convolutional layer; Represents the element-wise multiplication operator.

9. The embodied intelligent AI inspection method based on digital twins according to claim 3 is characterized in that: The contrastive learning framework enables the model to learn the feature differences under different anomaly accumulation rates; the loss function of contrastive learning is defined as: ； in represents the contrastive learning loss function; represents the feature representation of the current sample, Represents Paired positive sample feature representation, Represents the feature representation of other samples in the batch; represents the cosine similarity function; represents the temperature parameter, which controls the smoothness of the distribution; represents the sample index, , represents the original number of samples in the batch, and Indicates the total number of samples after considering data enhancement; represents the indicator function, when Time , otherwise ; Represents the exponential operator; Represents the natural logarithm operator.

10. A computer storage medium, characterized in that It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement an embodied intelligent AI inspection method based on digital twins as described in any one of claims 1 to 9.

Images