Inspection autonomous learning method and device based on embodied intelligence large model driving

Abstract

The application discloses a kind of based on embodiment intelligent large model driving's inspection autonomous learning method, device, equipment and medium, it is related to artificial intelligence technical field, method includes: control multi-sensor acquisition current inspection scene original data and extract feature data, original data includes scene data and / or equipment state data;Feature data and the priori knowledge matching of energy inspection association of light model;When similarity is less than preset threshold, trigger autonomous learning, generate new inspection operation rule for unknown scene, or generate new fault judgment basis for unknown fault;Based on matching result or autonomous learning result, output action instruction;Inspection robot executes instruction and collects entity interaction feedback data;Calibrate action according to feedback data, simultaneously feedback data, new rule or new criterion are uploaded to cloud, and incremental update is carried out to large model, and learning closed loop is formed.The application improves the flexibility and intelligent level of inspection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to an inspection autonomous learning method, device, equipment, and medium driven by an embodied intelligence large model. Background Technology

[0002] In the energy sector, the stable operation of energy equipment is crucial, and inspection is a key link in ensuring its normal operation. With continuous technological advancements, inspection technology is also constantly improving. Traditional manual inspection suffers from low efficiency and high labor intensity. The gradual introduction of automated inspection equipment has improved efficiency to some extent. The emergence of automated inspection technology reduces manpower, allowing inspections to be carried out more efficiently and playing a positive role in ensuring the stable operation of energy systems. Simultaneously, the development of information technology has brought new opportunities to inspection technology, with intelligent inspection gradually becoming a research and application hotspot. However, current energy inspection robots mostly rely on pre-programmed manual operations. Automated inspection based on fixed programs involves pre-setting inspection routes and actions, with the equipment following the program. While this improves efficiency, it struggles to handle complex and ever-changing inspection scenarios. When faced with unknown scenarios or equipment malfunctions, manual reprogramming and debugging are required, resulting in poor operational flexibility. Therefore, the current level of intelligence and autonomy in energy inspection is low, failing to meet the diverse inspection needs of energy scenarios such as photovoltaic, wind power, and substations. Summary of the Invention

[0003] To address the aforementioned technical issues, this application provides a method, apparatus, device, and medium for autonomous learning of inspection based on an embodied intelligence large model.

[0004] Firstly, this application provides an inspection self-learning method driven by an embodied intelligence large model, comprising: applied to an energy inspection system, the energy inspection system including a cloud model center and an inspection robot, the inspection robot being equipped with multiple sensors and an edge computing unit, the cloud model center deploying a dedicated embodied intelligence large model for energy inspection, and the edge computing unit deploying a lightweight version of the embodied intelligence large model; the method includes: controlling multiple sensors to collect raw data of the current inspection scene and extracting feature data corresponding to the raw data, wherein the raw data includes scene data and / or equipment status data of the current inspection scene; matching the feature data with pre-set prior knowledge associated with the energy inspection scene in the lightweight version of the embodied intelligence large model; when the similarity of the matching result is less than a preset threshold, triggering self-learning, the self-learning including: for unknown scenarios, processing the raw data... The data undergoes semantic segmentation to identify inspection objects, extracts geometric features of these objects to generate a set of detection points, and plans and generates new inspection operation rules based on these detection points and obstacle information in the scene. Alternatively, for unknown faults, fault features are extracted from the raw data, associated with prior knowledge about the equipment structure, and new fault judgment criteria that include the relationship between fault features and equipment structure are generated. Based on successful matching results or new inspection operation rules or new fault judgment criteria generated through autonomous learning, action commands are output. The inspection robot is driven to execute the action commands, and entity interaction feedback data is collected simultaneously. The action commands are adjusted in real time based on the entity interaction feedback data to complete action calibration. At the same time, the entity interaction feedback data, new inspection operation rules, or new fault judgment criteria are uploaded to the cloud model center for incremental updates to the embodied intelligence big model, forming a learning closed loop.

[0005] By adopting the above technical solutions, the energy inspection system can use multiple sensors to collect data and extract features, match them with prior knowledge, trigger autonomous learning when the similarity is low, generate new inspection operation rules or new fault judgment criteria, output action commands to drive the inspection robot to execute and collect feedback data, adjust and calibrate the action commands in real time, and upload data to update the embodied intelligent big model, realizing autonomous learning and closed-loop optimization of energy inspection, and improving the flexibility and intelligence level of inspection.

[0006] Optionally, the prior knowledge is obtained by structurally refining inspection data from a multi-scenario inspection sample library. The multi-scenario inspection sample library is constructed from multimodal perception data collected under normal and fault conditions of photovoltaic modules, wind turbine blades, and substation equipment. The multimodal perception data includes at least one of image, point cloud, and acoustic data. The prior knowledge includes quantitative parameters of equipment features, quantitative indicators of operational safety boundaries, and basic thresholds of fault features.

[0007] By adopting the above technical solutions, the multi-scenario inspection sample library is constructed from multimodal perception data of photovoltaic modules, wind turbine blades, and substation equipment under normal and fault conditions. Prior knowledge is obtained through structured extraction, providing comprehensive and rich energy inspection-related information. This prior knowledge includes quantitative parameters of equipment characteristics, quantitative indicators of operational safety boundaries, and basic thresholds for fault characteristics. It provides accurate reference for subsequent feature matching, autonomous learning, and other operations, improving the accuracy and reliability of energy inspections. New inspection operation rules or new fault judgment criteria generated by the inspection robot's autonomous learning are supplemented to the multi-scenario inspection sample library and the prior knowledge base after structured processing.

[0008] Optionally, new inspection operation rules are generated for unknown scenarios, specifically including: controlling the inspection robot to conduct multi-angle observations at a speed not exceeding a preset moving speed threshold to collect complete scene data; retrieving feature samples similar to the complete scene data from a multi-scenario inspection sample library using the K-nearest neighbor algorithm; and autonomously generating quantitative operation rules that include inspection path planning and detection point selection based on similar feature samples and complete scene data using a linear interpolation algorithm. If the validity of the quantitative operation rules is confirmed through entity interaction verification, the quantitative operation rules are determined as new inspection operation rules.

[0009] By adopting the above technical solutions, the inspection robot is controlled to conduct multi-angle observations at a speed not exceeding a preset moving speed threshold, which can collect complete scene data and provide a comprehensive data foundation for the subsequent generation of new inspection operation rules. The K-nearest neighbor algorithm is used to retrieve similar feature samples from a multi-scene inspection sample library, which can effectively draw on existing experience. Based on similar feature samples and complete scene data, a linear interpolation algorithm is used to autonomously generate quantitative operation rules that include inspection path planning and detection point selection. These rules are then verified by entity interaction and confirmed as new inspection operation rules. This enables effective response to unknown scenarios, improves the adaptability and accuracy of inspection operations, and makes inspection operations more standardized and operable through quantitative operation rules.

[0010] Optionally, new fault judgment criteria can be generated by summarizing unknown faults. Specifically, this includes: extracting fault depth features from the original data through a convolutional neural network, and autonomously summarizing and generating new fault judgment criteria according to the following steps: using the measured value of the fault depth features as the midpoint of the interval of the new fault judgment criteria; reusing the judgment interval width of similar faults in the multi-scenario inspection sample library to calculate the upper and lower limits of the new fault judgment criteria; reusing the multi-feature logical judgment relationship of similar faults, and combining sensor errors and individual differences of equipment to perform boundary correction of the upper and lower limits.

[0011] By adopting the above technical solutions, convolutional neural networks can effectively extract fault depth features from the original data, providing a more accurate basis for subsequent judgments. Using the measured value of the fault depth features as the midpoint of the new fault judgment basis interval can ensure that the judgment center fits the actual situation. Reusing the width of the similar fault judgment interval can improve efficiency by leveraging existing experience. Reusing the logical judgment relationship of multiple features of similar faults and combining it with sensor errors and individual equipment differences for boundary correction can make the new fault judgment basis more scientific and accurate, thereby improving the ability to judge unknown faults and enhancing the accuracy and reliability of fault diagnosis in the energy inspection system.

[0012] Optionally, the action commands are adjusted in real time based on the entity interaction feedback data to complete the action calibration. Specifically, this includes: comparing the entity interaction feedback data with preset operation accuracy thresholds, equipment contact pressure thresholds, and robot motion posture deviation thresholds to identify action deviations; based on the action deviations, correcting at least one of the motion coordinates, end effector angles, and contact pressure parameters in the action commands using a PID control algorithm, generating calibration action commands and driving the inspection robot to execute them until the entity interaction feedback data falls within the preset qualified range defined by the operation accuracy threshold, equipment contact pressure threshold, and robot motion posture deviation threshold.

[0013] By adopting the above technical solution, the action deviation is identified by comparing the entity interaction feedback data with multi-dimensional preset thresholds. Then, the action command parameters are corrected by the PID control algorithm. The action of the inspection robot can be adjusted in real time so that its entity interaction feedback data falls into the preset qualified range, thus completing the action calibration and improving the accuracy and stability of the inspection action. At the same time, the embodied intelligent large model is incrementally updated by uploading data, forming a learning closed loop and improving the model performance and inspection effect.

[0014] Optionally, the embodied intelligent big model adopts a three-tier architecture: a basic big model layer, a scenario adaptation layer, and an interaction decision layer. The basic big model layer is built based on the Transformer architecture and is used to extract the underlying features common to all energy inspection scenarios. The scenario adaptation layer sets up independent adaptation modules for different types of energy inspection scenarios and has built-in structured prior knowledge for each scenario. The energy inspection scenarios include the following different types: photovoltaic, wind power, and substations. The interaction decision layer integrates an entity feedback interface to receive entity interaction feedback data and link calibration action commands.

[0015] By adopting the above technical solutions, the basic large model layer is built on the Transformer architecture and can extract the common underlying features of all energy inspection scenarios. The scenario adaptation layer sets up independent adaptation modules for different types of energy inspection scenarios and has built-in structured prior knowledge, which can adapt to a variety of energy inspection scenarios. The interactive decision layer integrates entity feedback interface, which can receive entity interaction feedback data and link calibration action commands, so that the embodied intelligent large model can be better applied to the energy inspection system and improve inspection efficiency and accuracy.

[0016] Optionally, the above method further includes: performing hierarchical transfer learning on the embodied intelligence large model to adapt to the new energy inspection scenario. The hierarchical transfer learning includes: hierarchical locking: locking the model parameters of the basic large model layer and releasing the module parameters of the scene adaptation layer corresponding to the new energy inspection scenario; targeted transfer: transferring the core logical parameters in the scene adaptation layer of the trained scenario to the corresponding adaptation module of the new energy inspection scenario as initial parameters; parameter-specific training: using the dataset of the new energy inspection scenario, training and updating the differentiated parameters in the scene adaptation layer of the new energy inspection scenario, keeping the transferred core logical parameters in the scene adaptation layer unchanged, and not training and updating the model parameters of the basic large model layer.

[0017] By adopting the above technical solution, hierarchical transfer learning is performed on the embodied intelligence large model. The model parameters of the basic large model layer are locked, and the parameters of the scene adaptation layer module corresponding to the new energy inspection scenario are released, which can avoid the basic large model layer being affected by the new scenario data. The core logical parameters in the scene adaptation layer of the already trained scenario are transferred to the corresponding adaptation module of the new scenario as initial parameters, which can accelerate the adaptation speed of the new scenario. The differentiated parameters in the scene adaptation layer are targeted for training and updating using the new scenario dataset. The core logical parameters of the transfer remain unchanged and the parameters of the basic large model layer are not trained. This can effectively adapt to the new energy inspection scenario and improve the performance and adaptability of the model in the new scenario.

[0018] Optionally, the above method further includes: performing meta-learning training on the embodied intelligence large model. The meta-learning training includes: constructing a cross-scenario task set: selecting non-standardized sub-scenario samples from a multi-scenario inspection sample library that provides prior knowledge for the embodied intelligence large model to construct a meta-learning task set; and meta-training: using a model-independent meta-learning algorithm to train the embodied intelligence large model, so that the embodied intelligence large model learns to make targeted adjustments to the core adaptation parameters of the model for a single non-standardized sub-scenario through a single batch of samples, so as to extract the general features across energy inspection scenarios and achieve adaptation to non-standardized sub-scenarios.

[0019] By adopting the above technical solution, non-standardized sub-scenario samples are selected from the multi-scenario inspection sample library to construct a meta-learning task set. The embodied intelligence big model is trained by the model-independent meta-learning algorithm. This enables the embodied intelligence big model to complete the targeted adjustment of the core adaptation parameters of the model through a single batch of samples, extract the general features across energy inspection scenarios, realize the adaptation of non-standardized sub-scenarios, and improve the adaptability and versatility of the embodied intelligence big model in different energy inspection scenarios.

[0020] Optionally, the above method also includes: using the trained embodied intelligence large model as the teacher model in the cloud; using the lightweight version as the student model in the edge computing unit of the inspection robot; and using the soft labels output by the teacher model to supervise the training of the student model through knowledge distillation training, with the goal of minimizing the preset loss function between the output of the student model and the soft labels, to complete the generation and deployment of the lightweight version.

[0021] By adopting the above technical solution and utilizing knowledge distillation training, the trained embodied intelligence large model is used as the teacher model, and the lightweight version is used as the student model. The soft labels output by the teacher model are used to supervise the training of the student model, which can minimize the preset loss function between the student model output and the soft labels. This enables the generation and deployment of the lightweight version, allowing the edge computing unit of the inspection robot to run the model more efficiently, reduce resource consumption, and improve the overall performance and efficiency of the energy inspection system.

[0022] In a second aspect of this application, an inspection autonomous learning device based on an embodied intelligence large model is also provided, located in an energy inspection system, for executing the inspection autonomous learning method based on an embodied intelligence large model as described above, comprising: a control module for controlling multiple sensors to collect raw data of the current inspection scenario and extract feature data corresponding to the raw data, wherein the raw data includes scene data and / or equipment status data of the current inspection scenario; a matching module for matching the feature data with pre-set prior knowledge associated with the energy inspection scenario in a lightweight version of the embodied intelligence large model; and a triggering module for triggering autonomous learning when the similarity of the matching result is less than a preset threshold, wherein the autonomous learning includes: for unknown scenarios, performing semantic segmentation on the raw data to identify the inspection object, and extracting the geometric features of the inspection object. The system generates a set of detection points and plans new inspection operation rules based on the detection points and obstacle information in the scene; or, for unknown faults, it extracts fault features from the raw data, associates them with prior knowledge about the equipment structure, and summarizes and generates new fault judgment criteria that include the relationship between fault features and equipment structure; the driving module outputs action commands based on the successful matching results or the new inspection operation rules or new fault judgment criteria generated by autonomous learning; it drives the inspection robot to execute the action commands and simultaneously collects entity interaction feedback data; the processing module adjusts the action commands in real time according to the entity interaction feedback data to complete the action calibration, and uploads the entity interaction feedback data, new inspection operation rules or new fault judgment criteria to the cloud model center to incrementally update the embodied intelligence big model and form a learning closed loop.

[0023] In a third aspect of this application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the program to implement the method steps of any of the above claims.

[0024] In a fourth aspect of this application, a computer-readable storage medium is also provided, which stores instructions that, when executed, perform the method steps of any of the above claims.

[0025] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. The energy inspection system can use multiple sensors to collect data and extract features, match them with prior knowledge, and trigger autonomous learning when the similarity is low. This generates new inspection operation rules or new fault judgment criteria, outputs action commands to drive the inspection robot to execute and collect feedback data, adjusts and calibrates the action commands in real time, and uploads data to update the embodied intelligent big model, realizing autonomous learning and closed-loop optimization of energy inspection, and improving the flexibility and intelligence level of inspection. 2. Based on similar feature samples and complete scene data, a linear interpolation algorithm is used to autonomously generate quantitative operation rules that include inspection path planning and inspection point selection. These rules are then verified by entity interaction and confirmed as new inspection operation rules. This enables effective response to unknown scenarios, improves the adaptability and accuracy of inspection operations, and makes inspection operations more standardized and operable through quantitative operation rules. 3. By comparing the entity interaction feedback data with the preset threshold to identify action deviations, and then correcting the action command parameters through the PID control algorithm, the robot's actions can be adjusted in real time to ensure that its entity interaction feedback data falls within the preset qualified range, thus completing action calibration and improving the accuracy and stability of inspection actions. At the same time, by uploading data, the embodied intelligent large model is incrementally updated to form a learning closed loop, thereby improving model performance and inspection results. Attached Figure Description

[0026] Figure 1 This is a flowchart of an inspection self-learning method driven by an embodied intelligence large model, provided in an embodiment of this application. Figure 2 This is a structural block diagram of an inspection autonomous learning device driven by an embodied intelligence large model, provided in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0027] Explanation of reference numerals in the attached figures: 300 - Electronic device; 301 - Processor; 302 - Communication bus; 303 - User interface; 304 - Network interface; 305 - Memory. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0029] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0030] In the description of the embodiments of this application, the term "multiple" means two or more. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0031] It should be noted that the "embodied intelligence model" described in this application differs from general-purpose large models that primarily generate text or images. Its core lies in understanding the physical world and generating action sequences for interaction with the environment. This model is designed to process multimodal sensor data and combine it with the robot's physical action capabilities to complete specific tasks in real-world environments. Essentially, it constructs a closed-loop intelligent system encompassing perception, decision-making, action, and feedback.

[0032] The following is in conjunction with the appendix Figure 1 -Appendix Figure 3 This application is described below.

[0033] This application provides an inspection autonomous learning method driven by an embodied intelligence large model, referring to... Figure 1 , Figure 1 This application provides a flowchart of an inspection self-learning method based on an embodied intelligence large model, applied to an energy inspection system. The energy inspection system includes a cloud model center and an inspection robot. The inspection robot is equipped with multiple sensors and an edge computing unit. The cloud model center deploys a dedicated embodied intelligence large model for energy inspection, and the edge computing unit deploys a lightweight version of the embodied intelligence large model. The method includes the following steps: Step S101: Control multiple sensors to collect raw data of the current inspection scenario and extract feature data corresponding to the raw data. The raw data includes scene data and / or equipment status data of the current inspection scenario. Step S102: Match the feature data with the pre-set prior knowledge associated with the energy inspection scenario in the lightweight version of the embodied intelligence big model; Step S103: When the similarity of the matching results is less than a preset threshold, autonomous learning is triggered. Autonomous learning includes: for unknown scenarios, performing semantic segmentation on the original data to identify the inspection object, extracting the geometric features of the inspection object to generate a set of detection points, and planning and generating new inspection operation rules based on the detection points and obstacle information in the scenario; or, for unknown faults, extracting fault features from the original data, associating them with prior knowledge about the equipment structure, and summarizing and generating new fault judgment criteria that include the correlation between fault features and equipment structure. Step S104: Based on the successful matching result or the new inspection operation rules or new fault judgment criteria generated by autonomous learning, output action instructions; drive the inspection robot to execute the action instructions, and simultaneously collect entity interaction feedback data; Step S105: Adjust the action instructions in real time based on the entity interaction feedback data to complete the action calibration. At the same time, upload the entity interaction feedback data, new inspection operation rules or new fault judgment criteria to the cloud model center to incrementally update the embodied intelligent big model and form a learning closed loop.

[0034] Through the above steps, the energy inspection system can use multiple sensors to collect data and extract features, match them with prior knowledge, trigger autonomous learning when the similarity is low, generate new inspection operation rules or new fault judgment criteria, output action commands to drive the inspection robot to execute and collect feedback data, adjust and calibrate the action commands in real time, upload data to update the embodied intelligence big model, perform incremental updates to the embodied intelligence big model, and synchronize the updated knowledge to the lightweight version of the embodied intelligence big model, forming a learning closed loop, realizing autonomous learning and closed-loop optimization of energy inspection, and improving the flexibility and intelligence level of inspection.

[0035] This application's embodiments are based on a cloud-edge collaborative embodied intelligent large-scale model architecture, constructing a full-process autonomous learning system for inspection, encompassing "data acquisition, feature matching, autonomous learning, action execution, feedback calibration, and model iteration." The core relies on the collaborative work of the full cloud version and the lightweight edge version of the energy inspection embodied intelligent large-scale model, combined with multi-sensor perception and entity interaction feedback, to achieve autonomous adaptation and learning for the inspection robot. A massive "energy inspection-specific embodied intelligent large-scale model" is deployed in the cloud, acting as an "expert brain" possessing a vast amount of prior knowledge about energy equipment, scenarios, and fault modes. The inspection robot itself, acting as the edge terminal, is equipped with multiple sensors (eyes, ears) and an edge computing unit (local cerebellum). The edge computing unit deploys a "lightweight version" of the cloud-based large-scale model, ensuring that the robot can perform real-time, rapid localized reasoning and response even when disconnected from the cloud. First, the inspection robot collects raw data of the current scene (including scene structure, equipment appearance, operating parameters, etc.) through multiple sensors (such as cameras, infrared, lidar, etc.). The edge computing unit extracts key feature data (such as equipment temperature, meter readings, equipment outline, etc.) from the raw data and matches it with the preset "prior knowledge" about the energy scene in the local "lightweight version" model. This matching process determines "whether I recognize / are familiar with the current situation". When the similarity of the matching result is less than a preset threshold, it means that the robot has encountered an "unknown" or "unfamiliar" scene or fault, and the fixed program fails. At this time, the autonomous learning mechanism is triggered. The robot no longer relies on manual reprogramming but explores and analyzes autonomously: For unknown scenes: the robot performs deep processing on the raw sensor data (such as semantic segmentation) to understand "what is" in the picture (such as pipes, valves). Then, it extracts the geometric features (size, position, shape) of these new objects and autonomously generates a set of "detection points" (i.e., from which angle and distance to inspect this new object). Finally, combining real-time obstacle information, a completely new "inspection operation rule" is planned and generated (e.g., first bypass obstacle A, then take a picture from a specific angle in front of the newly discovered valve B). For unknown faults: the robot extracts abnormal "fault features" from the data (e.g., abnormal vibration frequency, areas of excessively high local temperature). Then, it no longer simply matches the fault database, but associates prior knowledge about "equipment structure" in the model (e.g., this vibrating component is a bearing, that high-temperature area is a junction box connector), and through inductive reasoning, autonomously generates a new "fault judgment basis" containing the correlation between "fault features" and "equipment structure" (e.g., when the vibration frequency of the bearing reaches X and is accompanied by temperature Y, it may be a new fault mode of bearing wear). Based on the mature rules that have been successfully matched (familiar with the situation), or new rules / basis generated through autonomous learning, the system outputs specific "action instructions."When the similarity of the matching results is greater than or equal to a preset threshold, the robot outputs action commands based on the prior knowledge of the successful match, driving the inspection robot to perform the corresponding inspection actions. The robot drives its own mechanical structure to execute commands (such as moving, adjusting the gimbal, taking pictures, and measuring temperature), and interacts with the physical world. This interaction process is not a one-way command execution, but rather a synchronous collection of "physical interaction feedback data," such as the resistance felt when performing a grasping action, the quality of newly captured images after adjusting the gimbal, and the positional deviation detected by the LiDAR after moving to a certain point. The "physical interaction feedback data" is used to fine-tune the actions being performed in real time, completing action calibration (e.g., adjusting the grasping force based on resistance, and fine-tuning the gimbal angle based on image clarity). After the task is completed, the robot uploads all valuable data generated during this interaction (physical interaction feedback data, newly generated inspection rules, and newly discovered fault judgment criteria) to the cloud. The "embodied intelligence big model" in the cloud uses this new data for incremental updates, feeding back the practical experience of the individual to the "expert brain," continuously enriching and evolving the knowledge base of the cloud model. The next time other robots or the local machine encounter a similar scenario, they can directly match the updated "lightweight version" locally without repeating autonomous learning, thus forming a continuous optimization loop of "practice-learning-evolution-re-practice". In related technologies, robots based on fixed programs are helpless when they encounter unknown scenarios or faults outside the preset program, requiring manual intervention for programming and debugging. This embodiment, through an "autonomous learning" mechanism, enables the robot to autonomously cope with unknown situations and generate new rules in real time, greatly improving the flexibility of operations. In related technologies, inspection is essentially programmed automatic execution rather than intelligent decision-making. This embodiment introduces the concepts of "embodied intelligence" and "large model", allowing the robot not only to "perceive" but also to "cognize" and "make decisions". Through interaction with the environment and continuous learning, it truly achieves autonomy and intelligence. In scenarios such as photovoltaic, wind power, and substations, the environment is complex, the equipment is diverse, and the faults are varied. The adaptive learning capability of this embodiment enables it to continuously adapt to new equipment, new layouts, and new fault modes, thereby meeting diverse and dynamically changing inspection needs. This embodiment establishes a closed-loop autonomous learning mechanism encompassing perception, decision-making, action, feedback, and iteration, enabling the inspection robot to adapt to complex and ever-changing energy inspection scenarios without human intervention.

[0036] Imagine a large photovoltaic power plant that has deployed the inspection robot system described in this application. Scenario: The power plant has recently installed a batch of new bifacial photovoltaic modules, equipment the robot has never seen before.

[0037] Existing technical problems: Traditional robots perform inspections along fixed routes, only checking the parameters of existing single-sided components. When encountering new components, they may overlook them or generate false alarms due to data mismatch. Engineers must go to the site to reprogram the robot, informing it of the location of new components, inspection points, and inspection standards.

[0038] The working process of this embodiment: Perception and Mismatch: The robot inspects along its usual route, and its multiple sensors scan for this batch of new components. The edge computing unit extracts features (shape, size, texture) and matches them with prior knowledge in the local "lightweight model" (all known component models). The result shows extremely low similarity, and the scene is judged as "unknown". Self-learning trigger: The system triggers the self-learning module "for unknown scenarios"; New rule generation: The robot performs a 360-degree scan of the new component, uses a semantic segmentation algorithm to identify it as a "photovoltaic module," and calculates its outline. Then, based on the optimal detection angle and distance rules for the component, it autonomously generates a set of detection points (e.g., taking a panoramic view 2 meters in front, taking a view of edge hot spots 1.5 meters to the left at a 45-degree angle). At the same time, it scans the surrounding environment, identifies obstacles such as pillars and fences, and finally plans a new inspection operation rule that can safely reach all detection points. Execution and Feedback: The robot moves according to the new rules. During the movement, it fine-tunes the path based on real-time feedback LiDAR data to avoid suddenly appearing obstacles (physical interaction feedback); after reaching the point, it adjusts the gimbal to focus and take pictures. Closed-loop learning: After the task is completed, the robot uploads the "new inspection operation rules" and the data from this interaction to the cloud. After learning, the cloud-based "embodied intelligent big model" updates its knowledge about "bifacial photovoltaic modules". The next time, whether it is this robot or other robots in the power station, when encountering the same type of module, it can directly call up the updated local model without repeating the learning process.

[0039] Taking the substation switchgear inspection scenario as an example, the robot detects an unusual partial discharge noise (unknown fault) in the switchgear that has never been preset. After collecting raw data such as the appearance and acoustics of the switchgear and extracting fault features, the robot finds that the matching with the prior knowledge at the edge does not meet the standard, triggering autonomous learning: it extracts the deep features of the fault through a convolutional neural network, associates them with the internal structure knowledge of the switchgear in the prior knowledge, and summarizes and generates a new judgment basis for the association between the features of the partial discharge fault and the switchgear structure. When the robot performs the fault verification action based on the new judgment, it calibrates the detection contact pressure and position through feedback data to ensure the accuracy of the fault judgment. At the same time, it uploads the new fault judgment basis and feedback data to the cloud to complete the model update, realizing the subsequent rapid identification of this type of unknown fault.

[0040] In an optional embodiment, the prior knowledge is obtained by structured extraction of inspection data from a multi-scenario inspection sample library; the multi-scenario inspection sample library is constructed from multimodal perception data collected under normal and fault conditions of photovoltaic modules, wind turbine blades, and substation equipment, and the multimodal perception data includes at least one of image, point cloud, and acoustic data; the prior knowledge includes quantitative parameters of equipment features, quantitative indicators of operational safety boundaries, and basic thresholds of fault features.

[0041] In the above embodiments, the multi-scenario inspection sample library is constructed from multimodal perception data of photovoltaic modules, wind turbine blades, and substation equipment under normal and fault conditions. Prior knowledge is obtained through structured extraction, providing comprehensive and rich energy inspection-related information. This prior knowledge includes quantitative parameters of equipment characteristics, quantitative indicators of operational safety boundaries, and basic thresholds for fault characteristics. It provides accurate reference for subsequent feature matching, autonomous learning, and other operations, improving the accuracy and reliability of energy inspections. New inspection operation rules or new fault judgment criteria generated by the inspection robot's autonomous learning are supplemented to the multi-scenario inspection sample library and the prior knowledge base after structured processing.

[0042] This embodiment constructs a multi-scenario inspection sample library as the underlying data foundation for prior knowledge. It then performs structured extraction of multimodal perception data from various types of energy equipment within the sample library, forming standardized and quantified prior knowledge. This provides accurate and unified reference for subsequent feature data matching, autonomous learning triggering, and autonomous judgment of unknown scenarios or faults. The core logic is as follows: First, multimodal data such as images, point clouds, and acoustics are collected from three core energy equipment types—photovoltaic, wind power, and substations—under normal and fault conditions to construct an inspection sample library covering multiple scenarios. Next, the sample library data is structured and quantitatively extracted to obtain three core prior knowledge categories: equipment feature quantification parameters, operational safety boundary quantification indicators, and fault feature basic thresholds. Finally, this prior knowledge is pre-defined in a lightweight version of the embodied intelligent large model, serving as the benchmark for feature data matching during the inspection process and supporting the effective triggering and implementation of subsequent autonomous learning. Structured extraction refers to extracting knowledge units with clear physical meaning, quantifiability, and direct computer usability from massive amounts of raw image, point cloud, and sound data through data processing and analysis. After refinement, prior knowledge is concretized into three quantifiable indicators: Quantifiable parameters of equipment characteristics: for example, the standard color range and geometric dimensions (aspect ratio) of a normal photovoltaic module in an image, the airfoil profile point cloud data of a wind turbine blade, and the standard attitude angle of a substation circuit breaker. These parameters allow the robot to "know" what a piece of equipment "should look like." Quantifiable indicators of operational safety boundaries: for example, the minimum safe distance (quantified in meters or centimeters) that an inspection robot must maintain from live parts when working around high-voltage equipment; and the safe distance from rotating parts when operating inside a wind turbine nacelle. These indicators set "red lines" for the robot's autonomous planning and action execution, ensuring the safety of both the equipment and the robot itself. Basic thresholds for fault characteristics: for example, the temperature threshold for hot spot faults in photovoltaic modules (e.g., exceeding the ambient temperature by X degrees), the alarm threshold for excessively high transformer oil temperature, and the abnormal range of gearbox vibration frequency. These thresholds serve as the "yardstick" for the robot to make preliminary fault judgments. This quantified prior knowledge collectively constitutes the initial knowledge core of a lightweight version of the embodied intelligence model. It serves as the benchmark and starting point for the robot to understand the world, judge what is "familiar" and "unfamiliar," and engage in autonomous learning. When the robot perceives its environment, the feature data it extracts (such as the current device temperature) is compared with these preset "quantified parameters" and "basic thresholds." If the temperature exceeds the "basic threshold of fault characteristics," the robot initially judges it as a "suspected fault." If the device's geometric dimensions differ significantly from the "quantified parameters of device characteristics," the robot identifies it as an "unknown device," thus triggering subsequent autonomous learning.Based on three core energy equipment types—photovoltaic, wind power, and substations—a multi-scenario sample library is constructed. The extracted prior knowledge can be adapted to different energy inspection scenarios, breaking the limitations of single-scenario data and providing basic data support for inspection robots to operate across scenarios. Multimodal perception data and structured prior knowledge provide reference samples and logical basis for autonomous learning of unknown scenarios / faults, making the generation of new inspection operation rules and new fault judgment criteria more reasonable and improving the quality and efficiency of autonomous learning.

[0043] In an optional embodiment, the method further includes: before training the embodied intelligence large model, using a combination strategy to augment the fault samples in the multi-scenario inspection sample library, wherein the combination strategy includes: data augmentation using geometric transformation, illumination transformation, and noise addition, and generative data augmentation using generative adversarial networks (GANs). Addressing the industry pain point of scarce fault samples in industrial inspections, before training the embodied intelligence large model, a combination strategy of traditional data augmentation + generative adversarial network (GAN) generative augmentation is used to specifically augment the fault samples in the multi-scenario inspection sample library, providing sufficient and diverse fault data support for model training. The core logic is as follows: Based on a multi-scenario inspection sample library, traditional data augmentation techniques such as geometric transformation, illumination transformation, and noise addition are first used to diversify the existing fault samples, rapidly increasing the sample quantity. Then, generative adversarial networks (GANs) are employed for generative data augmentation. Through adversarial training between the generator and discriminator, simulated fault samples with highly similar characteristics to real fault samples are generated, overcoming the sample homogenization problem of traditional augmentation. The combination of these two augmentation strategies achieves a dual increase in the quantity and diversity of fault samples, laying a high-quality data foundation for the subsequent training of the embodied intelligent large model and ensuring the effectiveness of the model's fault identification and autonomous learning capabilities. Taking the expansion of partial discharge fault samples in substation switchgear as an example, the sample library contained only 30 acoustic and image samples of partial discharge faults. Through traditional augmentation, the samples were scaled, contrast adjusted, and salt-and-pepper noise was added to obtain 300 samples. Then, 150 highly realistic partial discharge fault samples were generated using GANs, bringing the total number of samples to 450. The combined enhancement strategy achieves a multiple-fold increase in the number of fault samples, compensating for the lack of real fault sample collection and providing sufficient data support for the training of the embodied intelligence large model, avoiding insufficient training due to insufficient data. The abundant and diverse fault samples allow the embodied intelligence large model's fault identification and autonomous learning capabilities for unknown faults to be fully trained, significantly improving the accuracy of the model's fault judgment. At the same time, it provides a more comprehensive sample reference for the inductive generation of unknown fault criteria, making the autonomous learning results more reasonable. The algorithm generates highly realistic fault samples, replacing some of the work of manual on-site collection and labeling of real fault samples, greatly reducing the cost of acquiring fault samples in industrial inspection scenarios and improving the efficiency of model development and training.

[0044] In an optional embodiment, generating new inspection operation rules for unknown scenarios specifically includes: controlling the inspection robot to perform multi-angle observations at a speed not exceeding a preset moving speed threshold to collect complete scene data; retrieving feature samples similar to the complete scene data from a multi-scene inspection sample library using the K-nearest neighbor algorithm; and autonomously generating quantified operation rules that include inspection path planning and detection point selection using a linear interpolation algorithm based on the similar feature samples and the complete scene data. If the quantified operation rules are confirmed to be valid through entity interaction verification, they are then determined as new inspection operation rules.

[0045] In the above embodiments, the inspection robot is controlled to conduct multi-angle observations at a speed not exceeding a preset moving speed threshold, which can collect complete scene data and provide a comprehensive data foundation for the subsequent generation of new inspection operation rules. The K-nearest neighbor algorithm is used to retrieve similar feature samples from a multi-scene inspection sample library, which can effectively draw on existing experience. Based on similar feature samples and complete scene data, a quantitative operation rule containing inspection path planning and detection point selection is generated autonomously through a linear interpolation algorithm, and is confirmed as a new inspection operation rule through entity interaction verification. This can effectively cope with unknown scenarios, improve the adaptability and accuracy of inspection operations, and make inspection operations more standardized and operable through quantitative operation rules.

[0046] This embodiment uses a multi-scenario inspection sample library and structured prior knowledge as its data foundation. Through a standardized process of "low-speed, multi-angle data collection – algorithmic retrieval of similar samples – interpolation to generate quantified rules – entity verification to confirm validity," it generates new inspection operation rules for unknown scenarios. The core logic is as follows: After triggering autonomous learning for an unknown scenario, the inspection robot is first controlled to observe at low speed and from multiple angles to ensure the collection of complete quantified feature data of the unknown scenario, avoiding data gaps that could lead to rule generation bias. Then, the K-nearest neighbor algorithm is used to retrieve known samples with similar features to the unknown scenario from the multi-scenario inspection sample library, providing a reasonable reference for new rule generation. Subsequently, based on the operation rules of similar samples and the complete data of the unknown scenario, a linear interpolation algorithm is used to generate quantified operation rules containing inspection paths and detection points, ensuring the rules' adaptability to the unknown scenario. Finally, entity interaction is used to verify the actual execution effectiveness of the rules. Once verified, these rules are confirmed as official new inspection operation rules, providing a standardized basis for subsequent inspections of similar unknown scenarios.

[0047] When the robot encounters an unknown scene (such as new equipment or a new layout), it is first controlled to observe from multiple angles at a speed not exceeding a preset moving speed threshold (such as 0.2 m / s, or other values). Through low-speed movement and omnidirectional observation, the robot can minimize motion blur and avoid collision risks, thereby collecting complete and high-quality scene data (including images, point clouds, etc.), laying the foundation for accurate analysis later. After obtaining the complete scene data, the robot needs to understand what this new scene is and how to deal with it. It uses the K-nearest neighbor algorithm, using the currently collected complete scene data as a query to search a multi-scene inspection sample library. The core idea of ​​the K-nearest neighbor algorithm is "birds of a feather flock together." The robot searches the sample library for the K historical samples that are most similar to the current scene data. These similar samples represent old scenes that have been processed in the past and have certain similarities to the new scene, along with their corresponding successful inspection rules. With the complete data of the current scene and the K most similar historical samples and their rules (representing "learnable experience"), the robot enters the core rule generation stage. It employs a linear interpolation algorithm to make mathematical trade-offs and deductions between "new problems" and "old experience." The application of the linear interpolation algorithm here is very ingenious. For example, if the spatial size of the current scene is between two most similar historical scenes A (smaller space) and B (larger space), then the robot can use linear interpolation to calculate a new path length between the inspection path length of A and the inspection path length of B. Similarly, for the selection of inspection points, interpolation can be performed based on the best observation angle and distance of the equipment in similar scenes, thereby autonomously generating a set of quantitative and entirely new inspection path planning and inspection point selection rules. The generated rules are not directly adopted but need to be verified through physical interaction. The robot will execute this newly generated set of rules on a small scale and tentatively in the real physical world, and collect feedback data during the execution process (such as whether obstacle avoidance is safe, whether clear focus is possible, etc.). Only when the verification results prove that the rule is effective (for example, the robot can safely and completely complete the inspection of unknown equipment) will it be finally determined as a formal new inspection operation rule and stored in the knowledge base. This forms a rigorous closed loop of "generation-verification-confirmation".

[0048] The following example illustrates the situation using a photovoltaic power station encountering "new bifacial modules." The power station originally had 10,000 monofacial monocrystalline silicon modules (1.6m x 1.0m, fixed tilt angle of 30°), and the robot was already proficient in inspecting these modules (inspection path: traveling along the array centerline; inspection point: taking a picture 2.5m from the center of each module's front side). Currently, 200 new bifacial double-glass modules (1.8m x 1.1m, adjustable tilt angle, also generating electricity from the back) have been installed in the expanded area of ​​the power station. This is equipment the robot has never seen before. During its routine inspection to the edge of the expanded area, the robot's vision system detected the new modules.

[0049] Feature comparison: Prior knowledge in the local lightweight model indicates that the photovoltaic module should be 1.6m × 1.0m. However, visual recognition initially calculates the new module's length to be 1.8m and width to be 1.1m. The matching similarity is only 65%, lower than the preset threshold of 80%, triggering autonomous learning in the "unknown scene".

[0050] Perform low-speed observation: The robot reduces its movement speed from the normal 0.8 m / s to 0.2 m / s (preset movement speed threshold).

[0051] Multi-angle data acquisition: The robot conducted the following observations centered on the new component: Location A: Take a visible light image from a distance of 3.0m from the front; Location B: At a distance of 2.8m at a 45° angle to the left, capture infrared images and point clouds; Position C: Take a visible light image from a distance of 3.2m at a 30° angle to the right; Location D: Go around to the back of the component (first time), at a distance of 2.5m, and photograph the back structure and nameplate.

[0052] Data collected: A total of 12 images and 5,000 point cloud data points were collected, forming a complete scene data package for this new area (approximately 50MB).

[0053] The robot's edge computing unit initiates the K-nearest neighbor algorithm, using the current complete scene data package as the query object to retrieve the "multi-scene inspection sample library". K value setting: K=3 (retrieving the 3 most similar historical samples). Search results: Sample A (92% similarity): Inspection record of an adjustable bracket photovoltaic array from another power station. Although this array component is single-sided, its tilt angle is seasonally adjusted, and the robot has generated dynamic detection points adapted to different tilt angles. The path planning strategy recorded in this sample has a high weight. Sample B (87% similarity): A new component trial operation dataset from the experimental station. This dataset contains multi-angle observation records of a test component of similar size (1.75m × 1.05m), and its detection point selection rules (such as shooting distance and angle) are highly correlated with the current scene. Sample C (80% similarity): Inspection record of an existing single-sided component from this power station in a special terrain area. Due to slope changes in this area, the robot has generated non-standard inspection paths that avoid the terrain; its obstacle avoidance path planning logic can be referenced.

[0054] The robot fuses the current complete scene data with the three sample data retrieved from the K nearest neighbors and generates new rules through a linear interpolation algorithm.

[0055] Key parameters for the current scenario: Component dimensions: Length L_new=1.8m, Width W_new=1.1m; Component tilt angle: θ_new=25° (currently measured); Array layout: East-west spacing D_new=1.2m, North-South spacing H_new=0.5m; Key parameters of similar samples: Sample A: Component size (1.6×1.0), tilt angle dynamic range (20°~40°), path planning strategy: the main detection point is 2.5m from the front of each component, and the secondary detection point is 45° from the side. Sample B: Component size (1.75×1.05), inspection point rules: front distance = 2.8m, side 30° distance = 3.0m, back distance = 2.6m (for bifacial component prototypes); Sample C: Obstacle avoidance path planning: obstacle safety distance 0.3m, detour radius 1.5m.

[0056] New rules for linear interpolation generation: Detection point generation (based on sample B and current size interpolation): Front shooting distance: D_front=2.8m×(1.8 / 1.75)≈2.88m, Back shooting distance: D_back=2.6m×(1.8 / 1.75)≈2.67m, Side 30° distance: D_side30=3.0m×(1.8 / 1.75)≈3.09m; Inspection path planning (based on the strategy of sample A and the obstacle avoidance logic of sample C, combined with current layout interpolation): Considering that the component spacing (D_new=1.2m) is slightly smaller than the standard spacing in sample C, the robot adjusts the radius of the detour arc from 1.5m to 1.3m to ensure safe passage in narrow spaces. Borrowing the dynamic strategy of sample A, the main detection path is set: along the array centerline, stopping and taking pictures at 2.88m from the front of each component. New detection point: a stopping point is set at 2.67m from the back of each component (this is a completely new rule; existing single-sided components do not have this requirement). New auxiliary detection: at the end of each row of components, the side edges of the entire row of components are scanned from a 30° angle, 3.09m away.

[0057] Entity interaction verification: Instead of immediately incorporating these rules into the formal inspection tasks, the robot first conducted a small-scale verification. It moved to the front of the first bifacial module and took a picture at the newly generated distance of 2.88m. The LiDAR provided real-time feedback that the actual distance was 2.91m (with a slight error), but within the acceptable range. The image autofocused successfully, and the sharpness score reached 95 points (the preset passing score is 90 points). Next, it attempted to follow the newly planned path, circling around to the back of the module with an arc radius of 1.3m. During this process, the proximity sensor detected a minimum distance of 0.35m to the adjacent module, which was greater than the safe distance of 0.3m, verifying safety. Upon reaching a distance of 2.67m from the back, it successfully captured a clear image of the back. Verification conclusion: All interactive feedback data fell within the preset acceptable range, indicating that the rule is effective.

[0058] New inspection operation rules are generated: After verification, the robot encapsulates this complete set of rules and officially names it "Double-sided Double-glass Module Inspection Rules V1.0", which includes: Inspection path: Coordinate point sequence {P1(x1,y1,z1),P2(x2,y2,z2),...}; Detection points: {Front side 2.88m, Back side 2.67m, Side side 3.09m}; Action sequence: {Move -> Stop -> Focus -> Shoot -> Rotate gimbal ->...}.

[0059] In an optional embodiment, new fault judgment criteria are generated by inductive reasoning for unknown faults. Specifically, this includes: extracting fault depth features from the original data through a convolutional neural network, and autonomously inductively generating new fault judgment criteria according to the following steps: using the measured value of the fault depth features as the midpoint of the interval of the new fault judgment criteria; reusing the criterion interval width of similar faults in the multi-scenario inspection sample library to calculate the upper and lower limits of the new fault judgment criteria; reusing the multi-feature logical judgment relationship of similar faults, and combining sensor errors and individual differences of equipment to perform boundary correction on the upper and lower limits.

[0060] In the above embodiments, the convolutional neural network can effectively extract the fault depth features from the original data, providing a more accurate basis for subsequent judgments; using the measured value of the fault depth features as the midpoint of the new fault judgment basis interval can ensure that the judgment center fits the actual situation; reusing the width of the similar fault judgment interval can improve efficiency by leveraging existing experience; reusing the logical judgment relationship of multiple features of similar faults and combining sensor errors and individual differences of equipment for boundary correction can make the new fault judgment basis more scientific and accurate, thereby improving the ability to judge unknown faults and enhancing the accuracy and reliability of fault diagnosis in the energy inspection system.

[0061] This application's embodiments rely on a multi-scenario inspection sample library and structured prior knowledge as core support to achieve standardized and rational generation of criteria for judging unknown faults. The core logic is as follows: After triggering autonomous learning of unknown faults, a convolutional neural network is first used to deeply mine the raw data, extracting the fault depth features of the unknown fault, discarding surface data interference, and grasping the essential characteristics of the fault. Then, relying on the multi-scenario inspection sample library, a three-step method is used to quantitatively summarize new criteria. The measured value of the fault depth feature is used as the midpoint of the interval, and the upper and lower limits are calculated by reusing the interval width of similar fault criteria, giving the generation of new criteria clear sample references and quantitative logic. Finally, the multi-feature logical judgment relationship of similar faults is reused, and the upper and lower limits of the criteria are corrected by combining inherent sensor errors and individual equipment differences, making the generated new fault judgment criteria more consistent with actual inspection scenarios, ultimately forming a standardized basis that can be directly used for judging unknown faults. Convolutional neural networks are powerful deep learning models, adept at automatically learning and extracting hierarchical and abstract "deep features" from raw data. For example, for an infrared thermal image of a device, a convolutional neural network can not only extract the shallow feature of "highest temperature," but also automatically learn deeper and more discriminative features such as "the shape of the hotspot area, the texture of the temperature distribution, and the relative temperature difference pattern between the hotspot and the surrounding environment." These fault depth features constitute the most original and accurate digital description of a new fault. Next, it performs the most crucial first step: directly establishing the previously unseen fault depth feature value obtained from the actual measurement as the "midpoint of the interval" of the new judgment standard (criteria). In principle, this uses the objective facts of the current situation as the foundation and anchor point of new knowledge, ensuring the authenticity of the new rule's source; an isolated point (midpoint) cannot become a rule with fault tolerance. A rule must be a "range" with width. Where does the width of this interval come from? This is where the wisdom of this invention is reflected: "reuse (transfer)." The system searches its vast sample library for known faults "similar" to the current unknown fault features, and then "borrows" the "criteria interval width" of these similar faults. The underlying logic is that similar faults often exhibit similar physical stability and fluctuation ranges. Through this analogical reasoning, the robot assigns a reasonable fluctuation range, supported by historical experience, to this new fault point—its "upper and lower limits." The final step is to refine this initially formed "range" to make it more robust. This step involves two actions: reusing logical relationships: many faults are not determined by a single feature but by a combination of multiple features (e.g., "high temperature" and "abnormal noise"). The system will again draw upon the "multi-feature logical judgment relationships" (such as AND / OR logic) of similar faults to construct a multi-dimensional judgment framework for the new criteria, avoiding single-point misjudgments. Boundary correction: it combines theoretical models with physical reality, fully considering the two inevitable interference factors: "measurement errors inherent in the sensor itself" and "individual differences between different devices."By fine-tuning and compensating for the upper and lower limits, the interference of these non-fault factors is eliminated, making the final fault judgment criteria more accurate and reliable in the real world.

[0062] The quantitative criteria for medium-sized crack faults in wind turbine blades are illustrated as an example. 1) The core measured characteristics of unknown cracks in current wind turbine blades include: crack length = 5cm, width = 0.2mm, depth = 0.1mm, and abnormal noise frequency = 800Hz (this data is a precise quantitative value obtained by the robot through visual ranging, tactile sensing, and acoustic acquisition, and is the core anchor point for criterion summarization; all intervals revolve around this anchor point). 2) Mature criteria for the top 3 similar small crack faults extracted from the inspection sample library. For example, the verified criterion thresholds for the 3 small crack faults of wind turbine blades selected from the sample library are: length: 2cm~4cm, width: 0.1mm~0.15mm, depth: 0.05mm~0.09mm, and abnormal noise frequency: 600Hz~700Hz; the logical judgment relationship is that all features simultaneously meet the interval requirements (logical AND) before it can be judged as this type of small crack fault. 3) The measured values ​​of this unknown fault are the midpoints of the intervals for each characteristic, namely: midpoint of length = 5cm, midpoint of width = 0.2mm, midpoint of depth = 0.1mm, and midpoint of abnormal noise frequency = 800Hz. Referring to the interval width of similar faults, the upper and lower limits of the interval for the unknown fault are determined. The length interval width is 4cm-2cm=2cm, the width interval width is 0.15mm-0.1mm=0.05mm, the depth interval width is 0.09mm-0.05mm=0.04mm, and the abnormal noise frequency interval width is 700Hz-600Hz=100Hz. Therefore, the calculation of the upper and lower limits of the new criteria (midpoint ± 1 / 2 interval width) yields the following values: length: 4cm~6cm, width: 0.175mm~0.225mm, depth: 0.08mm~0.12mm, and abnormal noise frequency: 750Hz~850Hz. 4) Finally, the upper and lower limits are corrected based on sensor errors and individual equipment differences. It is assumed that the laser sensor has inherent errors of ±0.1cm, ±0.01mm, and ±0.01mm respectively in measuring crack length, width, and depth, and the acoustic sensor has an inherent error of ±10Hz in measuring abnormal noise frequency. Individual equipment differences exist: the wind turbine blade and the blades corresponding to small cracks in the sample library have slight differences in material and service life, requiring a reasonable tolerance margin. The corrected final range is: Length: 3.9cm~6.1cm; Width: 0.165mm~0.235mm; Depth: 0.07mm~0.13mm; Abnormal noise frequency: 740Hz~860Hz. This generates a new fault judgment criterion for the unknown crack fault of the wind turbine blade, which can be used for subsequent inspection fault determination. After generating the new fault judgment criterion, it is structured and entered into the fault database according to the fault database data structure of the multi-scenario inspection sample library, completing the incremental update of the fault identification database.

[0063] In an optional embodiment, the motion commands are adjusted in real time based on entity interaction feedback data to complete motion calibration. Specifically, this includes: comparing the entity interaction feedback data with preset operation accuracy thresholds, equipment contact pressure thresholds, and robot motion posture deviation thresholds to identify motion deviations; based on the motion deviations, correcting at least one of the motion coordinates, end effector angles, and contact pressure parameters in the motion commands using a PID control algorithm, generating calibration motion commands and driving the inspection robot to execute them until the entity interaction feedback data falls within a preset qualified range defined by the operation accuracy threshold, equipment contact pressure threshold, and robot motion posture deviation threshold.

[0064] In the above embodiments, the entity interaction feedback data is compared with multi-dimensional preset thresholds to identify action deviations. Then, the action command parameters are corrected through a PID control algorithm. This allows for real-time adjustment of the inspection robot's actions, ensuring that its entity interaction feedback data falls within a preset qualified range. This completes action calibration, improving the accuracy and stability of the inspection actions. Simultaneously, the uploaded data is used to incrementally update the embodied intelligent large model, forming a learning loop and improving model performance and inspection effectiveness.

[0065] This embodiment constructs a standardized action calibration process of "threshold comparison to identify deviations—algorithm correction and parameter adjustment—closed-loop verification to achieve qualification," providing precise real-time closed-loop control for the inspection robot's operation. The core logic is as follows: When the robot executes inspection action commands, the entity interaction feedback data collected synchronously serves as the actual basis for the action execution effect. First, this data is compared with three preset quantitative thresholds: operation accuracy, equipment contact pressure, and robot motion posture deviation, accurately identifying specific deviations in coordinates, angles, pressure, etc., during the action execution process. Then, relying on the closed-loop adjustment characteristics of the PID control algorithm, at least one of the motion coordinates, end effector angle, and contact pressure parameters in the action command is specifically corrected according to the identified deviation type, generating a calibrated action command and driving the robot to execute it. The cycle of "data acquisition—deviation identification—parameter correction—action execution" is continuously performed until the entity interaction feedback data falls within the preset qualified range defined by the three thresholds, completing the precise calibration of this inspection action and ensuring the accuracy and safety of the inspection action execution. For example, the operational accuracy threshold could be the allowable error range (e.g., ±1 mm) of the spatial coordinates (X, Y, Z) that the robotic arm's end effector needs to reach, or the allowable deviation (e.g., ±0.5 degrees) of the shooting angle that the gimbal needs to align with. The equipment contact pressure threshold refers to the controllable contact force when the robot needs to make physical contact with the inspected equipment (e.g., using probes to detect equipment vibration or operating switches). This threshold defines a safe force range (e.g., a maximum of 5 Newtons) to ensure effective contact without damaging precision equipment. The robot motion posture deviation threshold refers to the stability of the robot's posture (e.g., whether the robot body is tilted, the joint angle of the robotic arm) during movement or operation. This threshold defines the allowable range of posture changes (e.g., the tilt angle of the robot body does not exceed 0.5 degrees) to prevent tipping or collisions. The robot's various sensors collect real-time entity interaction feedback data (e.g., actual reached coordinates, pressure values ​​read by force sensors, and posture angles measured by the IMU) and compare this real-time data with the preset thresholds for the above three dimensions. Once a data point exceeds the threshold range, the system accurately identifies the specific type and magnitude of the motion deviation. After identifying the action deviation, the system enters the core calibration phase. This embodiment employs a classic and efficient PID control algorithm (proportional-integral-derivative control) to generate correction commands. The working principle of the PID controller is: based on the magnitude of the deviation (proportional term), the accumulation of historical deviations (integral term), and the trend of deviation changes (derivative term), a correction amount is dynamically calculated to adjust the original action command.Specifically, in this embodiment, the PID controller corrects the core execution parameters in the command: motion coordinates (adjusting the precise spatial position the robot needs to move to); end effector angle (adjusting the pointing angle of the robotic arm end or gimbal); and contact pressure parameter (adjusting the magnitude of the force applied to the device under test). The system generates the corrected calibration action command and drives the robot to execute it. Then, new interactive feedback data is acquired again through sensors, compared with the corresponding thresholds again, and corrected again by the PID algorithm. This process repeats at an extremely high frequency (tens or even hundreds of times per second), forming a negative feedback loop. Ultimately, this ensures that each entity interaction feedback data converges to a preset acceptable range defined by the three thresholds, thereby achieving high-precision and highly reliable action calibration. This embodiment provides a unified and clear standard for judging the robot's action execution deviations by presetting three specific quantification thresholds. It can accurately identify specific deviations in motion posture, contact pressure, and work accuracy, providing a clear basis for subsequent corrections. Relying on the closed-loop adjustment advantages of the PID control algorithm, the core parameters of the action command are specifically corrected according to the type of deviation. The correction process is supported by the algorithm and the adjustment is precise, avoiding the blindness of subjective corrections and greatly improving the efficiency and effectiveness of action calibration.

[0066] In an optional embodiment, the embodied intelligence big model adopts a three-tier architecture consisting of a basic big model layer, a scenario adaptation layer, and an interaction decision layer. The basic big model layer is built on the Transformer architecture and is used to extract the underlying features common to all energy inspection scenarios. The scenario adaptation layer sets up independent adaptation modules for different types of energy inspection scenarios and incorporates structured prior knowledge for each scenario. The energy inspection scenarios include the following different types: photovoltaic, wind power, and substations. The interaction decision layer integrates an entity feedback interface to receive entity interaction feedback data and link calibration action commands.

[0067] In the above embodiments, the basic large model layer is built on the Transformer architecture to extract the common underlying features of all energy inspection scenarios. The scenario adaptation layer sets up independent adaptation modules for different types of energy inspection scenarios and has built-in structured prior knowledge, which can adapt to a variety of energy inspection scenarios. The interactive decision layer integrates entity feedback interface to receive entity interaction feedback data and link calibration action commands, so that the embodied intelligent large model can be better applied to the energy inspection system and improve inspection efficiency and accuracy.

[0068] This embodiment establishes a three-tiered architecture consisting of a basic large model layer, a scene adaptation layer, and an interactive decision layer. This architecture enables layered collaboration and functional closure of the model through "general feature extraction, scene-specific adaptation, and entity feedback linkage," providing core model support for the entire inspection autonomous learning method. Specifically, a foundational large model layer is constructed using the Transformer architecture, serving as the general capability base for the model. This layer uniformly extracts the underlying features common to all energy inspection scenarios, including photovoltaic, wind power, and substations. Its role is to process undifferentiated raw data from all energy scenarios (photovoltaics, wind power, and substations), learning the underlying common features required for all energy inspection tasks. Examples include how to identify the edges between "equipment" and "background" in images, how to understand the structure and geometry of objects in three-dimensional space from point cloud data, and how to capture periodic and trend changes from time-series data (such as vibration and current). It also provides general semantic understanding capabilities, such as understanding basic concepts like "switch," "valve," and "pointer," enabling cross-scenario common feature recognition. The scenario adaptation layer, as the core of scenario-specific adaptation, sets up independent adaptation modules for different energy inspection scenarios. It incorporates structured prior knowledge for each scenario, achieving specific adaptation and decision support for common features to specific scenarios, addressing the most critical aspects in the energy field. For the three different scenarios of photovoltaics, wind power, and substations, independent adaptation modules are set up. Each adaptation module contains built-in structured prior knowledge specific to that scenario. For example, the photovoltaic adaptation module contains knowledge such as the standard size of photovoltaic modules, tilt angle range, common hot spot shapes, inverter models and parameters; the wind power adaptation module contains knowledge such as blade airfoil data, nacelle structure, typical vibration spectrum of gearbox and generator, and natural frequency of tower; the substation adaptation module contains knowledge such as appearance models of various circuit breakers, disconnect switches, and instrument transformers, meter reading range, insulator shape characteristics, and safe distances of high-voltage equipment. The interactive decision layer, as the physical linkage hub between the model and the inspection robot, integrates physical feedback interfaces, receives real-time physical interaction feedback data from the robot, and calibrates inspection action commands based on this data. This allows for deep integration of model decisions and robot physical actions, truly implementing the core idea of ​​embodied intelligence. The three-level architecture performs its respective functions and works collaboratively to support the efficient implementation of the entire inspection autonomous learning process. This embodiment significantly improves the model's generalization ability and adaptability in multiple scenarios; the general base layer can be shared, and the development of new scenarios only requires focusing on building a new "scenario adaptation module" and injecting the corresponding prior knowledge, without having to train the entire large model from scratch. This allows the system to be rapidly extended to new energy scenarios at extremely low cost.

[0069] In an optional embodiment, the above method further includes: performing hierarchical transfer learning on the embodied intelligence large model to adapt to the new energy inspection scenario. The hierarchical transfer learning includes: hierarchical locking: locking the model parameters of the basic large model layer and releasing the module parameters of the scene adaptation layer corresponding to the new energy inspection scenario; targeted transfer: transferring the core logic parameters in the scene adaptation layer of the trained scenario to the corresponding adaptation module of the new energy inspection scenario as initial parameters; parameter-specific training: using the dataset of the new energy inspection scenario, training and updating the differentiated parameters in the scene adaptation layer of the new energy inspection scenario, keeping the transferred core logic parameters in the scene adaptation layer unchanged, and not training and updating the model parameters of the basic large model layer.

[0070] In the above embodiments, hierarchical transfer learning is performed on the embodied intelligence big model. The model parameters of the basic big model layer are locked, and the parameters of the scene adaptation layer module corresponding to the new energy inspection scenario are released, which can avoid the basic big model layer being affected by the new scene data. The core logic parameters in the scene adaptation layer of the already trained scenario are transferred to the corresponding adaptation module of the new scenario as initial parameters, which can accelerate the adaptation speed of the new scenario. The differentiated parameters in the scene adaptation layer are targeted for training and updating using the new scene dataset. The core logic parameters of the transfer remain unchanged and the parameters of the basic big model layer are not trained. This can effectively adapt to the new energy inspection scenario and improve the performance and adaptability of the model in the new scenario.

[0071] This embodiment proposes a layered transfer learning adaptation method based on the three-level architecture of the embodied intelligence big model, achieving efficient model adaptation to new energy inspection scenarios. The core relies on the "layered decoupling" characteristic of the three-level architecture, allowing the model to adapt to new scenarios without requiring full retraining; only the scenario adaptation layer needs targeted updates. The core logic is as follows: leveraging the cross-scenario universality of the basic big model layer, the parameters of the basic big model layer are first locked and fixed layer by layer, releasing only the parameters of the scenario adaptation layer module corresponding to the new scenario, avoiding ineffective training of the general capability layer; then, through targeted transfer, the core logic parameters of the scenario adaptation layer of mature scenarios are transferred to the new scenario adaptation module as initial parameters, giving the new scenario adaptation module basic adaptation logic without needing to build it from scratch; finally, through parameter-targeted training, the dataset of the new scenario is used to update only the differentiated parameters of the scenario adaptation layer, keeping the core logic parameters unchanged and not touching the parameters of the basic big model layer. Ultimately, this achieves rapid and efficient model adaptation to new energy inspection scenarios while retaining the model's original general capabilities and the adaptation logic of already trained scenarios. In this embodiment, a new energy inspection scenario dataset with a significantly reduced sample size compared to traditional full-scale training is used. The number of samples is smaller than that of the dataset used to train the basic large model layer. Only the differential parameters in the scenario adaptation layer of the new energy inspection scenario are trained and updated, rather than retraining all parameters of the scenario adaptation layer.

[0072] Suppose an energy technology company has trained a large-scale embodied intelligence model capable of handling inspection tasks in three major scenarios: photovoltaic, wind power, and substations. The model employs a three-tier architecture including: Basic large model layer: Based on the Transformer architecture, with 200 million parameters, it has been pre-trained on massive energy data; Scenario adaptation layer: Includes three independent modules - photovoltaic adaptation module (20 million parameters), wind power adaptation module (25 million parameters), and substation adaptation module (22 million parameters); Interactive decision layer: Integrates entity feedback interface, with 5 million parameters.

[0073] Current new task: The company has won a bid for an intelligent inspection project for a hydroelectric power station. The core equipment of the hydroelectric power station includes turbines, generators, excitation systems, and governors, and its structure and inspection requirements differ from the three existing scenarios. However, the project timeline is tight, requiring model adaptation and deployment to be completed within a few weeks. Data situation: The newly collected labeled data for the hydroelectric power station consists of only 5,000 images, 200 hours of vibration waveforms, and 100 inspection records (a very small amount of data compared to the 200,000 images in the photovoltaic scenario).

[0074] First, a layered locking operation is performed on the existing model. For example, the parameters of existing scene adaptation modules in the basic large model layer and the scene adaptation layer are locked. This means that in subsequent backpropagation, the parameters of these layers will not be updated, and the training optimizer will only operate on the "released" layer parameters. Second, the core logic of similar scenarios is referenced. By analyzing the equipment characteristics of hydropower stations, it is found that their inspection logic is similar to that of some existing scenarios. For example, turbine runners and wind turbine blades are both large rotating components that require multi-angle observation, detection of blade / runner cracks, and monitoring of vibration. The transferable core logic parameters include the inspection strategy for rotating components (detection point generation algorithm, observation angle planning logic). Another example is generators and substation transformers, which are core power equipment that require monitoring of temperature, vibration, and partial discharge, and reading instrument data. The transferable core logic parameters include the power equipment status monitoring logic (multi-sensor fusion strategy, threshold judgment algorithm). According to statistics, the new hydropower adaptation module has a total of approximately 2 million parameters, of which 1.5 million (75%) are core logic parameters migrated from existing modules, and the remaining 500,000 (25%) are differential parameters to be trained. Furthermore, 5,000 newly collected hydropower station images and 200 hours of vibration waveform data were used to construct training and validation sets; for example, the turbine image training set had 2,000 images and the validation set had 500 images, the generator image training set had 1,600 images and the validation set had 400 images, and the instrument panel image training set had 400 images and the validation set had 100 images. After training, the new "hydropower adaptation module" was integrated into the embodied intelligence large model. The newly added hydropower module has only 2 million parameters, and the overall model size increased from 272 million to 274 million, an increase of less than 1%. Through knowledge distillation, a lightweight version was generated and distributed to the edge computing unit of the hydropower station inspection robot.

[0075] In an optional embodiment, the above method further includes: performing meta-learning training on the embodied intelligence large model, wherein the meta-learning training includes: constructing a cross-scenario task set: selecting non-standardized sub-scenario samples from a multi-scenario inspection sample library that provides prior knowledge for the embodied intelligence large model to construct a meta-learning task set; and meta-training: training the embodied intelligence large model using a model-independent meta-learning algorithm, so that the embodied intelligence large model learns to make targeted adjustments to the core adaptation parameters of the model for a single non-standardized sub-scenario through a single batch of samples, so as to extract general features across energy inspection scenarios and achieve adaptation to non-standardized sub-scenarios.

[0076] In the above embodiments, non-standardized sub-scenario samples are selected from the multi-scenario inspection sample library to construct a meta-learning task set. The embodied intelligence big model is trained by the model-independent meta-learning algorithm. This allows the embodied intelligence big model to complete the targeted adjustment of the core adaptation parameters of the model through a single batch of samples, extract the general features across energy inspection scenarios, realize the adaptation of non-standardized sub-scenarios, and improve the adaptability and versatility of the embodied intelligence big model in different energy inspection scenarios.

[0077] Leveraging the "learning-to-learn" characteristic of model-independent meta-learning algorithms, and combining non-standardized samples from a multi-scenario inspection sample library, the model learns the ability to quickly adapt to non-standardized scenarios with small samples. The core logic is as follows: First, non-standardized sub-scenario samples are selected from the multi-scenario inspection sample library supporting the model's prior knowledge. These sub-scenarios include equipment misalignment, complex terrain, and random obstacle distribution. This task set does not simply use standard, regulated scenario data; instead, it specifically selects non-standardized sub-scenario samples from the "multi-scenario inspection sample library." For example, several modules in a photovoltaic array are temporarily shaded, forming an atypical shadow distribution; a rare crack morphology appears on wind turbine blades, not recorded in the sample library; and in a substation, materials are temporarily piled around equipment due to construction, forming an unconventional inspection channel layout. These non-standardized scenarios collectively constitute a "meta-learning task set." Each task represents a small problem or challenge that the model needs to quickly adapt to. By constructing a training set containing numerous "small tasks," a "simulated exam" environment is provided for the model, forcing it to constantly face new situations and honing its ability to adapt quickly. A cross-scenario task set covering multiple energy inspection types is constructed, providing a standardized learning sample foundation for meta-training. Then, a model-agnostic meta-learning algorithm is used to conduct meta-training on the embodied intelligence model. During training, the model uses a single non-standardized sub-scenario from the cross-scenario task set as its learning unit, learning to adjust the core adaptation parameters of the model using single batches of samples. Ultimately, the model masters the general feature extraction and rapid adaptation logic for cross-energy inspection scenarios, enabling rapid adaptation to small samples of unseen non-standardized sub-scenarios. This capability is deeply integrated with the three-level architecture: the basic large model layer ensures general feature extraction, and the scenario adaptation layer completes targeted parameter adjustments, making non-standardized scenario adaptation more efficient. Through thousands of such "rapid adaptation" exercises, the model gradually learns: when facing a new problem, which features are key and how to adjust its internal parameters to adapt most quickly. Ultimately, the model learned the ability to extract common features across energy inspection scenarios. That is, regardless of the "non-standardized" situation faced by photovoltaic, wind power, or substations, it can quickly grasp the essence of the problem, find a general solution, and adapt rapidly based on this. After training, the model acquired a valuable "meta-capability." When encountering a completely new and unseen non-standardized sub-scenario in the real world, it only needs to utilize a single batch of samples from that scenario (i.e., a small amount of real-time observation data) to quickly adjust its core adaptation parameters, thereby achieving rapid adaptation to that sub-scenario.

[0078] In an optional embodiment, the method further includes: using the trained embodied intelligence large model as a teacher model in the cloud; using the lightweight version as a student model in the edge computing unit of the inspection robot; and using the soft labels output by the teacher model to supervise the training of the student model through knowledge distillation training, with the goal of minimizing the preset loss function between the output of the student model and the soft labels, to complete the generation and deployment of the lightweight version.

[0079] In the above embodiments, knowledge distillation training is used, with the trained embodied intelligence large model as the teacher model and the lightweight version as the student model. The soft labels output by the teacher model supervise the training of the student model, which can minimize the preset loss function between the student model output and the soft labels, thereby completing the generation and deployment of the lightweight version. This enables the edge computing unit of the inspection robot to run the model more efficiently, reduce resource consumption, and improve the overall performance and efficiency of the energy inspection system.

[0080] Leveraging knowledge distillation model compression technology, a collaborative training system of "cloud-based teacher model - edge-based student model" is constructed. This system achieves lightweight model compression and efficient edge deployment while preserving the core capabilities of the complete cloud-based model, adapting to the hardware resources and real-time requirements of the inspection robot's edge computing unit. The core logic is as follows: A fully embodied intelligent large-scale model trained in the cloud, possessing complete autonomous learning and decision-making capabilities for energy inspection, serves as the teacher model, rich in core knowledge such as feature extraction, scene matching, and autonomous learning. The model to be trained in the inspection robot's edge computing unit serves as the student model, acting as a lightweight version carrier. Through knowledge distillation training, the teacher model outputs soft labels containing soft probability distributions. These soft labels supervise the training process of the student model, transferring the teacher model's core knowledge to the student model. Training aims to minimize the preset loss function between the student model's output and the soft labels, allowing the student model to significantly compress the number of parameters and reduce computational complexity while fully inheriting the teacher model's core inspection capabilities. Ultimately, this results in the generation of a lightweight version and its deployment in the edge computing unit, achieving efficient migration of cloud-based model capabilities to the edge.

[0081] In an optional embodiment, the energy inspection system includes multiple inspection robots. The method further includes a group collaborative learning step: a cloud-based model center receives entity interaction feedback data, new inspection operation rules, and new fault judgment criteria uploaded by multiple inspection robots, and performs unified incremental training optimization on the embodied intelligence large model; the incrementally trained and optimized embodied intelligence large model is then lightweighted and distributed to the edge computing units of each inspection robot to update the lightweight version, achieving synchronous iterative upgrades of the model capabilities of all inspection robots. In this embodiment, a group collaborative learning mechanism is constructed based on multi-robot collaboration and a cloud-edge collaboration architecture. The principle is as follows: multiple inspection robots are used as distributed perception and learning terminals. The entity interaction feedback, newly generated inspection operation rules, and new fault judgment criteria generated by each robot during on-site inspections are uploaded to the cloud model center in real time. The cloud, based on global data, performs incremental training and optimization on the embodied intelligence model, transforming the local learning results of a single robot into global knowledge for the entire model. The optimized model is then lightweighted, uniformly distributed, and updated to the edge terminals of all robots, forming a closed-loop group learning system of "single-end learning → cloud aggregation → global iteration → full synchronization," achieving continuous and unified upgrades of model capabilities. This solves the problems of independent learning by a single inspection robot, lack of knowledge sharing, low learning efficiency, and high costs of repeated trial and error in related technologies.

[0082] This application also provides an inspection autonomous learning device based on an embodied intelligence large model, located in an energy inspection system, for executing the inspection autonomous learning method based on an embodied intelligence large model in any of the foregoing embodiments, such as... Figure 2 As shown, the device includes: The control module is used to control multiple sensors to collect raw data of the current inspection scenario and extract feature data corresponding to the raw data. The raw data includes scene data and / or equipment status data of the current inspection scenario. The matching module is used to match feature data with the pre-set prior knowledge associated with the energy inspection scenario in the lightweight version of the embodied intelligence big model; The trigger module is used to trigger autonomous learning when the similarity of the matching results is less than a preset threshold. Autonomous learning includes: for unknown scenarios, performing semantic segmentation on the original data to identify the inspection object, extracting the geometric features of the inspection object to generate a set of detection points, and planning and generating new inspection operation rules based on the detection points and obstacle information in the scene; or, for unknown faults, extracting fault features from the original data, associating them with prior knowledge about the equipment structure, and summarizing and generating new fault judgment criteria that include the correlation between fault features and equipment structure. The driving module is used to output action commands based on the successful matching results or new inspection operation rules or new fault judgment criteria generated by autonomous learning; drive the inspection robot to execute the action commands, and simultaneously collect entity interaction feedback data. The processing module is used to adjust the action instructions in real time based on the entity interaction feedback data to complete the action calibration. At the same time, it uploads the entity interaction feedback data, new inspection operation rules or new fault judgment criteria to the cloud model center to incrementally update the embodied intelligent big model and form a learning closed loop.

[0083] It should be noted that the devices or systems provided in the above embodiments are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept. Other device or system embodiments correspond to the aforementioned method embodiments. Other technical features are described in the previous embodiments and will not be repeated here.

[0084] This application also provides a computer-readable storage medium storing instructions that, when executed, perform the steps of any of the methods described above.

[0085] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0086] This application also discloses an electronic device. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0087] The communication bus 302 is used to enable communication between these components.

[0088] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0089] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0090] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the electronic device (such as a server) using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0091] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program based on an inspection autonomous learning method driven by an embodied intelligence large model.

[0092] exist Figure 3 In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call an application program stored in the memory 305, which is a patrol autonomous learning method driven by an embodied intelligence large model. When executed by one or more processors 301, the electronic device 300 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0093] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0094] In the various embodiments provided in this application, it should be understood that the disclosed apparatus or system can be implemented in other ways. For example, the apparatus or system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0095] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the disclosure herein.

[0096] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art that are not described in this disclosure.

Claims

1. A self-learning method for inspection based on an embodied intelligence large model, characterized in that, The method is applied to an energy inspection system, which includes a cloud-based model center and an inspection robot. The inspection robot is equipped with multiple sensors and an edge computing unit. The cloud-based model center deploys a dedicated embodied intelligent large-scale model for energy inspection, and the edge computing unit deploys a lightweight version of the embodied intelligent large-scale model. The system controls the multiple sensors to collect raw data of the current inspection scenario and extracts the feature data corresponding to the raw data. The raw data includes scene data and / or equipment status data of the current inspection scenario. The feature data is matched with the pre-defined prior knowledge associated with the energy inspection scenario in the lightweight version of the embodied intelligence big model; When the similarity of the matching results is less than a preset threshold, autonomous learning is triggered. The autonomous learning includes: for unknown scenarios, performing semantic segmentation on the original data to identify the inspection object, extracting the geometric features of the inspection object to generate a set of detection points, and planning and generating new inspection operation rules based on the detection points and obstacle information in the scenario; or, for unknown faults, extracting fault features from the original data, associating them with prior knowledge about the equipment structure, and summarizing and generating new fault judgment criteria that include the correlation between fault features and equipment structure. Based on the successful matching result, the new inspection operation rules generated by the autonomous learning, or the new fault judgment criteria, an action command is output; the inspection robot is driven to execute the action command, and entity interaction feedback data is collected simultaneously. The action instructions are adjusted in real time based on the entity interaction feedback data to complete the action calibration. At the same time, the entity interaction feedback data, the new inspection operation rules or the new fault judgment criteria are uploaded to the cloud model center to incrementally update the embodied intelligent big model and form a learning closed loop. The prior knowledge is obtained by structured extraction of inspection data from a multi-scenario inspection sample library. The multi-scenario inspection sample library is constructed from multimodal perception data collected under normal and fault conditions of photovoltaic modules, wind turbine blades, and substation equipment. The multimodal perception data includes at least one of image, point cloud, and acoustic data. The prior knowledge includes quantitative parameters of equipment features, quantitative indicators of operational safety boundaries, and basic thresholds for fault features. To generate new inspection operation rules for unknown scenarios, the process includes: controlling the inspection robot to perform multi-angle observations at a speed not exceeding a preset moving speed threshold to collect complete scene data; retrieving feature samples similar to the complete scene data from the multi-scene inspection sample library using the K-nearest neighbor algorithm; and autonomously generating quantified operation rules that include inspection path planning and detection point selection using a linear interpolation algorithm based on the similar feature samples and the complete scene data. Once the validity of the quantified operation rules is verified through entity interaction, these quantified operation rules are identified as the new inspection operation rules.

2. The method according to claim 1, characterized in that, The criteria for generating new fault judgments based on the induction of unknown faults specifically include: The fault depth features in the original data are extracted by a convolutional neural network, and the new fault judgment criteria are generated autonomously by following these steps: the measured value of the fault depth features is used as the midpoint of the interval of the new fault judgment criteria; the criterion interval width of similar faults in the multi-scenario inspection sample library is reused to calculate the upper and lower limits of the new fault judgment criteria; the multi-feature logical judgment relationship of similar faults is reused, and the upper and lower limits are corrected by combining sensor error and individual equipment differences.

3. The method according to claim 1, characterized in that, The action commands are adjusted in real time based on the entity interaction feedback data to complete the action calibration, specifically including: The entity interaction feedback data is compared with preset operation accuracy thresholds, equipment contact pressure thresholds, and robot motion posture deviation thresholds to identify motion deviations. Based on the motion deviation, at least one of the motion coordinates, end effector angle, and contact pressure parameters in the motion command is corrected by a PID control algorithm to generate a calibration motion command and drive the inspection robot to execute it until the entity interaction feedback data falls within a preset qualified range defined by the operation accuracy threshold, the equipment contact pressure threshold, and the robot motion posture deviation threshold.

4. The method according to claim 1, characterized in that, The embodied intelligence big model adopts a three-tier architecture consisting of a basic big model layer, a scenario adaptation layer, and an interaction decision layer. The basic large model layer is built on the Transformer architecture and is used to extract the underlying features common to all energy inspection scenarios; The scenario adaptation layer sets up independent adaptation modules for different types of energy inspection scenarios, and has built-in structured prior knowledge for each scenario. The energy inspection scenarios include the following different types: photovoltaic, wind power and substation. The interactive decision layer integrates an entity feedback interface, which is used to receive the entity interaction feedback data and calibrate the action command accordingly.

5. The method according to claim 4, characterized in that, The method further includes: performing hierarchical transfer learning on the embodied intelligence large model to adapt it to new energy inspection scenarios, wherein the hierarchical transfer learning includes: Layered locking: Lock the model parameters of the basic large model layer and release the module parameters of the scene adaptation layer corresponding to the new energy inspection scenario; Targeted migration: The core logic parameters in the scene adaptation layer of the trained scene are migrated to the corresponding adaptation module of the new energy inspection scene as initial parameters; Parameter-specific training: Using the dataset of the new energy inspection scenario, the differentiated parameters in the scenario adaptation layer of the new energy inspection scenario are trained and updated, while keeping the core logic parameters of the transfer in the scenario adaptation layer unchanged, and the model parameters of the basic large model layer are not trained and updated.

6. An inspection autonomous learning device driven by an embodied intelligence large model, characterized in that, Located in the energy inspection system, the method for executing the inspection autonomous learning method based on embodied intelligence large model driven by any one of claims 1 to 5 includes: The control module is used to control the multi-sensor to collect raw data of the current inspection scene and extract feature data corresponding to the raw data, wherein the raw data includes scene data and / or equipment status data of the current inspection scene; The matching module is used to match the feature data with the pre-set prior knowledge associated with the energy inspection scenario in the lightweight version of the embodied intelligent big model; The triggering module is used to trigger autonomous learning when the similarity of the matching results is less than a preset threshold. The autonomous learning includes: for unknown scenarios, performing semantic segmentation on the original data to identify the inspection object, extracting the geometric features of the inspection object to generate a set of detection points, and planning and generating new inspection operation rules based on the detection points and obstacle information in the scenario; or, for unknown faults, extracting fault features from the original data, associating them with prior knowledge about the equipment structure, and summarizing and generating new fault judgment criteria that include the correlation between fault features and equipment structure. The driving module is used to output action commands based on the successful matching result, the new inspection operation rules generated by the autonomous learning, or the new fault judgment criteria; drive the inspection robot to execute the action commands, and simultaneously collect entity interaction feedback data. The processing module is used to adjust the action instructions in real time according to the entity interaction feedback data to complete the action calibration. At the same time, it uploads the entity interaction feedback data, the new inspection operation rules or the new fault judgment criteria to the cloud model center to incrementally update the embodied intelligent big model and form a learning closed loop. The prior knowledge is obtained by structured extraction of inspection data from a multi-scenario inspection sample library. The multi-scenario inspection sample library is constructed from multimodal perception data collected under normal and fault conditions of photovoltaic modules, wind turbine blades, and substation equipment. The multimodal perception data includes at least one of image, point cloud, and acoustic data. The prior knowledge includes quantitative parameters of equipment features, quantitative indicators of operational safety boundaries, and basic thresholds for fault features. The triggering module is used to generate new inspection operation rules for unknown scenarios in the following ways: controlling the inspection robot to perform multi-angle observations at a speed not exceeding a preset moving speed threshold to collect complete scene data; retrieving feature samples similar to the complete scene data from the multi-scene inspection sample library using the K-nearest neighbor algorithm; based on the similar feature samples and the complete scene data, autonomously generating quantitative operation rules that include inspection path planning and detection point selection using a linear interpolation algorithm; and determining the validity of the quantitative operation rules through entity interaction verification as the new inspection operation rules.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 5.

Images