Virtual operation fault training method, system and equipment for live working robot

Abstract

The invention provides a virtual operation fault training method, system and device for a hot-line work robot, and the method comprises the steps: collecting the operation data of the hot-line work robot in a real operation environment through a data collection module; a virtual operation environment consistent with the real operation environment is established based on the operation data through an operation management module, and a target operation process is executed in the virtual operation environment; simulating at least one fault scene of the hot-line work robot in the virtual work environment based on a target fault simulation stage and a corresponding target fault injection mechanism in the execution process of the target work flow through a fault management module; and responding to manual intervention operation executed by an operator aiming at the fault event triggered in each fault scene through a user management module. According to the invention, the training cost and the time cost can be greatly reduced, the training effect is improved, and the emergency processing capability and the operation experience of an operator during real fault processing can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of live working, and in particular to a method, system and equipment for training virtual working faults of a live working robot. Background Art

[0002] In the current power operation environment, live-line operation robots have gradually become important auxiliary tools, and their safety and efficiency are crucial to power maintenance. After the existing live-line operation robots complete their operations, they will upload relevant data to the cloud platform, which will score and analyze the operations through the processing capabilities of the cloud platform. These data usually include detailed records of the operation steps, the joint angles of the robot arm, point selection information, point cloud data and other key information, which are of great significance for understanding the operation status, optimizing the operation process and improving the operation quality.

[0003] However, when these data are integrated into the digital twin platform, although the operation process can be intuitively displayed and in-depth analysis can be performed, the existing system still has obvious deficiencies in terms of operator training. Traditional training systems are often based on theoretical knowledge explanations or simple simulation operations, and it is difficult to simulate various complex situations and failures that may occur in actual operating environments. This results in operators often lacking sufficient emergency response capabilities and operating experience when facing real failures, thus affecting the safety and efficiency of operations. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a virtual operation fault training method, system and equipment for live working robots, which can not only greatly reduce the training cost and time cost and improve the training effect, but also help to improve the operator's emergency handling ability and operating experience when dealing with real faults.

[0005] In a first aspect, an embodiment of the present invention provides a virtual operation fault training method for a live operation robot, the method is applied to a virtual operation fault training system, the virtual operation fault training system includes a data acquisition module, an operation management module, a fault management module and a user management module, the method includes: Through the data acquisition module, the operation data of the live working robot in the real working environment is collected; Through the operation management module, a virtual operation environment consistent with the real operation environment is built based on the operation data, and the target operation process is executed in the virtual operation environment; Through the fault management module, during the execution of the target operation process, based on the target fault simulation phase and its corresponding target fault injection mechanism, at least one fault scenario of the live operation robot is simulated in the virtual operation environment; Through the user management module, respond to the operator's manual intervention operations for the fault events triggered in each fault scenario to complete the fault training corresponding to the target operation process.

[0006] In one embodiment, the fault management module is configured with a candidate fault simulation stage and a candidate fault injection mechanism; Different candidate fault simulation stages correspond to fault scenarios of different complexity; The candidate fault injection mechanisms are multiple trigger mechanisms of fault events included in the fault scenario. The trigger mechanisms are divided into rule-based trigger mechanisms and / or probability-based trigger mechanisms; among them, the rule-based trigger mechanism triggers the fault according to preset conditions, and the probability-based trigger mechanism triggers the fault according to a preset probability distribution.

[0007] In one embodiment, the rule-based trigger mechanism includes time trigger, job step trigger and job progress trigger; wherein, the time trigger is to trigger a fault when the target job process is executed to a specified time point, the job step trigger is to trigger a fault when the target job process is executed to a specified job step, and the job progress trigger is to trigger a fault when the target job process is executed to a specified job progress; The probability-based trigger mechanism includes random probability triggering and conditional probability triggering; wherein, the random probability triggering is to determine the first fault triggering probability according to the preset probability distribution, and trigger the fault according to the first fault triggering probability at each time point or each operation step in the target operation process; the conditional probability triggering is to calculate the second fault triggering probability based on multiple preset fault triggering conditions, and trigger the fault according to the second fault triggering probability; The trigger mechanism based on rules and probability includes rule priority triggering and probability adjustment triggering; wherein, rule priority triggering means that the priority of the rule-based trigger mechanism is higher than the priority of the probability-based trigger mechanism, and probability adjustment triggering means adjusting the first fault trigger probability or the second fault trigger probability according to the rules, and triggering the fault according to the adjusted first fault trigger probability or the second fault trigger probability.

[0008] In one embodiment, through the fault management module, during the execution of the target operation process, based on the target fault simulation phase and its corresponding target fault injection mechanism, at least one fault scenario of the live working robot is simulated in the virtual working environment, including: In response to a selection operation on a candidate fault simulation stage, a target fault simulation stage is determined; Through the fault management module, during the execution of the target operation process, the target fault injection mechanism corresponding to the target fault simulation stage is randomly determined from the candidate fault injection mechanisms, so as to simulate at least one fault scenario of the live working robot in the virtual working environment based on the target fault injection mechanism.

[0009] In one embodiment, the virtual operation fault training system is communicatively connected to the digital twin system, and the digital twin system is configured with a robot model corresponding to the live operation robot; the method further includes: The execution log corresponding to the target job process and the fault attributes corresponding to each fault scenario are sent to the digital twin system, so that the digital twin system can control the robot model to execute actions based on the execution log and fault attributes, and display the action process of the robot model.

[0010] In one embodiment, the method further comprises: Based on the preset intervention time limit and operation scoring criteria, the scoring result corresponding to the manual intervention operation is determined.

[0011] In one embodiment, the method further comprises: If the operation data changes, the new operation data of the live operation robot in the real operation environment is collected through the data acquisition module, the new operation data is denoised and filtered, the processed new operation data is temporarily cached, and all the new operation data after temporary caching is synchronized to the virtual operation environment.

[0012] In a second aspect, an embodiment of the present invention further provides a virtual operation fault training system for a live operation robot, the virtual operation fault training system comprising a data acquisition module, an operation management module, a fault management module and a user management module: The data acquisition module is used to collect the operation data of the live working robot in the real working environment; The operation management module is used to build a virtual operation environment consistent with the real operation environment based on the operation data, and execute the target operation process in the virtual operation environment; The fault management module is used to simulate at least one fault scenario of the live working robot in a virtual working environment based on the target fault simulation phase and its corresponding target fault injection mechanism during the execution of the target working process; The user management module is used to respond to the manual intervention operations performed by the operator for the fault events triggered in each fault scenario to complete the fault training corresponding to the target operation process.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided by the first party.

[0015] An embodiment of the present invention provides a virtual operation fault training method, system and equipment for a live working robot. The virtual operation fault training system includes a data acquisition module, a job management module, a fault management module and a user management module. First, the data acquisition module is used to collect the operation data of the live working robot in a real working environment; then, the job management module is used to build a virtual working environment consistent with the real working environment based on the operation data, and the target operation process is executed in the virtual working environment; then, during the execution of the target operation process, the fault management module is used to simulate at least one fault scenario of the live working robot in the virtual working environment based on the target fault simulation stage and its corresponding target fault injection mechanism; finally, the user management module is used to respond to the operator's manual intervention operations performed on the fault events triggered in each fault scenario to complete the fault training corresponding to the target operation process. The above method provides a complete data processing process, which covers the collection of operation data from the real operation environment and synchronization to the virtual operation fault training platform to build a virtual operation environment consistent with the real operation environment, and use the target fault simulation stage and its corresponding target fault injection mechanism in the virtual operation environment to simulate the fault scenario to train the operator's fault handling ability. The embodiment of the present invention can more realistically simulate the faults that may occur during live operation and guide the operator to perform manual intervention operations on the faults that occur, thereby realizing fault training, which can not only greatly reduce the training cost and time cost and improve the training effect, but also help to improve the operator's emergency handling ability and operating experience when handling real faults.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a virtual operation fault training method for a live operation robot provided by an embodiment of the present invention; Figure 2 A connection diagram of a virtual operation fault training system provided by an embodiment of the present invention; Figure 3 A grounding ring preparation operation flow chart provided in an embodiment of the present invention; Figure 4 A fault rule diagram provided by an embodiment of the present invention; Figure 5 A data flow diagram provided by an embodiment of the present invention; Figure 6 A training operation flow chart provided for an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, the relevant technologies have at least the following problems: 1. Insufficient emergency handling capabilities: When faced with sudden failures, operators need to quickly and accurately determine the cause of the failure and take corresponding treatment measures. However, due to the limitations of traditional training methods, operators often lack sufficient emergency handling capabilities. The embodiments of the present invention can help operators quickly locate the cause of the failure and take corresponding treatment measures by providing treatment solutions and suggestions for various failures, thereby improving their emergency handling capabilities; 2. High training costs and long time: Traditional training methods usually require a lot of manpower, material resources and time, and the results are not satisfactory. The embodiments of the present invention can greatly reduce training costs and time costs and improve training effects by providing a virtual operating environment for fault simulation and training; 3. Different operators may have different learning abilities and operating habits, and traditional training methods are difficult to meet the needs of personalized training. The embodiments of the present invention can provide operators with personalized training suggestions and improvement plans by recording and analyzing the operating data and performance of operators in a virtual operating environment, thereby better meeting their learning needs.

[0022] Based on this, the implementation of the present invention provides a virtual operation fault training method, system and equipment for live working robots, which can not only greatly reduce the training cost and time cost and improve the training effect, but also help to improve the operator's emergency handling ability and operating experience when dealing with real faults.

[0023] To facilitate understanding of this embodiment, a virtual operation fault training method for a live operation robot disclosed in an embodiment of the present invention is first described in detail. The method is applied to a virtual operation fault training system. The virtual operation fault training system includes a data acquisition module, an operation management module, a fault management module and a user management module. Figure 1 The flowchart of a virtual operation fault training method for a live operation robot is shown in FIG. 1 , and the method mainly includes the following steps S102 to S108: Step S102, collecting operation data of the live working robot in a real working environment through a data acquisition module.

[0024] In one example, a virtual operation fault training system can be connected to a real robot system of a live operation robot through an interface, thereby collecting data generated when the live operation robot performs live operations in a real operating environment. This data is the operation data. After denoising and filtering, the operation data is cached in the virtual operation fault training system.

[0025] Step S104: Building a virtual operating environment consistent with the real operating environment based on the operating data through the operating management module, and executing the target operating process in the virtual operating environment.

[0026] In one example, a plurality of operation types and corresponding operation processes may be configured in the operation management module. The operation management module may call cached operation data to build a virtual operation environment consistent with the real operation environment, and respond to selection operations for operation types and operation processes to determine a target operation process, and then execute the target operation process in the virtual operation environment.

[0027] Step S106, through the fault management module, during the execution of the target operation process, based on the target fault simulation phase and its corresponding target fault injection mechanism, at least one fault scenario of the live working robot is simulated in the virtual working environment.

[0028] In one example, all possible fault scenarios are listed, including but not limited to robot arm protection, emergency stop, command execution failure, stop operation, pause operation, skip operation, step back, etc., and clear trigger conditions and manifestations are defined for each fault scenario. The fault management module is configured with candidate fault simulation stages and candidate fault injection mechanisms. Different candidate fault simulation stages correspond to fault scenarios of different complexity. The candidate fault injection mechanism is a variety of trigger mechanisms for fault events contained in the fault scenario. The rules and probabilities involved in the candidate fault injection mechanism are related to the candidate fault simulation stage to which it belongs. Before executing the target operation process, the selection operation for the candidate fault simulation stage can be performed accordingly to determine the target fault simulation stage, and the target operation process is executed on this basis, so that the target fault injection mechanism corresponding to the target fault simulation stage is randomly selected during the execution process to more realistically simulate the fault scenario in the virtual operation environment.

[0029] Step S108, through the user management module, responding to the manual intervention operation performed by the operator for the fault event triggered in each fault scenario, so as to complete the fault training corresponding to the target operation process.

[0030] In one example, when a live-line working robot in a virtual working environment encounters a fault, it is necessary to simulate the human intervention process in real situations. For example, when a live-line working robot encounters a failure in executing an instruction, it can simulate the operator's actions such as resending the instruction through the associated terminal or choosing to skip the current step.

[0031] The above method provides a complete data processing process, which covers the collection of operation data from the real operation environment and synchronization to the virtual operation fault training platform to build a virtual operation environment consistent with the real operation environment, and use the target fault simulation stage and its corresponding target fault injection mechanism in the virtual operation environment to simulate the fault scenario to train the operator's fault handling ability. The embodiment of the present invention can more realistically simulate the faults that may occur during live operation and guide the operator to perform manual intervention operations on the faults that occur, thereby realizing fault training, which can not only greatly reduce the training cost and time cost and improve the training effect, but also help to improve the operator's emergency handling ability and operating experience when handling real faults.

[0032] For ease of understanding, the embodiment of the present invention first explains the virtual operation fault training system. Figure 2 The connection diagram of a virtual operation fault training system is shown in the figure. The whole system is divided into data acquisition module, operation management module, fault management module, user management module, etc. The real operation data is transmitted to the virtual operation environment through the interface. By simulating the operation and triggering the fault, the operator is trained and the fault is handled. Specifically: (1) The real operation data is synchronized to the virtual operation fault training system through the data acquisition module. The data acquisition module first collects the operation data in the real operation environment, filters it, and caches the filtered data.

[0033] (2) The job management module applies the job data collected by the data acquisition module. The job management module calls the cached job data and responds to the selection operation for the job type and job process, thereby executing the target job process in the virtual job environment built based on the job data, which facilitates the subsequent fault management module to perform fault simulation and fault handling guidance in the process of executing the target job process.

[0034] To facilitate understanding of the aforementioned operation process, the embodiment of the present invention provides the following Figure 3The flowchart of a grounding ring preparation operation shown includes: (2.1) A virtual operation is performed by taking the installation of a grounding ring operation type as an example. First, the grounding ring is prepared, and the robot arm identifies the line. (2.2) The stripping tool on the robot arm strips the line. (2.3) When the tool strips the line, the correct line point is not identified, and manual intervention is performed. Return to the previous step to re-identify the line point or manually intervene to the correct point. (2.4) After the tool strips the line, the right arm of the robot and the tool are withdrawn. (2.5) The left arm of the robot moves the grounding ring to the line stripping position. When performing this step, a prompt indicating that the command recognition failed is displayed. Check the connection between the iPad and the robot and the network communication. (2.6) Connect the grounding ring to the stripped position through the tool on the left arm of the robot. (2.7) The robot is reset to the initial state and the operation process ends.

[0035] (3) Set fault triggering, fault handling and analysis during the operation process. The fault management module is mainly used to simulate faults and provide fault handling guidance during the execution of the target operation process.

[0036] (4) Operator operation process and handling of sudden failures. The user management module is used to provide fault handling instructions to operators, conduct operation training and operation evaluation for operators.

[0037] In actual applications, when building a virtual operation fault training system, it is necessary to design and implement a complete data processing process, which covers the collection of key operation and fault data from the real operation environment, and the use of encryption, compression and real-time transmission technology to ensure the security and timeliness of the data, and then decrypt, decompress and synchronize the data in the virtual operation fault training system, and finally use these data to build operation scenarios, simulate faults and analyze them in the virtual operation environment. This process is designed to ensure that the virtual platform can reflect the status of the real live operation robot in real time and accurately, and provide a solid data foundation for operator training, robot optimization and fault handling strategy formulation.

[0038] For the aforementioned fault management module, the fault management module is configured with a candidate fault simulation phase and a candidate fault injection mechanism. Specifically: (a) Different candidate fault simulation stages correspond to fault scenarios of different complexity. In one example, the fault simulation is divided into three stages: primary, intermediate, and advanced, and each stage contains fault scenarios of different difficulty and complexity.

[0039] (a1) Primary stage: Simulate simple and easily identifiable faults, such as warning errors, to familiarize operators with the basic process of fault identification and handling.

[0040] (a2) Intermediate stage: The number and complexity of faults increase, such as command execution failure, high temperature, and simultaneous failure of multiple sensors, requiring operators to be able to quickly judge and handle multiple faults simultaneously.

[0041] (a3) Advanced stage: Simulate complex fault scenarios, such as protective stops or unknown faults of the robot arm, to test the operator's adaptability and troubleshooting capabilities.

[0042] (b) The candidate fault injection mechanisms are multiple trigger mechanisms of the fault events included in the fault scenario. The trigger mechanisms are divided into rule-based trigger mechanisms and / or probability-based trigger mechanisms.

[0043] (b1) The rule-based trigger mechanism triggers faults according to preset conditions. Specifically, the trigger mechanism triggers specific faults according to preset conditions and rules. These rules are defined based on time, specific job steps, and job progress. That is, the rule-based trigger mechanism includes time trigger, job step trigger, and job progress trigger: (b1.1) Time triggering means triggering a fault when the target operation process is executed to a specified time point, that is, setting the robot operation in the virtual operation environment to trigger a fault at a certain time point.

[0044] (b1.2) Operation step triggering is to trigger a fault when the target operation process is executed to the specified operation step. That is, when the robot executes a key operation step, such as wire grabbing, wire stripping, etc., a fault can be triggered to test the robot's response.

[0045] (b1.3) Job progress triggering is to trigger a fault when the target job process is executed to a specified job progress, that is, to trigger a fault through the job progress of each job type.

[0046] (b2) The probability-based trigger mechanism triggers the fault according to the preset probability distribution. This method can simulate more random fault conditions and improve the comprehensiveness and authenticity of the training. Specifically, the probability-based trigger mechanism includes random probability triggering and conditional probability triggering: (b2.1) Random probability triggering is to determine the first fault triggering probability according to the preset probability distribution, and trigger the fault according to the first fault triggering probability at each time point or each operation step in the target operation process. That is, at each operation step or time point, it is decided whether to trigger the fault according to a certain probability distribution (based on the environment where the actual robot operation fails).

[0047] (b2.2) Conditional probability triggering is to calculate the second fault trigger probability based on multiple preset fault trigger conditions, and trigger the fault according to the second fault trigger probability. In other words, the probability of triggering a fault is calculated by combining multiple conditions. For example, when the robot is in a high temperature environment (such as over 40°C) and the battery power is less than 30%, and it is performing high-energy operations (such as continuous use of high-power tools), the probability of triggering multiple faults (such as power outages and tool performance degradation) increases significantly.

[0048] (b3) Combine rule-based and probability-based trigger mechanisms to achieve more flexible and diverse fault simulation. Specifically, the rule-based and probability-based trigger mechanisms include rule-priority triggering and probability-adjusted triggering: (b3.1) Rule-based triggering is a trigger mechanism based on rules that has a higher priority than a trigger mechanism based on probability. That is, the fault is triggered first according to the preset rules. If no rule conditions are currently met, the fault is triggered randomly according to the probability distribution.

[0049] (b3.2) Probability adjustment trigger (also called probability adjustment rule) is to adjust the first fault trigger probability or the second fault trigger probability according to the rule, and trigger the fault according to the adjusted first fault trigger probability or the second fault trigger probability. That is, the probability of fault triggering is adjusted according to the rule. For example, in a specific operation step, a higher fault trigger probability can be set to focus on testing the fault handling capability of the step.

[0050] Based on this, an embodiment of the present invention provides an implementation method for simulating fault scenarios, which first responds to a selection operation for a candidate fault simulation stage to determine a target fault simulation stage; then, through a fault management module, during the execution of a target job process, a target fault injection mechanism corresponding to the target fault simulation stage is randomly determined from the candidate fault injection mechanisms to simulate at least one fault scenario of a live working robot in a virtual working environment based on the target fault injection mechanism.

[0051] For example, see Figure 4 A fault rule diagram is shown, including: listing all possible fault scenarios; defining trigger conditions and manifestations; rule-based fault injection and probability-based fault injection, including time triggering, conditional probability triggering and random probability triggering; rule priority triggering and probability adjustment rules; simulating fault information feedback; simulating manual intervention process; recording job logs; synchronizing the real working environment; fault analysis and optimization.

[0052] In one example, the virtual operation fault training system is connected to the digital twin system, and the digital twin system is equipped with a robot model corresponding to the live operation robot. Specifically, the execution log corresponding to the target operation process and the fault attributes corresponding to each fault scenario are sent to the digital twin system, so that the digital twin system controls the robot model to perform actions based on the execution log and fault attributes, and displays the action process of the robot model.

[0053] Please continue to see Figure 4 , after triggering a fault event, it is necessary to simulate fault information feedback, simulate manual intervention process, and record operation logs. Simulating fault information feedback means: when simulating fault events in a virtual operation environment, it is necessary to feed back fault information to the digital twin platform in real time, including fault type, occurrence time, fault location, etc. This information is used for subsequent fault analysis and processing. Simulating manual intervention process means: when a robot in a virtual operation environment encounters a fault, it is necessary to simulate the process of manual intervention in real situations. For example, when a robot encounters a failure in executing an instruction, it can simulate the operator resending instructions through an iPad or choosing to skip the current step. Recording operation logs means: when simulating operations in a virtual operation environment, it is necessary to record complete operation logs, including the execution status of each operation step, fault triggering, and processing process, etc. These logs are used for subsequent fault analysis and training the robot's ability to cope with faults.

[0054] In one example, if the operation data changes, the new operation data of the live working robot in the real working environment is collected through the data acquisition module, the new operation data is denoised and filtered, the processed new operation data is temporarily cached, and all the new operation data after temporary caching is synchronized to the virtual working environment.

[0055] Please continue to see Figure 4 The synchronization shown is the real working environment. In order to keep the virtual working environment synchronized with the real working environment, it is necessary to regularly synchronize the robot's working data, fault data, etc. in the real working environment to the virtual working environment, so as to update the parameters and rules of the virtual working environment to make it closer to the real situation. For details, please refer to Figure 5 The data flow diagram shown includes: a data acquisition module collects real data through an interface, reduces noise and filters the collected data, temporarily caches the sorted data, and synchronizes all cached data to the virtual operation environment.

[0056] In one example, based on the preset intervention time limit and operation scoring criteria, the scoring result corresponding to the manual intervention operation is determined. Figure 4The fault analysis and optimization shown in the figure simulates the fault in the virtual operation environment, and then uses the digital twin platform to analyze the fault to find out the cause and solution of the fault. The analysis results are used to optimize the robot's operation process and fault handling strategy.

[0057] Combination Figures 3 to 5 , mainly through the simulation of operations, a process of triggering faults and handling faults during the operation process. It includes: 1) Collect historical real data into the virtual working environment through the data acquisition module.

[0058] 2) The data acquisition module reduces noise and filters redundant and incomplete data.

[0059] 3) The operator follows the engineering staff to perform virtual operations.

[0060] 4) The engineer selects the type of work to be performed and installs the grounding ring.

[0061] 5) The engineers perform the work according to the inherent working steps, including grounding ring preparation (starting the work), wire stripping identification, wire stripping with the right arm, wire stripping with tools, wire stripping withdrawal, grounding ring preparation, tool grounding ring, and robot reset.

[0062] 6) When the engineer was performing the operation process, a fault was triggered. The fault was triggered by event triggering and random probability triggering, which were respectively in the tool stripping step and the grounding ring preparation step.

[0063] 7) The digital twin page can display robot actions, key information, and operation logs in real time.

[0064] 8) According to the prompt information in the digital twin operation log, the tool did not meet expectations and the instruction execution recognition failed. It can be determined that there are two reasons for the failure. The first one is that the tool did not recognize the point of the line. There are two ways to solve the problem. Return the operation process to the previous step, re-identify the correct point, and move the tool to the correct point through manual intervention. The second main reason for the failure is to check the connection between the iPad and the robot and check the network communication of the robot.

[0065] 9) When the job is completed, the robot will return to its original position.

[0066] 10) The operator then works in a virtual working environment and quickly understands the fault handling process by triggering different faults.

[0067] 11) Score and analyze each operator’s virtual operation and fault handling methods.

[0068] Furthermore, the embodiment of the present invention also provides an operation training process, including: familiarizing with the operation process of each operation type and determining all fault types; simulating in stages; explaining the standard process of fault handling in detail to the operator, including steps such as fault identification, cause analysis, handling measures and verification; analyzing the fault operations that have been operated, so that the operator can understand the handling methods and calculations of different faults. During the analysis, operation training is carried out in combination with the simulation situation in the virtual operation environment; practical exercises are carried out in the virtual operation environment, and the operator personally handles various faults. Time limits and equal-division standards are set during the operation exercises.

[0069] For details, please refer to Figure 6 A training operation flow chart shown includes: training preparation stage; determining fault types and scenarios; entering fault simulation and experience stage; primary stage simulation, intermediate stage simulation, and advanced stage simulation; fault handling training; fault handling process explanation, case analysis, and practical exercises; evaluation and other processes. For example, 1) set up a real working environment in a virtual environment, such as installing a grounding ring. 2) When the tool fails to strip the wire during the operation, determine the cause of the operation failure. 3) Through the display of real-time data on the digital twin page, the cause of this failure is that the robot fails to select a point, resulting in the tool being unable to strip the wire. 4) By operating the robot iPad to step back, the operation returns to the previous link, re-select the line, and continue the operation. 5) After the operation, analyze and train the faults that occurred in this operation, point out that the key to the problem is that the robot fails to correctly identify the line, and propose a solution because the joint angle of the robot arm does not correspond to the corresponding line, and adjust the angle between the robot arm and the line. 6) The trainee operates according to the above operation steps to trigger the fault and analyze it. 7) Evaluate each operator’s work and point out operational problems and deficiencies in troubleshooting.

[0070] The entire troubleshooting process is gradually deepened from elementary to advanced, through theoretical learning, simulation experience, case analysis to actual operation drills, to help trainees or handlers fully master the methods and skills of troubleshooting. Each stage has its specific tasks and steps, and has corresponding evaluations and summaries.

[0071] In summary, the virtual operation fault training method for a live working robot provided by the embodiment of the present invention has at least the following characteristics: 1. Improved the authenticity of fault simulation. By simulating various conditions and parameters in the real working environment, it is possible to simulate various fault scenarios that the robot may encounter in actual operation, providing a more realistic test environment for testing the robot's fault handling capabilities.

[0072] (ii) Enhanced reliability and stability of robots. Pre-simulating and testing various fault conditions in a virtual operating environment helps to timely discover and improve potential problems in robot design, thereby reducing the occurrence of faults in actual applications.

[0073] (III) Virtual fault simulation reduces testing costs and risks. Compared with physical testing, virtual testing does not require a lot of time and resources, and avoids safety issues such as robot damage or personal injury caused by testing, thereby reducing testing costs and potential risks.

[0074] (IV) Accelerates the development and optimization process of robots. Through virtual testing, problems can be quickly discovered and iterative optimization can be performed, shortening the development cycle and improving development efficiency.

[0075] (V) Virtual fault simulation provides a safe training environment for operators. Simulating operations in a virtual environment helps operators become familiar with robot operation and fault handling processes, improving operational proficiency and safety.

[0076] On the basis of the above-mentioned embodiments, the embodiments of the present invention provide a virtual operation fault training system for a live operation robot, the virtual operation fault training system includes a data acquisition module, an operation management module, a fault management module and a user management module: The data acquisition module is used to collect the operation data of the live working robot in the real working environment; The operation management module is used to build a virtual operation environment consistent with the real operation environment based on the operation data, and execute the target operation process in the virtual operation environment; The fault management module is used to simulate at least one fault scenario of the live working robot in a virtual working environment based on the target fault simulation phase and its corresponding target fault injection mechanism during the execution of the target working process; The user management module is used to respond to the manual intervention operations performed by the operator for the fault events triggered in each fault scenario to complete the fault training corresponding to the target operation process.

[0077] The above system provides a complete data processing process, which covers the collection of operation data from the real operation environment and synchronization to the virtual operation fault training platform to build a virtual operation environment consistent with the real operation environment, and use the target fault simulation stage and its corresponding target fault injection mechanism in the virtual operation environment to simulate the fault scenario to train the operator's fault handling ability. The embodiment of the present invention can more realistically simulate the faults that may occur during live operation and guide the operator to perform manual intervention operations on the faults that occur, thereby realizing fault training, which can not only greatly reduce the training cost and time cost and improve the training effect, but also help to improve the operator's emergency handling ability and operating experience when handling real faults.

[0078] The implementation principle and technical effects of the system provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0079] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0080] Figure 7 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 70, a memory 71, a bus 72 and a communication interface 73, wherein the processor 70, the communication interface 73 and the memory 71 are connected via the bus 72; the processor 70 is used to execute an executable module stored in the memory 71, such as a computer program.

[0081] The memory 71 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0082] The bus 72 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0083] Among them, the memory 71 is used to store programs, and the processor 70 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 70 or implemented by the processor 70.

[0084] The processor 70 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 70. The above processor 70 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 71, and the processor 70 reads the information in the memory 71 and completes the steps of the above method in combination with its hardware.

[0085] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0086] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A virtual operation fault training method for a live operation robot, characterized in that: The method is applied to a virtual operation fault training system, which includes a data acquisition module, an operation management module, a fault management module and a user management module. The method includes: The data collection module is used to collect the operation data of the live working robot in a real working environment; By means of the operation management module, a virtual operation environment consistent with the real operation environment is built based on the operation data, and a target operation process is executed in the virtual operation environment; By means of the fault management module, during the execution of the target operation process, based on the target fault simulation phase and its corresponding target fault injection mechanism, at least one fault scenario of the live working robot is simulated in the virtual working environment; The user management module responds to the manual intervention operation performed by the operator for the fault event triggered in each fault scenario to complete the fault training corresponding to the target operation process.

2. The virtual operation fault training method for live working robots according to claim 1 is characterized in that: The fault management module is configured with a candidate fault simulation stage and a candidate fault injection mechanism; Different candidate fault simulation stages correspond to fault scenarios of different complexity; The candidate fault injection mechanism is a variety of trigger mechanisms for the fault events included in the fault scenario, and the trigger mechanism is divided into a rule-based trigger mechanism and / or a probability-based trigger mechanism; wherein the rule-based trigger mechanism triggers a fault according to a preset condition, and the probability-based trigger mechanism triggers a fault according to a preset probability distribution.

3. The virtual operation fault training method for live working robots according to claim 2 is characterized in that: The rule-based trigger mechanism includes time trigger, job step trigger and job progress trigger; wherein, the time trigger is to trigger a fault when the target job process is executed to a specified time point, the job step trigger is to trigger a fault when the target job process is executed to a specified job step, and the job progress trigger is to trigger a fault when the target job process is executed to a specified job progress; The probability-based trigger mechanism includes random probability triggering and conditional probability triggering; wherein the random probability triggering is to determine the first fault triggering probability according to the preset probability distribution, and trigger the fault according to the first fault triggering probability at each time point or each operation step in the target operation process; the conditional probability triggering is to calculate the second fault triggering probability based on multiple preset fault triggering conditions, and trigger the fault according to the second fault triggering probability; The trigger mechanism based on rules and probability includes rule priority triggering and probability adjustment triggering; wherein, the rule priority triggering means that the priority of the rule-based triggering mechanism is higher than the priority of the probability-based triggering mechanism, and the probability adjustment triggering means adjusting the first fault triggering probability or the second fault triggering probability according to the rules, and triggering the fault according to the adjusted first fault triggering probability or the second fault triggering probability.

4. The virtual operation fault training method for live working robots according to claim 2 is characterized in that: By means of the fault management module, during the execution of the target operation process, based on the target fault simulation phase and its corresponding target fault injection mechanism, at least one fault scenario of the live working robot is simulated in the virtual working environment, including: In response to a selection operation on the candidate fault simulation stage, a target fault simulation stage is determined; Through the fault management module, during the execution of the target operation process, the target fault injection mechanism corresponding to the target fault simulation stage is randomly determined from the candidate fault injection mechanisms, so as to simulate at least one fault scenario of the live working robot in the virtual working environment based on the target fault injection mechanism.

5. The virtual operation fault training method for live working robots according to claim 1 is characterized in that: The virtual operation fault training system is communicatively connected with the digital twin system, and the digital twin system is configured with a robot model corresponding to the live operation robot; the method further comprises: The execution log corresponding to the target job process and the fault attributes corresponding to each fault scenario are sent to the digital twin system, so that the digital twin system controls the robot model to perform actions based on the execution log and the fault attributes, and displays the action process of the robot model.

6. The virtual operation fault training method for live working robots according to claim 1, characterized in that: The method further comprises: Based on the preset intervention time limit and operation scoring criteria, a scoring result corresponding to the manual intervention operation is determined.

7. The virtual operation fault training method for live working robots according to claim 1 is characterized in that: The method further comprises: If the operation data changes, the new operation data of the live working robot in the real working environment is collected through the data acquisition module, the new operation data is denoised and filtered, the processed new operation data is temporarily cached, and all the new operation data after temporary caching are synchronized to the virtual working environment.

8. A virtual operation fault training system for live working robots, characterized in that: The virtual operation fault training system includes a data acquisition module, an operation management module, a fault management module and a user management module: The data acquisition module is used to collect the operation data of the live working robot in a real working environment; The operation management module is used to build a virtual operation environment consistent with the real operation environment based on the operation data, and execute the target operation process in the virtual operation environment; The fault management module is used to simulate at least one fault scenario of the live working robot in the virtual working environment based on the target fault simulation phase and its corresponding target fault injection mechanism during the execution of the target working process; The user management module is used to respond to the manual intervention operation performed by the operator for the fault event triggered in each of the fault scenarios to complete the fault training corresponding to the target operation process.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images