Robot abnormality processing method and device, computer device, and storage medium

Abstract

The application relates to the technical field of artificial intelligence, can be applied to business system platforms such as medical health and financial technology, and discloses a robot abnormality processing method and device, computer equipment and a storage medium, state data and environment data when a target robot executes a target task according to a task execution strategy are collected in real time; the state data and the environment data are analyzed by using a pre-trained visual language large model, target abnormal factors are identified; the influence degree of the target abnormal factors on the target robot executing the target task is evaluated, and an evaluation result is generated; based on the evaluation result, the visual language large model is used to adjust the task execution strategy, and a target strategy is generated; the target robot is controlled to re-execute the target task according to the target strategy; thereby the adaptability of the robot in a complex environment and the success rate of task execution can be improved, and the negative influence of abnormal conditions is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a robot abnormality processing method and device, computer equipment and a computer readable storage medium. BACKGROUND

[0002] At present, with the rapid development of robot technology, robots are increasingly widely used in industries, logistics, services, medical health and financial technology, etc. Robots usually need to perform various tasks in complex and variable environments, such as material handling in factories, goods sorting in warehouses, and providing guidance services in service places, etc. In these application scenarios, robots may encounter various abnormal situations, such as sensor failure, environmental changes, mechanical failure, etc., which may adversely affect the execution of tasks, and even cause task failure or robot damage.

[0003] Currently, traditional robot abnormality processing methods mainly rely on pre-set rules and simple sensor feedback. For example, when the sensor detects an abnormal signal, the robot will trigger a pre-set alarm or perform a simple obstacle avoidance action. However, these methods have the following limitations:

[0004] 1. Lack of real-time: Traditional methods can only handle abnormalities after they occur, and cannot predict and prevent abnormalities in real time.

[0005] 2. Limited processing capacity: Traditional methods rely on fixed rules and are difficult to cope with complex and variable abnormal situations, especially those not previously defined.

[0006] 3. Lack of intelligence: Traditional methods cannot analyze abnormal situations in depth, cannot assess the impact of abnormalities on task execution, and cannot dynamically adjust task execution strategies.

[0007] In the medical health field, robots are widely used in rehabilitation therapy, medicine delivery, etc. For example, rehabilitation robots need to dynamically adjust training programs according to the rehabilitation progress of patients, and traditional robot abnormality processing methods cannot meet the needs of high precision and high dynamics. In addition, medical robots also need to cope with sudden emergencies in hospital environments, such as patient emergencies or equipment failure, and the coping ability of traditional methods in such complex scenarios is obviously insufficient.

[0008] In the field of financial technology, robots are applied to intelligent customer service, transaction monitoring and other scenarios. For example, intelligent customer service robots need to process complex customer problems in real time, and traditional robot exception handling methods cannot accurately identify and handle non-standard customer problems. Transaction monitoring robots need to analyze a large amount of transaction data in real time to detect abnormal transaction behavior, and traditional rule-based methods are difficult to cope with complex transaction exception situations. In addition, financial technology robots also need to quickly respond and handle exceptions under the premise of ensuring data security, and the shortcomings of traditional methods in security and real-time performance limit their application range.

[0009] Therefore, how to provide a robot exception handling method, device, computer equipment and computer readable storage medium can improve the adaptability of robots in complex environments and the success rate of task execution, while reducing the negative impact of abnormal situations on robots and tasks, is a problem that needs to be solved by the technical personnel in the field at present. SUMMARY

[0010] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a robot exception handling method, device, computer equipment and computer readable storage medium, which aims to solve the problem of how to improve the adaptability of robots in complex environments and the success rate of task execution, while reducing the negative impact of abnormal situations on robots and tasks.

[0011] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0012] In a first aspect, the present application provides a robot exception handling method, comprising:

[0013] Real-time collection of state data and environment data of a target robot executing a target task according to a task execution strategy;

[0014] Using a pre-trained visual language large model to analyze the state data and the environment data, and identifying target abnormal factors;

[0015] Evaluating the influence degree of the target abnormal factors on the target robot executing the target task, and generating an evaluation result;

[0016] Based on the evaluation result, adjusting the task execution strategy using the visual language large model to generate a target strategy;

[0017] According to the target strategy, controlling the target robot to re-execute the target task.

[0018] In a second aspect, the present application provides a robot exception handling device, comprising:

[0019] The collection module is configured to collect state data and environment data in real time when the target robot executes a target task according to a task execution strategy.

[0020] The analysis module is configured to analyze the state data and the environment data by using a pre-trained visual language large model, and identify a target abnormal factor.

[0021] The evaluation module is configured to evaluate an influence degree of the target abnormal factor on execution of the target task by the target robot, and generate an evaluation result.

[0022] The adjustment module is configured to adjust the task execution strategy by using the visual language large model based on the evaluation result, and generate a target strategy.

[0023] The control module is configured to control the target robot to re-execute the target task according to the target strategy.

[0024] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the robot abnormal processing method as described above when executing the computer program.

[0025] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the robot abnormal processing method as described above.

[0026] Compared with the prior art, the present application provides a robot abnormal processing method, device, computer device and computer readable storage medium, wherein the state data and the environment data when the target robot executes the target task according to the task execution strategy are collected in real time; the state data and the environment data are analyzed by using a pre-trained visual language large model, and a target abnormal factor is identified; the influence degree of the target abnormal factor on execution of the target task by the target robot is evaluated, and an evaluation result is generated; the task execution strategy is adjusted by using the visual language large model based on the evaluation result, and a target strategy is generated; the target robot is controlled to re-execute the target task according to the target strategy; thereby the adaptability of the robot in a complex environment and the success rate of task execution can be improved, and the negative influence of abnormal situations on the robot and the task can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

[0028] Figure 1 An application environment schematic diagram of a robot abnormality processing method provided by an embodiment of the present application.

[0029] Figure 2 A flow schematic diagram of a robot abnormality processing method provided by an embodiment of the present application.

[0030] Figure 3 A program module schematic diagram of a robot abnormality processing device provided by an embodiment of the present application.

[0031] Figure 4 A structure schematic diagram of a computer device provided by an embodiment of the present application.

[0032] Figure 5 Another structure schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of the present application.

[0034] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0035] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0036] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0037] In addition, the description in the description of the application and the appended claims, the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0038] In the description of the application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in additional some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0039] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0040] In order to illustrate the technical solutions of the application, the following specific embodiments are described.

[0041] An embodiment of the application provides a robot exception processing method, which can be applied to, for example Figure 1In the application environment shown, the client and the server communicate through a network. The client includes, but is not limited to, a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computer device, a personal digital assistant (PDA), and the like computer device. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, and the like basic cloud computing services.

[0042] Referring to Figure 2 An embodiment of the present application provides a robot abnormality processing method, wherein the method comprises the following steps:

[0043] S100, real-time collection of state data and environment data of a target robot performing a target task according to a task execution strategy;

[0044] S200, analysis of the state data and the environment data by using a pre-trained visual language large model, identification of a target abnormality factor;

[0045] S300, evaluation of an influence degree of the target abnormality factor on the target robot performing the target task, generation of an evaluation result;

[0046] S400, adjustment of the task execution strategy by using the visual language large model based on the evaluation result, generation of a target strategy;

[0047] S500, control of the target robot to re-perform the target task according to the target strategy.

[0048] In implementation, the robot abnormality processing method of the embodiment realizes efficient adaptation of the robot in a complex environment and improvement of the success rate of task execution, while significantly reducing the negative impact of abnormal situations on the robot and the task. Specifically, the method first collects state data and environment data of the target robot during task execution according to the task execution strategy in real time, which ensures comprehensive monitoring of the robot operation and the surrounding environment. Subsequently, the pre-trained visual language large model is used to analyze the collected data in depth, which can accurately identify the target abnormal factors. This analysis method based on advanced models far exceeds traditional methods and can handle complex and variable abnormal situations. Further, by evaluating the influence degree of the abnormal factors on the task execution and generating an evaluation result, the method can quantify the potential impact of the abnormality and provide a scientific basis for subsequent strategy adjustment. Based on the evaluation result, the visual language large model plays a role again, dynamically adjusts the task execution strategy, and generates an optimized target strategy. This process embodies high intelligence and flexibility, enabling the robot to flexibly adjust its behavior according to real-time conditions. Finally, according to the optimized target strategy, the robot re-executes the task to ensure that the task can be efficiently and safely completed. Through this systematic abnormality processing process, the robot can quickly respond and adjust strategies when facing complex environments and unexpected abnormalities, thereby improving the success rate of tasks, reducing the loss caused by abnormalities, and significantly improving the reliability and practicality of robots in actual applications.

[0049] Understandably, the robot abnormality processing method provided by the embodiment of the present application can be applied to the robot abnormality processing scene related to the medical and health field. The following is a specific example:

[0050] Scene description: In the medical and health field, rehabilitation therapy robots are widely used to assist patients in rehabilitation training, such as limb movement rehabilitation, balance training, etc. These robots need to dynamically adjust the training program according to the patient's rehabilitation progress and physical condition to ensure the rehabilitation effect. Various abnormal situations may occur during rehabilitation, such as abnormal patient body reactions, equipment failures, etc., which may affect the rehabilitation effect and even cause harm to the patient.

[0051] Apply the robot abnormality processing method of the present application:

[0052] 1. Real-time data collection: The rehabilitation therapy robot collects its state data (such as robot motion parameters, force feedback, etc.) and environment data (such as patient vital signs, rehabilitation progress, etc.) in real time when performing rehabilitation training tasks. For example, the robot can collect data such as the patient's heart rate, blood pressure, limb strength, as well as information such as the angle of its own joints, movement speed, etc.

[0053] 2. Abnormal factor identification: Use the pre-trained visual language large model to analyze the collected data and identify the target abnormal factor. For example, the model may detect that the patient's heart rate is abnormally high, indicating that there may be excessive fatigue or physical discomfort; or detect that the robot's force feedback data is abnormal, indicating that there may be equipment failure.

[0054] 3. Impact assessment: Evaluate the identified abnormal factors and generate an evaluation result. For example, the evaluation result shows that the patient's heart rate is abnormally high, which may affect the effectiveness of rehabilitation training, even pose a risk to the patient's health, and the impact is high.

[0055] 4. Task execution strategy adjustment: Based on the evaluation result, use the visual language large model to adjust the task execution strategy. For example, adjust the intensity of rehabilitation training, reduce the difficulty of movement, or pause training and remind medical personnel to check. At the same time, perform self-checking on the equipment to ensure that the robot is running normally.

[0056] 5. Re-execute the task: According to the adjusted target strategy, control the rehabilitation robot to re-execute the rehabilitation training task. The robot continues to provide rehabilitation training for the patient according to the new strategy, ensuring the safety and effectiveness of the training.

[0057] Technical effects: Through the abnormal processing method of the present application, the rehabilitation robot can detect and handle abnormal situations in real time during rehabilitation training, dynamically adjust the training plan, and ensure the safety and effectiveness of rehabilitation training. This method can significantly improve the success rate of rehabilitation, reduce the risk of patient health due to abnormal situations, and significantly improve the reliability and practicality of the rehabilitation robot.

[0058] It can be understood that the robot abnormal processing method provided by the embodiments of the present application can also be applied to the robot abnormal processing scene in the field of financial technology. The following is a specific example:

[0059] Scene description: In the field of financial technology, intelligent customer service robots are widely used to handle customer inquiries and transaction requests. These robots need to handle a large number of customer problems in real time and provide accurate answers and transaction support. Any abnormal situation, such as customer questions beyond the preset range or system failure, may result in decreased customer satisfaction or failed transactions.

[0060] Apply the robot abnormal processing method of the present application:

[0061] 1. Real-time data collection: When handling customer inquiries, the intelligent customer service robot collects its state data (such as system response time, customer waiting time, etc.) and environmental data (such as the type of customer questions, real-time changes in transaction data, etc.) in real time.

[0062] 2. Abnormal factor identification: Use a pre-trained visual language large model to analyze the collected data and identify the target abnormal factor. For example, the model may detect that the question raised by the customer exceeds the preset knowledge base range, prompting that there may be an abnormal consultation.

[0063] 3. Impact assessment: Evaluate the identified abnormal factors and generate an evaluation result. For example, the evaluation result shows that the abnormal factor may cause customer satisfaction to decrease, with a high impact degree.

[0064] 4. Task execution strategy adjustment: Based on the evaluation result, use the visual language large model to adjust the task execution strategy. For example, adjust the retrieval strategy of the knowledge base, introduce external information sources (such as real-time financial news) to provide more accurate answers, and optimize system response time to ensure that customer inquiries can be processed quickly.

[0065] 5. Re-execution of tasks: According to the adjusted target strategy, control the intelligent customer service robot to re-process customer inquiries. The robot continues to provide services according to the new strategy to ensure that customer problems are answered in a timely and accurate manner.

[0066] Technical effects: Through the abnormal processing method of the present application, the intelligent customer service robot can detect and handle abnormal situations in real time when processing customer inquiries, improve customer satisfaction and transaction success rate, reduce the risk of customer loss and transaction failure caused by abnormal situations, and significantly improve the reliability and practicality of the financial technology robot.

[0067] Further, in one embodiment, the robot abnormal processing method, wherein the real-time collection of state data and environmental data of the target robot during execution of the target task according to the task execution strategy comprises the steps of:

[0068] Obtain the user's language instruction and parse the language instruction through natural language processing technology, and construct a task execution strategy according to the parsing result;

[0069] Control the target robot to execute the target task according to the task execution strategy;

[0070] Collect state data and environmental data of the target robot during execution of the target task through various sensors carried by the target robot in real time;

[0071] Wherein, the state data includes position information, speed information, attitude information and battery information of the target robot; the environmental data includes temperature information, humidity information, obstacle information and task object information.

[0072] In specific implementation, the specific implementation process of the steps of the present embodiment is as follows:

[0073] 1. Obtain and parse user language commands.

[0074] Step 1.1: Instruction Acquisition and Preprocessing

[0075] Implementation process: The user's speech commands are acquired through a speech recognition device and converted into text data. The text data is preprocessed, including noise removal, punctuation processing, and word segmentation, in preparation for subsequent natural language processing.

[0076] Example: A user says, "Have the robot go to warehouse A to pick up the goods and deliver them to warehouse B." The voice recognition device converts this into the text "Have the robot go to warehouse A to pick up the goods and deliver them to warehouse B."

[0077] Step 1.2: Natural Language Parsing

[0078] Implementation process: Natural Language Processing (NLP) technology is used to perform semantic parsing on the preprocessed text instructions. Key information is extracted, such as the task objective (pickup, delivery), location (warehouse A, warehouse B), and task object (goods).

[0079] Example: The parsing result is that the task objective is "pick up and deliver goods", the starting point is "warehouse A", the destination is "warehouse B", and the task object is "goods".

[0080] Step 1.3: Construct the task execution strategy

[0081] Implementation process: Based on the analysis results, and combined with the robot's capabilities and environmental information, a task execution strategy is constructed. The strategy includes path planning, task prioritization, and resource allocation.

[0082] Example: The constructed task execution strategy includes path planning from the current location to warehouse A, pickup action planning, and delivery path planning from warehouse A to warehouse B.

[0083] 2. Control the target robot to perform tasks

[0084] Step 2.1: Task Assignment and Initialization

[0085] Implementation process: Based on the constructed task execution strategy, tasks are assigned to the target robot. Relevant robot parameters, such as position, velocity, and attitude, are initialized to ensure the robot is in a state capable of performing the task.

[0086] Example: After receiving the task, the robot initializes its position to its current position, sets its speed to the default value, and adjusts its posture to face warehouse A.

[0087] Step 2.2: Task Execution Control

[0088] Implementation process: the robot executes the task according to the task execution strategy. Through the control system of the robot, motion instructions and operation instructions are sent, so that the robot can complete the tasks of picking up and delivering goods.

[0089] Example: the robot moves to warehouse A along the planned path according to the path planning, performs the picking action, and then moves to warehouse B along the planned path, performs the delivery action.

[0090] 3. Real-time data acquisition

[0091] Step 3.1: sensor data acquisition

[0092] Implementation process: through various sensors carried by the robot, real-time acquisition of state data and environmental data. State data includes position information, speed information, attitude information and battery information; environmental data includes temperature information, humidity information, obstacle information and task object information.

[0093] Example: the GPS sensor carried by the robot acquires position information, the IMU sensor acquires attitude information, the speed sensor acquires speed information, and the battery management system acquires battery information. At the same time, the environmental sensor acquires temperature and humidity information, the laser radar or camera acquires obstacle information, and the RFID or visual sensor acquires task object information.

[0094] Through the above process, this embodiment realizes the acquisition and analysis of user language instructions, the construction and execution of task execution strategy, and the acquisition of real-time data, which provides a solid foundation for subsequent abnormality detection and processing.

[0095] Further, in one embodiment, the robot abnormality processing method, wherein the pre-trained visual language large model is used to analyze the state data and the environmental data, identify target abnormal factors, specifically including steps of:

[0096] Preprocessing the state data and the environmental data;

[0097] Feature extraction is performed on the preprocessed state data and environmental data, and the extracted features are fused to obtain fused features;

[0098] The fusion features are input into the pre-trained visual language large model for abnormality identification, a plurality of candidate abnormal factors are generated, and a context perception algorithm is used to screen out target abnormal factors that meet the preset rules from the plurality of candidate abnormal factors.

[0099] Further, the robot abnormality processing method, wherein the feature extraction is performed on the preprocessed state data and environmental data, and the extracted features are fused to obtain fused features, specifically including steps of:

[0100] encoding the pre-processed state data and environment data using the visual language large model to obtain corresponding state feature vectors and environment feature vectors;

[0101] aligning the state feature vectors and environment feature vectors, mapping the state feature vectors and environment feature vectors to the same feature space through linear transformation or nonlinear mapping, and fusing the state feature vectors and environment feature vectors using an attention mechanism to generate fused features.

[0102] In specific implementation, the specific implementation process of the steps of the embodiment is approximately as follows:

[0103] 1. Data preprocessing

[0104] Step 1.1: Data cleaning

[0105] Implementation process: Clean the collected state data and environment data, remove invalid data, missing values and outliers. For example, remove error data caused by sensor failure, fill in missing values, and ensure data integrity and accuracy.

[0106] Example: If a sensor returns a value outside the normal range at a certain time (such as a temperature sensor returning -50℃, while the normal range is 0℃ to 50℃), it is marked as invalid data and processed.

[0107] Step 1.2: Data normalization

[0108] Implementation process: Normalize the data to a unified range (such as [0, 1] or [-1, 1]) so that data from different sensors can be effectively compared and fused. Normalization methods can include min-max normalization, Z-score normalization, etc.

[0109] Example: For the speed data of the robot (the range may be 0 to 10 meters / second), use min-max normalization to convert it to the [0, 1] range.

[0110] Step 1.3: Data denoising

[0111] Implementation process: Denoise the data to remove noise interference and improve data quality. You can use filtering algorithms (such as low-pass filters, Gaussian filters) or deep learning-based denoising methods.

[0112] Example: For the obstacle distance data collected by the laser radar, use a Gaussian filter to remove random noise and make the data smoother.

[0113] 2. Feature extraction and fusion

[0114] Step 2.1: Data Encoding Process

[0115] Implementation Process: Utilize the encoder module of the pre-trained visual language large model to encode the preprocessed state data and environment data, generating corresponding state feature vectors and environment feature vectors.

[0116] Example: Input the robot's position information (x, y, z) and velocity information (vx, vy, vz), the encoder module converts them into state feature vectors S; input the temperature T and obstacle position O in the environment data, the encoder module converts them into environment feature vectors E.

[0117] Step 2.2: Feature Alignment Process

[0118] Implementation Process: Align the state feature vectors S and environment feature vectors E, mapping them to the same feature space through linear transformation or nonlinear mapping.

[0119] Example: Suppose the dimension of state feature vectors S is ds, and the dimension of environment feature vectors E is de. Through a linear transformation W, S is mapped to the same dimension de as E:

[0120] S' = W·S

[0121] Where W is a de×ds matrix.

[0122] Step 2.3: Feature Fusion

[0123] Implementation Process: Use attention mechanisms to fuse the aligned state feature vectors S' and environment feature vectors E, generating fusion features F.

[0124] 3. Abnormality Identification and Screening

[0125] Step 3.1: Abnormality Identification

[0126] Implementation Process: Input the fusion features F into the pre-trained visual language large model for abnormality identification, generating multiple candidate abnormal factors. The model can automatically learn abnormal patterns based on deep learning algorithms (such as CNN, Transformer).

[0127] Example: The model identifies the following candidate abnormal factors: sensor failure, abnormally high environmental temperature, obstacles on the path.

[0128] Step 3.2: Context-aware Screening

[0129] Implementation process: Use the context-aware algorithm, combined with the context information of the task (such as task target, historical data, real-time environment, etc.), to filter out the target abnormal factor that meets the preset rules from multiple candidate abnormal factors. The preset rules can include the type, impact degree, and frequency of abnormal factors.

[0130] Example: The preset rule is "abnormal factors affecting task safety are prioritized". According to the context information, "obstacles on the path" are selected as the target abnormal factor because they directly affect the robot's path planning and task execution safety.

[0131] Step 3.3: Target abnormal factor confirmation

[0132] Implementation process: Final confirmation of the selected target abnormal factor to ensure its accuracy and reliability. The authenticity of the abnormal factor can be confirmed through further verification (such as comparison with historical data, real-time sensor data verification).

[0133] Example: Through real-time data verification of laser radar and camera, it is confirmed that the obstacles on the path do exist, and finally "obstacles on the path" are confirmed as the target abnormal factor.

[0134] This embodiment realizes the complete process from data preprocessing to feature extraction and fusion, and then to abnormal identification and screening, providing accurate target abnormal factors for subsequent abnormal handling.

[0135] Further, in one embodiment, the robot abnormal handling method, wherein the evaluation of the influence degree of the target abnormal factor on the target robot executing the target task generates an evaluation result, specifically including steps:

[0136] Based on historical data and predefined abnormal types, an abnormal factor influence model is constructed;

[0137] Using the abnormal factor influence model, the degree of the target abnormal factor affecting the target robot executing the target task is evaluated, and an evaluation result is generated.

[0138] In specific implementation, the specific implementation process of the steps of this embodiment is roughly as follows:

[0139] 1. Construct an abnormal factor influence model

[0140] Step 1.1: Data collection and labeling

[0141] Implementation process: Collect historical and real-time data, including the state data of the robot when performing tasks, environmental data, and records of abnormal factors. Label these data, and clearly describe the content of each abnormal factor, the degree of influence, the range of influence, the expected risk, and the suggested response measures.

[0142] Example: Collect data when the robot performs a cargo handling task in the warehouse, label abnormal factors (such as sensor failure, obstacles on the path, etc.) and their influence (such as task delay, path change).

[0143] Step 1.2: Model design and training

[0144] Implementation process: Design an abnormal factor influence model, which can be a machine learning or deep learning-based model, such as decision tree, random forest, neural network, etc. Train the model using labeled data so that it can predict the influence of abnormal factors on task execution according to the input.

[0145] Example: Design a neural network-based model, input is the feature vector of abnormal factors, output is the degree of influence, range of influence, expected risk, etc. Use historical data to train the model so that it can accurately predict the influence of abnormal factors.

[0146] 2. Use the abnormal factor influence model for evaluation

[0147] Step 2.1: Abnormal factor feature extraction

[0148] Implementation process: Extract features of the target abnormal factor to generate a feature vector for model input. Feature extraction can include the type of abnormal factor, frequency of occurrence, duration, etc.

[0149] Example: For the abnormal factor "obstacle on the path", extract its feature vector, including the size, location, frequency of the obstacle, etc.

[0150] Step 2.2: Impact assessment

[0151] Implementation process: Input the extracted feature vector into the abnormal factor influence model, and the model will evaluate the degree of influence of the target abnormal factor on the target robot performing the target task according to the trained weights and parameters, and generate the evaluation result.

[0152] Example: Input the feature vector of the obstacle into the model, and the model outputs the degree of influence of this abnormal factor as "moderate", the range of influence as "local path", the expected risk as "low", and the suggested response measures as "replan path".

[0153] 3. Generate evaluation results

[0154] Step 3.1: Evaluation Result Compilation

[0155] Implementation Process: Compile the evaluation results output by the model to generate a detailed evaluation report. The evaluation report should include the content description of abnormal factors, the degree of influence, the range of influence, the expected risk, and the suggested countermeasures.

[0156] Example: The generated evaluation report content is as follows:

[0157] Abnormal Factor Description: Obstacles appear on the path, with a size of 0.5 meters x 0.5 meters, located 10 meters from the robot's forward path.

[0158] Influence Degree: Moderate.

[0159] Influence Range: Local path.

[0160] Expected Risk: Low.

[0161] Suggested Countermeasures: Re-plan the path to avoid obstacles.

[0162] Step 3.2: Evaluation Result Verification

[0163] Implementation Process: Verify the generated evaluation results to ensure their accuracy and reliability. Verification can be done by comparing with historical data, real-time sensor data verification, etc.

[0164] Example: Through real-time data verification of laser radar and camera, confirm the existence and location of obstacles, and verify the accuracy of the evaluation results.

[0165] This embodiment realizes the complete process from the construction of the abnormal factor influence model to the generation of the evaluation results, providing accurate evaluation basis for subsequent task execution strategy adjustment.

[0166] Further, in one embodiment, the robot abnormal handling method, wherein, based on the evaluation result, the visual language large model is used to adjust the task execution strategy to generate a target strategy, specifically including steps:

[0167] According to the evaluation result, build a prompt text for adjusting the task execution strategy;

[0168] Input the task execution strategy and the prompt text into the visual language large model to generate a preliminary execution strategy;

[0169] In the virtual environment, verify the preliminary execution strategy for the target robot, and when the verification result meets the preset requirements, the preliminary execution strategy is taken as the target strategy.

[0170] In implementation, the specific implementation process of the steps in this embodiment is roughly as follows:

[0171] 1. Constructing prompt text

[0172] Step 1.1: Analyzing evaluation results

[0173] Implementation process: Detailed analysis of evaluation results, extraction of key information, including content description of abnormal factors, impact degree, impact range, expected risk, and suggested countermeasures.

[0174] Example: Analyzing evaluation results, extracting abnormal factors as "obstacles on the path", impact degree as "moderate", impact range as "local path", expected risk as "low", and suggested countermeasures as "replanning the path".

[0175] Step 1.2: Generating prompt text

[0176] Implementation process: According to the analyzed evaluation results, construct the prompt text for adjusting the task execution strategy. The prompt text should contain detailed information of abnormal factors and suggested adjustment direction.

[0177] Example: The generated prompt text is: "Obstacles are detected on the path, with moderate impact, and it is suggested to replan the path to avoid obstacles."

[0178] 2. Generating initial execution strategy

[0179] Step 2.1: Inputting visual language large model

[0180] Implementation process: Input the current task execution strategy and the constructed prompt text into the pre-trained visual language large model. The model will combine the information in the task execution strategy and the prompt text to generate the initial execution strategy.

[0181] Example: Input the current task execution strategy (such as path planning, speed setting, etc.) and the prompt text, and the model outputs the initial execution strategy, including new path planning and speed adjustment.

[0182] Step 2.2: Generating initial execution strategy

[0183] Implementation process: The visual language large model generates the initial execution strategy according to the input task execution strategy and prompt text. The initial execution strategy should include specific adjustment measures, such as path adjustment, speed adjustment, task priority adjustment, etc.

[0184] Example: The generated initial execution strategy includes: new path planning to avoid obstacles, reducing speed to ensure safety, and adjusting task priority to prioritize critical tasks.

[0185] 3. Verifying the initial execution strategy

[0186] Step 3.1: Virtual Environment Setup

[0187] Implementation Process: Set up a virtual environment similar to the actual task environment to verify the effectiveness and feasibility of the preliminary execution strategy. The virtual environment can include robot models, task scenarios, obstacles, etc.

[0188] Example: Set up a virtual warehouse environment with robot models, cargo locations, paths, and obstacles.

[0189] Step 3.2: Preliminary Execution Strategy Verification

[0190] Implementation Process: In the virtual environment, simulate the target robot executing the task according to the preliminary execution strategy to verify its effectiveness and safety. Verification content includes task completion rate, path planning rationality, obstacle avoidance ability, etc.

[0191] Example: Run the robot in the virtual environment to verify whether the new path can avoid obstacles and whether the task can be completed smoothly.

[0192] Step 3.3: Verification Result Evaluation

[0193] Implementation Process: Evaluate the verification results according to the pre-set verification standards (requirements). Pre-set requirements can include task completion time, path length, obstacle avoidance success rate, etc.

[0194] Example: Pre-set requirements are that task completion time should not exceed the original plan by more than 10%, path length should not increase by more than 20%, and obstacle avoidance success rate should be above 95%.

[0195] 4. Confirm Target Strategy

[0196] Step 4.1: Target Strategy Confirmation

[0197] Implementation Process: If the verification results meet the pre-set requirements, confirm the preliminary execution strategy as the target strategy. The target strategy will be used to control the robot to re-execute the task.

[0198] Example: Verification results show that task completion time increased by 5%, path length increased by 15%, and obstacle avoidance success rate is 98%, meeting the pre-set requirements, so the preliminary execution strategy is confirmed as the target strategy.

[0199] Step 4.2: Target Strategy Output

[0200] Implementation Process: Output the confirmed target strategy for use by the robot control system. The target strategy should include specific execution parameters and adjustment measures.

[0201] Example: Output target strategy, including new path planning, speed settings, and task priority adjustments, for the robot control system to execute.

[0202] This embodiment realizes the complete process from the construction of the evaluation results to the construction of the prompt text, to the generation and verification of the preliminary execution strategy, and finally confirms the target strategy, ensuring that the adjustment of the task execution strategy is both scientific and reliable.

[0203] Further, in one embodiment, the robot abnormality processing method, wherein the target robot is controlled to re-execute the target task based on the target strategy, specifically comprising the steps of:

[0204] According to the task decomposition requirements in the target strategy, the target task is decomposed into a plurality of sub-tasks with a clear execution order;

[0205] According to the task execution instructions of each sub-task in the target strategy, the target robot is controlled to process each sub-task in sequence according to the execution order.

[0206] In specific implementation, the specific implementation process of the steps of this embodiment is roughly as follows:

[0207] 1. Target strategy analysis

[0208] Step 1.1: Strategy analysis

[0209] Implementation process: Detailed analysis of the target strategy, extracting the task decomposition requirements and the task execution instructions of each sub-task. The task decomposition requirements define how to split the target task into multiple sub-tasks, while the task execution instructions define the specific operation steps, parameters and priority of each sub-task.

[0210] Example: The target strategy analysis result shows that the target task is decomposed into three sub-tasks: sub-task 1 (navigate to warehouse A), sub-task 2 (pick up goods), and sub-task 3 (navigate to warehouse B and deliver goods). Each sub-task has clear execution instructions, such as sub-task 1's instructions including path planning and speed settings.

[0211] 2. Task decomposition

[0212] Step 2.1: Task decomposition

[0213] Implementation process: According to the task decomposition requirements in the target strategy, the target task is decomposed into a plurality of sub-tasks with a clear execution order. Each sub-task should have clear start and end conditions, as well as dependency relationships with other sub-tasks.

[0214] Example: The target task "pick up goods from warehouse A and deliver to warehouse B" is decomposed into:

[0215] 1. Subtask 1: Navigate to Warehouse A.

[0216] 2. Subtask 2: Pick up goods.

[0217] 3. Subtask 3: Navigate to Warehouse B and deliver goods.

[0218] The execution order of each subtask is clear, subtask 1 must be completed before subtask 2, and subtask 2 must be completed before subtask 3.

[0219] 3. Subtask Execution Control

[0220] Step 3.1: Subtask Initialization

[0221] Implementation process: According to the task execution instructions of each subtask in the target strategy, initialize the execution environment and parameters of the subtask. This includes setting the initial position, speed, posture and other parameters of the robot to ensure that the subtask can start smoothly.

[0222] Example: For subtask 1 (navigate to Warehouse A), initialize the position of the robot as the current position, set the speed to the default value, and adjust the posture to face the direction of Warehouse A.

[0223] Step 3.2: Subtask Execution

[0224] Implementation process: Control the target robot to process each subtask in turn according to the execution order. According to the task execution instructions of each subtask, send the corresponding control instructions to the robot to enable it to complete the operation of the subtask.

[0225] Example: After the robot receives the execution instruction of subtask 1, it starts to navigate to Warehouse A. After reaching Warehouse A, the robot executes subtask 2 (pick up goods), and after completing the pick-up action, the robot starts to execute subtask 3 (navigate to Warehouse B and deliver goods).

[0226] Step 3.3: Subtask State Monitoring

[0227] Implementation process: During the execution of the subtask, real-time monitoring of the state data and environmental data of the robot is carried out to ensure the smooth progress of the subtask. If abnormal conditions are detected, the execution strategy is adjusted or the subtask execution is suspended in time.

[0228] Example: During the execution of subtask 1, the position and speed of the robot are monitored in real time to ensure that it advances according to the planned path. If obstacles are detected on the path, adjust the path planning to avoid obstacles.

[0229] 4. Task Completion Confirmation

[0230] Step 4.1: Subtask Completion Confirmation

[0231] Implementation process: After all sub-tasks are processed in sequence, confirm the completion of each sub-task. Check if the sub-tasks are completed as expected, including key indicators such as task completion quality, efficiency, resource consumption, etc.

[0232] Example: Confirm whether sub-task 1 (navigate to warehouse A) successfully reaches the destination, sub-task 2 (pick up goods) successfully picks up goods, and sub-task 3 (navigate to warehouse B and deliver goods) successfully delivers goods.

[0233] Step 4.2: Task completion feedback

[0234] Implementation process: Feedback the task completion to the system or user, including confirmation information of successful completion, key data of task execution, and possible improvement suggestions. If the task is not completely completed as expected, analyze the reasons for not meeting the target and adjust the target strategy as needed.

[0235] Example: After the task is completed, the system feeds back the task completion information, including total time, resource consumption, exception handling, etc. If the task is not completely completed, analyze the reasons (such as unreasonable path planning or robot failure) and suggest adjusting the strategy or performing equipment maintenance.

[0236] Through the above process, this embodiment realizes the complete process from target strategy analysis to task decomposition, then to the sequential execution of sub-tasks and the confirmation of task completion, ensuring that the robot can efficiently and safely re-execute the target task.

[0237] From the above method embodiment, it can be seen that the robot abnormal handling method provided by the present application comprises: collecting state data and environment data of a target robot executing a target task according to a task execution strategy in real time; using a pre-trained visual language large model to analyze the state data and the environment data, and identifying target abnormal factors; evaluating the influence degree of the target abnormal factors on the target robot executing the target task, and generating an evaluation result; based on the evaluation result, adjusting the task execution strategy using the visual language large model to generate a target strategy; and controlling the target robot to re-execute the target task according to the target strategy. In this way, the method of the present application can improve the adaptability of the robot in a complex environment and the success rate of task execution, while reducing the negative impact of abnormal situations on the robot and the task.

[0238] It should be understood that although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps can be included based on conventional or non-inventive labor, and the operation steps are not necessarily executed in the order of the embodiments or flowcharts. The order of steps listed in the embodiments or flowcharts is only one of the many execution orders, and does not represent the only execution order. It should be noted that there is no certain sequence between the above steps, and those skilled in the art can understand from the description of the embodiments of the present application that the above steps can have different execution orders in different embodiments, that is, they can be executed in parallel, or they can be exchanged and executed, etc. Moreover, at least part of the steps in the embodiments or flowcharts can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation, alternation or synchronization with other steps or sub-steps or stages of other steps.

[0239] Based on the above method embodiments, please refer to Figure 3 Another embodiment of the present application also provides a robot abnormality processing device, wherein the device comprises:

[0240] The acquisition module 11 is configured to acquire state data and environment data of a target robot in real time when the target robot executes a target task according to a task execution strategy;

[0241] The analysis module 12 is configured to analyze the state data and the environment data by using a pre-trained visual language large model, and identify a target abnormal factor;

[0242] The evaluation module 13 is configured to evaluate an influence degree of the target abnormal factor on the target robot executing the target task, and generate an evaluation result;

[0243] The adjustment module 14 is configured to adjust the task execution strategy by using the visual language large model based on the evaluation result, and generate a target strategy;

[0244] The control module 15 is configured to control the target robot to re-execute the target task according to the target strategy.

[0245] Further, in one embodiment, the robot abnormality processing device, wherein the real-time acquisition of the state data and the environment data of the target robot when the target robot executes the target task according to the task execution strategy specifically comprises:

[0246] Obtaining a language instruction of a user, and analyzing the language instruction by using a natural language processing technology, and constructing a task execution strategy according to an analysis result;

[0247] According to the task execution strategy, the target robot is controlled to execute the target task;

[0248] The state data and the environment data of the target robot when executing the target task are collected in real time by various sensors carried by the target robot;

[0249] The state data includes position information, speed information, attitude information and battery information of the target robot; and the environment data includes temperature information, humidity information, obstacle information and task object information.

[0250] Further, in one embodiment, the robot abnormality processing apparatus, wherein the pre-trained visual language large model is used to analyze the state data and the environment data to identify target abnormal factors, specifically including:

[0251] The state data and the environment data are preprocessed;

[0252] The preprocessed state data and environment data are feature extracted, and the extracted features are fused to obtain fused features;

[0253] The fused features are input into the pre-trained visual language large model for abnormality identification to generate a plurality of candidate abnormal factors, and a context perception algorithm is used to screen target abnormal factors meeting a preset rule from the plurality of candidate abnormal factors.

[0254] Further, the robot abnormality processing apparatus, wherein the preprocessed state data and environment data are feature extracted, and the extracted features are fused to obtain fused features, specifically including:

[0255] The preprocessed state data and environment data are encoded by the visual language large model to obtain corresponding state feature vectors and environment feature vectors;

[0256] The state feature vectors and the environment feature vectors are aligned, the state feature vectors and the environment feature vectors are mapped to the same feature space through linear transformation or nonlinear mapping, and the state feature vectors and the environment feature vectors are fused through attention mechanism to generate fused features.

[0257] Further, in one embodiment, the robot abnormality processing apparatus, wherein the influence degree of the target abnormal factor on the target robot executing the target task is evaluated to generate an evaluation result, specifically including:

[0258] An abnormal factor influence model is constructed based on historical data and predefined abnormal types;

[0259] The abnormal factor influence model is used to evaluate the degree of influence of the target abnormal factor on the target robot performing the target task, and an evaluation result is generated.

[0260] Further, in an embodiment, the robot abnormality processing apparatus, wherein the target strategy is generated by adjusting the task execution strategy based on the evaluation result and using the visual language large model, specifically comprising:

[0261] According to the evaluation result, a prompt text for adjusting the task execution strategy is constructed;

[0262] The task execution strategy and the prompt text are input into the visual language large model to generate a preliminary execution strategy;

[0263] In the virtual environment, the target robot is verified by executing the preliminary execution strategy, and when the verification result meets the preset requirement, the preliminary execution strategy is taken as the target strategy.

[0264] Further, in an embodiment, the robot abnormality processing apparatus, wherein the target strategy is generated by adjusting the task execution strategy based on the evaluation result and using the visual language large model, specifically comprising:

[0265] According to the task decomposition requirement in the target strategy, the target task is decomposed into a plurality of subtasks with a clear execution sequence;

[0266] According to the task execution instruction of each subtask in the target strategy, the target robot is controlled to process each subtask in sequence according to the execution sequence.

[0267] It should be noted that the information interaction, execution process and the like between the above modules in the device embodiment of the present application are based on the same concept as the method embodiment of the present application, and the specific functions and technical effects brought by them can be referred to the method embodiment part.

[0268] Based on the above method embodiment, another embodiment of the present application further provides a computer device, which can be a server, and the internal structure diagram thereof can be as shown in Figure 4As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the robot exception handling method server-side as described in any of the above method embodiments.

[0269] Based on the above method embodiments, another embodiment of the present invention also provides a computer device, which can be a client, and its internal structure diagram can be as follows. Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements the functions or steps of the robot exception handling method on the client side as described in any of the above method embodiments.

[0270] Those skilled in the art will understand that Figure 4 and Figure 5 The structural schematic diagram shown is only a schematic diagram of a part of the structure related to the present invention and does not constitute a limitation on the computer device on which the present invention is applied. The specific computer device may include more components than shown in the figure, or combine certain components, or have different component arrangements.

[0271] The processor referred to herein can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0272] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be a memory of the computer device, and the internal memory provides an environment for running the operating system and the computer readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the computer device, and in other embodiments, can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory can include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0273] Based on the above method embodiments, another embodiment of the present application further provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the robot exception handling method in any one of the above method embodiments. The computer readable storage medium can be non-volatile or volatile.

[0274] It should be noted that the functions or steps that the computer readable storage medium or the computer device can achieve and the technical effects brought by the functions / steps can be referred to the related description in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0275] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc. The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable type of memory.

[0276] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, in the device embodiment of the present application, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0277] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0278] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0279] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0280] It should be noted that if non-company software tools or components appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A robot exception handling method characterized by, The method comprises: real-time collection of state data and environment data of a target robot when the target robot executes a target task according to a task execution strategy; analysis of the state data and the environment data by using a pre-trained visual language large model to identify a target abnormal factor; evaluation of an influence degree of the target abnormal factor on execution of the target task by the target robot, to generate an evaluation result; adjustment of the task execution strategy by using the visual language large model based on the evaluation result, to generate a target strategy; re-execution of the target task by the target robot according to the target strategy; the analysis of the state data and the environment data by using the pre-trained visual language large model to identify the target abnormal factor comprises: preprocessing of the state data and the environment data; feature extraction of the preprocessed state data and environment data, and fusion of the extracted features to obtain fused features; input of the fused features into the pre-trained visual language large model for abnormality identification, to generate a plurality of candidate abnormal factors, and use of a context perception algorithm to screen a target abnormal factor meeting a preset rule from the plurality of candidate abnormal factors; the feature extraction of the preprocessed state data and environment data, and the fusion of the extracted features to obtain fused features, comprise: encoding processing of the preprocessed state data and environment data by using the visual language large model to obtain corresponding state feature vectors and environment feature vectors; alignment processing of the state feature vectors and the environment feature vectors, linear transformation or nonlinear mapping of the state feature vectors and the environment feature vectors into the same feature space, and fusion of the state feature vectors and the environment feature vectors by using an attention mechanism to generate fused features; the adjustment of the task execution strategy by using the visual language large model based on the evaluation result to generate a target strategy, comprises: construction of a prompt text for adjusting the task execution strategy according to the evaluation result; input of the task execution strategy and the prompt text into the visual language large model to generate a preliminary execution strategy; verification of the target robot executing the preliminary execution strategy in a virtual environment, and taking the preliminary execution strategy as a target strategy when the verification result meets a preset requirement.

2. The robot anomaly handling method of claim 1, wherein, The real-time collection of state data and environment data of a target robot when the target robot executes a target task according to a task execution strategy comprises: acquisition of a language instruction of a user, analysis of the language instruction by using natural language processing technology, construction of a task execution strategy according to an analysis result; control of a target robot to execute a target task according to the task execution strategy; real-time collection of state data and environment data of the target robot when the target robot executes the target task by using a plurality of sensors carried by the target robot; wherein the state data comprises position information, speed information, attitude information and battery information of the target robot, and the environment data comprises temperature information, humidity information, obstacle information and task object information.

3. The robot exception handling method of claim 1, wherein, The evaluation of the influence degree of the target abnormal factor on the target robot performing the target task generates an evaluation result, including: Based on historical data and predefined abnormal types, an abnormal factor influence model is constructed; The influence of the target abnormal factor on the target robot performing the target task is evaluated using the abnormal factor influence model, and an evaluation result is generated.

4. The robot exception handling method of claim 1, wherein, Based on the target strategy, the target robot is controlled to re-execute the target task, including: According to the task decomposition requirement in the target strategy, the target task is decomposed into a plurality of sub-tasks with a clear execution order; According to the task execution instruction of each sub-task in the target strategy, the target robot is controlled to process each sub-task in turn according to the execution order.

5. A robot abnormality processing device characterized by comprising: It includes: The acquisition module is used for collecting the state data and environment data of the target robot performing the target task according to the task execution strategy in real time; The analysis module is used for analyzing the state data and the environment data by using the pre-trained visual language large model, and identifying the target abnormal factor; The evaluation module is used for evaluating the influence degree of the target abnormal factor on the target robot performing the target task, and generating an evaluation result; The adjustment module is used for adjusting the task execution strategy based on the evaluation result by using the visual language large model, and generating a target strategy; The control module is used for controlling the target robot to re-execute the target task according to the target strategy; The pre-trained visual language large model is used to analyze the state data and the environment data, and identify the target abnormal factor, including: The state data and the environment data are preprocessed; The state data and the environment data after preprocessing are feature extracted, and the extracted features are fused to obtain fusion features; The fusion features are input into the pre-trained visual language large model for anomaly identification, a plurality of candidate abnormal factors are generated, and a context perception algorithm is used to screen out target abnormal factors that meet the preset rules from the plurality of candidate abnormal factors; The state data and the environment data after preprocessing are feature extracted, and the extracted features are fused to obtain fusion features, including: The visual language large model is used to encode the state data and the environment data after preprocessing to obtain corresponding state feature vectors and environment feature vectors; The state feature vectors and the environment feature vectors are aligned, the state feature vectors and the environment feature vectors are mapped to the same feature space through linear transformation or nonlinear mapping, and the state feature vectors and the environment feature vectors are fused by using attention mechanism to generate fusion features; Based on the evaluation result, the task execution strategy is adjusted by using the visual language large model to generate a target strategy, including: According to the evaluation result, a prompt text for adjusting the task execution strategy is constructed; The task execution strategy and the prompt text are input into the visual language large model to generate a preliminary execution strategy; In the virtual environment, the preliminary execution strategy for the target robot is verified, and when a verification result meets preset requirements, the preliminary execution strategy is taken as a target strategy.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the robot exception handling method according to any one of claims 1-4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the robot exception handling method according to any one of claims 1-4.

Images