Robot state transition model construction method and device, terminal equipment and robot action execution state adjustment method

Abstract

The invention discloses a robot state transition model construction method and device, terminal equipment and a robot action execution state adjustment method, and belongs to the field of artificial intelligence. The method comprises the steps that a natural language instruction, an initial state vector and an actual state transition matrix sequence in each demonstration sample are used as input data of a deep learning model; taking the predicted and generated state transition matrix sequence as output data of the deep learning model, and carrying out iterative training on the deep learning model until the trained deep learning model is obtained, and taking the trained deep learning model as a robot state transition model; and in each training process, determining whether the current deep learning model meets a convergence condition according to a state transition matrix sequence generated by prediction of the current deep learning model and a corresponding actual state transition matrix sequence, if so, completing training, and if not, adjusting model parameters to continue training. By implementing the method, the problem of poor robot action adaptability in an electric power operation scene in the prior art can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a robot state transition model construction method and device, a terminal device, and a robot action execution state adjustment method. BACKGROUND

[0002] With the continuous advancement and development of industrial automation, in power equipment inspection, power grid operation and other special scenarios, the demand for robot understanding and accurate execution of complex action instructions is increasingly urgent. In such scenarios, the robot needs to complete fine operations such as power line inspection, substation equipment maintenance, and distribution room component disassembly, and the accuracy of action execution is directly related to the safety of power grid operation.

[0003] Current robot action control mainly relies on artificially preset fixed execution sequences or rule-based state transition logic. However, such solutions have significant limitations. On the one hand, the working environment of the robot in the power grid scenario is dynamic and variable (such as differences in device layout and different line directions), resulting in differences in the initial state of the robot joints. On the other hand, different power equipment (such as circuit breakers, insulators, and transformers) require different maintenance actions, and fixed execution sequences cannot meet the mapping relationship between the diversified initial state and the specific target state, resulting in low action execution accuracy and poor adaptability, and may also touch the live components of the power grid due to action deviation, causing safety hazards, and far from meeting the accuracy and safety requirements of robot action in complex scenarios such as power grid operation. SUMMARY

[0004] The present application provides a robot state transition model construction method, device, terminal device, and robot action execution state adjustment method, which can solve the problem of poor robot action adaptability in the prior art power operation scenario.

[0005] To solve the above technical problems, an embodiment of the present application provides a robot state transition model construction method, comprising: Obtaining an action instruction sample set; wherein the action instruction sample set contains demonstration samples of several different power operation scenarios; each demonstration sample includes a natural language instruction, an initial state vector before joint execution instruction, and an actual state transition matrix sequence; Taking the natural language instruction, the initial state vector, and the actual state transition matrix sequence in each demonstration sample as the input data of a preset deep learning model, and taking the generated state transition matrix sequence as the output data of the deep learning model, the deep learning model is iteratively trained until a trained deep learning model is obtained; The trained deep learning model is used as a robot state transition model; Wherein, in each training process, according to the state transition matrix sequence generated by the current deep learning model prediction and the corresponding actual state transition matrix sequence, it is determined whether the current deep learning model meets the preset convergence condition, if yes, the iterative training is terminated, and the trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted for the next training.

[0006] Further, the generation process of the actual state transition matrix sequence comprises: For each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence; The semantic vector sequence is input into a preset encoder, so that the encoder extracts the hidden state conversion represented by the semantic vector sequence, and inputs the hidden state conversion into a preset decoder; The initial state vector corresponding to the current natural language instruction and the semantic vector sequence are input into the decoder, so that the decoder combines the hidden state conversion, the semantic vector sequence and the initial state vector corresponding to the current natural language instruction to generate an actual state transition matrix sequence; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after joint execution instruction after matrix transformation.

[0007] Further, the convergence condition comprises that the mean square error value is less than a preset value; The convergence condition comprises that the mean square error value is less than a preset value; According to the state transition matrix sequence generated by the current deep learning model prediction and the corresponding initial state vector, the current final state vector is calculated; According to the current final state vector and the corresponding target final state vector, the state deviation is calculated; According to the state deviation, the current mean square error value is calculated; It is determined whether the current mean square error value is less than a preset value, If yes, the iterative training is terminated, and the trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted for the next training.

[0008] Further, the convergence condition further comprises that the deviation norm is not greater than a preset feasibility threshold; The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets a preset convergence condition, and if yes, the iterative training is terminated to obtain the trained deep learning model, and if not, the model parameters of the current deep learning model are adjusted for next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding initial state vector are used to calculate a current final state vector; The current final state vector and the corresponding target final state vector are used to calculate a state deviation; The state deviation is used to calculate a current deviation norm; It is determined whether the current deviation norm is not greater than a preset feasibility threshold, If yes, the iterative training is terminated to obtain the trained deep learning model, and if not, the model parameters of the current deep learning model are adjusted for next training.

[0009] An embodiment of the present application further provides a robot state transition model construction device, which comprises a data acquisition module configured to acquire an action instruction sample set; wherein the action instruction sample set comprises demonstration samples of different electric power operation scenes; each demonstration sample comprises a natural language instruction, an initial state vector before joint execution instruction and an actual state transition matrix sequence; A training module is configured to perform iterative training on a preset deep learning model by taking the natural language instruction, the initial state vector and the actual state transition matrix sequence in each demonstration sample as input data of the deep learning model and taking a state transition matrix sequence generated by prediction as output data of the deep learning model, until a trained deep learning model is obtained; wherein in each training process, the state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets a preset convergence condition, and if yes, the iterative training is terminated to obtain the trained deep learning model, and if not, the model parameters of the current deep learning model are adjusted for next training. A model generation module is configured to take the trained deep learning model as a robot state transition model.

[0010] Further, the generation process of the actual state transition matrix sequence comprises: For each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence; input the semantic vector sequence into a preset encoder, so that the encoder extracts a hidden state transition represented by the semantic vector sequence, and inputs the hidden state transition into a preset decoder; input the initial state vector corresponding to the current natural language instruction and the semantic vector sequence into the decoder, so that the decoder generates an actual state transition matrix sequence in combination with the hidden state transition, the semantic vector sequence and the initial state vector corresponding to the current natural language instruction; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after the joint execution instruction through matrix transformation.

[0011] Further, the convergence condition includes that the mean square error value is less than a preset value. The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: determine whether the current mean square error value is less than a preset value, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training.

[0012] Further, the convergence condition further includes that the bias norm is not greater than a preset feasibility threshold. The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training, including: determining whether the current deviation norm is not greater than a preset feasibility threshold value, If yes, the iterative training is terminated to obtain the trained deep learning model, and if no, the model parameters of the current deep learning model are adjusted for the next training.

[0013] An embodiment of the present application also provides a robot action execution state adjustment method, comprising: obtaining a target action instruction in a power operation scene input by a user; identifying an initial unexecuted state of a required joint of the target action instruction; inputting the target action instruction and the initial unexecuted state into a robot state transition model to obtain a target state transition matrix sequence output by the robot state transition model; wherein the robot state transition model is determined by the robot state transition model construction method; controlling gradual adjustment of action execution states of corresponding required joints according to the target state transition matrix sequence and a preset state machine.

[0014] The present application also provides a terminal device, comprising: one or more processors; a memory coupled to the processor, for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the robot state transition model construction method as described in the above embodiments.

[0015] The present application has the following beneficial effects: The present application provides a robot state transition model construction method, device, terminal device and robot action execution state adjustment method, the action instruction sample set obtained by the method contains natural language instruction, initial state vector before joint execution instruction and actual state transition matrix sequence for each demonstration sample, covering full-dimensional data of different devices, different initial states and different action requirements in power grid scenarios; in the deep learning model training process, because the input data fits the actual scenario difference of power grid operation and maintenance, the model can learn the mapping relationship between the natural language instruction, the initial state vector and the actual state transition matrix sequence; and the iterative training process enables the deep learning model to fully master the diversified action execution rules in the power grid scenario, which can adapt to different operation scenarios such as power transmission line inspection and substation maintenance, and can also cope with situations such as joint initial state difference, effectively solving the problem that fixed execution sequence cannot adapt to dynamic differences, greatly improving the action execution accuracy and avoiding safety hazards caused by action deviation touching live components. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described below only relate to some of the embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0017] Figure 1 is a flowchart of a robot state transition model construction method provided by an embodiment of the present application; Figure 2 is a GRU-based sequence-to-sequence model architecture diagram provided by an embodiment of the present application; Figure 3 is a structural diagram of a robot state transition model construction device provided by an embodiment of the present application; Figure 4 is a flowchart of a robot action execution state adjustment method provided by an embodiment of the present application; Figure 5 is a control principle diagram of a state machine provided by an embodiment of the present application; Figure 6 is a structural diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort are within the scope of protection of the present application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0021] Reference to an“embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase that the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of other embodiments. It is explicitly contemplated that embodiments described herein can be combined with other embodiments in combinations that are not explicitly stated nor inherently opposed.

[0022] In the description of the embodiments of the application, the term“and / or” only means a relationship between associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, the character“ / ” herein generally means that the front and rear associated objects are in an“or” relationship.

[0023] In the description of the embodiments of the application, the term“multiple” refers to two or more (including two), and similarly, “multiple groups” refers to two or more groups (including two groups), and “multiple pieces” refers to two or more pieces (including two pieces).

[0024] In the description of the embodiments of the application, unless otherwise explicitly specified and limited, the technical terms“mounting”,“connection”,“connection”,“fixing” and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanical connection, or it can be electrical connection; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the application can be understood according to the specific circumstances.

[0025] Reference Figure 1 To solve the problem of poor action adaptability of robots in the existing power operation scene, an embodiment of the application provides a robot state transition model construction method, which comprises: S101, acquiring an action instruction sample set; wherein the action instruction sample set contains demonstration samples of several different power operation scenes; each demonstration sample includes a natural language instruction, an initial state vector before joint execution instruction, and an actual state transition matrix sequence; Illustratively, in order to better enable the subsequent deep learning model to learn the mapping rule of natural language instruction, joint state and action execution in the power grid scene, before that, it is necessary to obtain demonstration samples containing several different power operation scenes, and each demonstration sample includes a natural language instruction, an initial state vector before joint execution instruction, and an actual state transition matrix sequence, to ensure the full dimension and scene adaptability of sample data; Specifically, due to the semantic complexity and expression diversity of natural language instructions, when obtaining the natural language instructions, the natural language instructions need to be semantically analyzed and vectorized first, so as to determine the actual state transition matrix sequence: In a preferred embodiment, the generation process of the actual state transition matrix sequence comprises: For each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence; The semantic vector sequence is input into a preset encoder, so that the encoder extracts the hidden state transition represented by the semantic vector sequence, and inputs the hidden state transition into a preset decoder; The initial state vector corresponding to the current natural language instruction and the semantic vector sequence are input into the decoder, so that the decoder combines the hidden state transition, the semantic vector sequence and the initial state vector corresponding to the current natural language instruction to generate an actual state transition matrix sequence; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after the joint execution instruction after matrix transformation; Illustratively, see Figure 2 The sequence-to-sequence model architecture based on GRU adopted by the present application includes core modules: encoder and decoder; Specifically, for each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence, which is then input into an encoder composed of multiple GRU units. The encoder processes the semantic vector sequence word by word and extracts the hidden state transition (i.e. context vector c) represented thereby; then the hidden state transition is input into a decoder also composed of multiple GRU units, while the initial state vector corresponding to the current natural language instruction before the joint execution instruction and the semantic vector sequence are input into the decoder; the decoder combines the hidden state transition, the semantic vector sequence and the initial state vector to gradually generate the target final state vector after the joint execution instruction , and outputs the actual state transition matrix sequence ; It should be noted that the actual state transition matrix sequence refers to the complete state transition trajectory of the robot from the initial state vector to the target final state vector after real action execution under the action of natural language instructions, i.e. a sequence composed of each instantaneous state transition.

[0026] S102, taking the natural language instructions, the initial state vector and the actual state transition matrix sequence in each demonstration sample as input data of a preset deep learning model, taking a state transition matrix sequence generated by prediction as output data of the deep learning model, iteratively training the deep learning model until a trained deep learning model is obtained; wherein in each training process, whether the current deep learning model meets a preset convergence condition is determined according to the state transition matrix sequence generated by prediction of the current deep learning model and the corresponding actual state transition matrix sequence, if yes, the iterative training is terminated to obtain the trained deep learning model, if no, the model parameters of the current deep learning model are adjusted for next training; Specifically, taking the natural language instructions, the initial state vector and the actual state transition matrix sequence in each demonstration sample as input data of a preset deep learning model, the deep learning model will first randomly generate a set of state transition matrix sequences As a starting point of iterative training, whether the current deep learning model meets a preset convergence condition is further determined based on the state transition matrix sequence generated by prediction of the current deep learning model and the corresponding actual state transition matrix sequence, specifically, two convergence conditions are set in the present application, as long as at least one convergence condition is met, it is determined to be converged, and the specific determination process is as follows: In a preferred embodiment, the convergence condition includes that the mean square error value is less than a preset value; The determination whether the current deep learning model meets a preset convergence condition according to the state transition matrix sequence generated by prediction of the current deep learning model and the corresponding actual state transition matrix sequence, if yes, the iterative training is terminated to obtain the trained deep learning model, if no, the model parameters of the current deep learning model are adjusted for next training, includes: The current final state vector is calculated according to the state transition matrix sequence generated by prediction of the current deep learning model and the corresponding initial state vector; The state deviation is calculated according to the current final state vector and the corresponding target final state vector; The current mean square error value is calculated according to the state deviation; It is judged whether the current mean square error value is less than a preset value, If yes, the iterative training is terminated to obtain the trained deep learning model, if no, the model parameters of the current deep learning model are adjusted for next training; Specifically, the current final state vector is calculated according to the state transition matrix sequence generated by prediction of the current deep learning model The current final state vector is calculated according to the state transition matrix sequence generated by prediction of the current deep learning model and the corresponding initial state vector; ; a current final state vector is calculated according to the state transition matrix sequence generated by the current deep learning model and the corresponding initial state vector a state deviation is calculated according to the current final state vector and the corresponding target final state vector ; ; a current mean square error value is calculated according to the state deviation ; ; wherein, dim represents the dimension of the final state vector ;it is determined whether the current mean square error value is less than a preset value, if yes, the iteration training is terminated to obtain the deep learning model after training, if not, the model parameters of the current deep learning model are adjusted, specifically, the learning rate of the current deep learning model needs to be adjusted, and then the state transition matrix sequence generated by the current prediction is adjusted according to the adjusted learning rate and the current mean square error value ; ; ; ; ; ; It should be noted that the preset value can be set according to actual conditions.

[0027] In a preferred embodiment, the convergence condition further includes that the deviation norm is not greater than a preset feasibility threshold; the current deep learning model is determined whether to meet the preset convergence condition according to the state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence, if yes, the iteration training is terminated to obtain the deep learning model after training, if not, the model parameters of the current deep learning model are adjusted for next training, including: a current final state vector is calculated according to the state transition matrix sequence generated by the current deep learning model and the corresponding initial state vector; a state deviation is calculated according to the current final state vector and the corresponding target final state vector a current deviation norm is calculated according to the state deviation it is determined whether the current deviation norm is not greater than a preset feasibility threshold, if yes, the iteration training is terminated to obtain the deep learning model after training, if not, the model parameters of the current deep learning model are adjusted for next training; Specifically, a current final state vector is calculated according to the state transition matrix sequence generated by the current deep learning model : ; Then, based on the current final state vector With the corresponding target final state vector The state deviation was calculated. ; ; Based on the state deviation, the current deviation norm is calculated. If the current deviation norm is not greater than a preset feasibility threshold... ,Right now The iterative training is terminated to obtain the trained deep learning model; otherwise, the model parameters of the current deep learning model are adjusted. Specifically, the learning rate of the current deep learning model needs to be adjusted. Make adjustments, and then based on the adjusted learning rate Combined with the current mean square error value For the currently predicted state transition matrix sequence Make adjustments, that is ; It should be noted that the feasibility threshold It can be set according to the actual situation.

[0028] S103. Use the trained deep learning model as the robot's state transition model.

[0029] See Figure 3 This invention provides a robot state transition model construction device according to an embodiment of the present invention, comprising: The data acquisition module is used to acquire a set of action instructions; wherein, the set of action instructions contains several demonstration samples of different power operation scenarios; each demonstration sample includes a natural language instruction, an initial state vector before the joint execution instruction, and a sequence of actual state transition matrices. The training module is used to iteratively train the deep learning model using the natural language instructions, the initial state vector, and the actual state transition matrix sequence from each demonstration sample as input data for a preset deep learning model, and the predicted state transition matrix sequence as output data for the deep learning model, until a trained deep learning model is obtained. During each training process, based on the predicted state transition matrix sequence and the corresponding actual state transition matrix sequence of the current deep learning model, it is determined whether the current deep learning model meets the preset convergence condition. If yes, the iterative training is terminated, and a trained deep learning model is obtained; otherwise, the model parameters of the current deep learning model are adjusted, and the next training iteration is performed. The model generation module is used to use the trained deep learning model as the robot's state transition model.

[0030] In a preferred embodiment, the generation process of the actual state transition matrix sequence comprises: For each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence; The semantic vector sequence is input into a preset encoder, so that the encoder extracts the hidden state transition represented by the semantic vector sequence and inputs the hidden state transition into a preset decoder; The initial state vector corresponding to the current natural language instruction and the semantic vector sequence are input into the decoder, so that the decoder combines the hidden state transition, the semantic vector sequence and the initial state vector corresponding to the current natural language instruction to generate an actual state transition matrix sequence; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after joint execution instruction after matrix transformation.

[0031] In a preferred embodiment, the convergence condition comprises a mean square error value less than a preset value; The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are determined to determine whether the current deep learning model satisfies the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, if not, the model parameters of the current deep learning model are adjusted for next training, comprising: The state transition matrix sequence generated by the current deep learning model and the corresponding initial state vector are calculated to obtain the current final state vector; The state deviation is calculated according to the current final state vector and the corresponding target final state vector; The current mean square error value is calculated according to the state deviation; It is judged whether the current mean square error value is less than a preset value, If yes, the iteration training is terminated to obtain the trained deep learning model, if not, the model parameters of the current deep learning model are adjusted for next training.

[0032] In a preferred embodiment, the convergence condition further comprises a bias norm not greater than a preset feasibility threshold; The state transition matrix sequence generated by the current deep learning model and the corresponding actual state transition matrix sequence are determined to determine whether the current deep learning model satisfies the preset convergence condition, if yes, the iteration training is terminated to obtain the trained deep learning model, if not, the model parameters of the current deep learning model are adjusted for next training, comprising: According to the state transition matrix sequence generated by the current deep learning model prediction and the corresponding initial state vector, the current final state vector is calculated; According to the current final state vector and the corresponding target final state vector, a state deviation is calculated; According to the state deviation, a current deviation norm is calculated; Determine whether the current deviation norm is not greater than a preset feasibility threshold, If yes, terminate the iterative training to obtain the trained deep learning model, and if not, adjust the model parameters of the current deep learning model for the next training.

[0033] Referring to Figure 4 is a robot action execution state adjustment method provided by an embodiment of the application, comprising: S201, obtaining a target action instruction in a power operation scene input by a user; Specifically, the target action instruction input by the user is obtained, and the instruction needs to clearly indicate the core task of power operation, such as driving the climbing robot to climb along the tower to 15 meters high and detecting the temperature of the wire joint.

[0034] S202, identifying the initial unexecuted state of the joint required by the target action instruction; Specifically, the initial unexecuted state of the corresponding joint is collected by the multi-dimensional sensor system carried by the climbing robot, such as the current posture of the moving joint (such as the tracked chassis and the climbing mechanism), to ensure that the vector dimension is consistent with the input requirements of the robot state transition model.

[0035] S203, inputting the target action instruction and the initial unexecuted state into the robot state transition model to obtain a target state transition matrix sequence output by the robot state transition model; wherein the robot state transition model is determined by the robot state transition model construction method; Specifically, the target action instruction and the initial unexecuted state are input into the robot state transition model to obtain a target state transition matrix sequence output by the robot state transition model.

[0036] S204, according to the target state transition matrix sequence and the preset state machine, controlling the required joints to gradually adjust the action execution state; Specifically, referring to Figure 5The control principle of the state machine is as follows: first, the target state transition matrix sequence is mapped and matched with a preset state machine, the state machine has predefined the action logic, safety constraint threshold and state switching condition of each joint (caterpillar chassis, climbing mechanism and detection holder) of the climbing robot; then, state conversion instructions are sent to the corresponding joints one by one in the time sequence order of the target state transition matrix sequence: the climbing mechanism is started and the posture of the caterpillar chassis is adjusted to ensure that the initial state vector enters the stable climbing state after matrix transformation; the uniform speed and stable climbing from the current height to the 15-meter target height is realized by gradually adjusting the traction force and joint angle of the climbing mechanism, during which the state machine verifies the actual state of the joint after each step of matrix transformation in real time, if the deviation exceeds the preset threshold, the error correction mechanism of the state machine is triggered; finally, the climbing mechanism is stopped, the body is locked and fixed, and the temperature probe holder is driven to adjust to the angle of the wire joint, the final stability of the action execution state is completed, and the entire process meets the safety specifications and precise control requirements of power high-voltage operation.

[0037] Referring to Figure 6 The embodiment of the present application also provides a terminal device, comprising: one or more processors; a memory coupled to the processors, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the robot state transition model construction method as described above.

[0038] The processor is configured to control the overall operation of the terminal device to complete all or part of the steps of the robot state transition model construction method described above. The memory is configured to store various types of data to support the operation of the terminal device, which may include, for example, instructions for any application or method operating on the terminal device, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0039] In an exemplary embodiment, the terminal device can be implemented by one or more Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic elements for executing the robot state transition model construction method as described in any one of the above embodiments and achieving the consistent technical effects as the above method.

[0040] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which are also considered within the scope of protection of the present application.

Claims

1. A method for constructing a robot state transition model, characterized by, The method comprises the following steps: acquiring an action instruction sample set, wherein the action instruction sample set comprises demonstration samples of different power operation scenarios, each demonstration sample comprises a natural language instruction, an initial state vector before joint execution instruction, and an actual state transition matrix sequence; using the natural language instruction, the initial state vector, and the actual state transition matrix sequence in each demonstration sample as input data of a preset deep learning model, using a predictively generated state transition matrix sequence as output data of the deep learning model, iteratively training the deep learning model until a trained deep learning model is obtained; using the trained deep learning model as a robot state transition model; in each training process, determining whether the current deep learning model meets a preset convergence condition according to a predictively generated state transition matrix sequence of the current deep learning model and a corresponding actual state transition matrix sequence, if yes, terminating the iterative training to obtain the trained deep learning model, if no, adjusting model parameters of the current deep learning model for the next training.

2. The robot state transition model construction method of claim 1, wherein, The generation process of the actual state transition matrix sequence comprises the following steps: for each natural language instruction, converting the current natural language instruction into a corresponding semantic vector sequence; inputting the semantic vector sequence into a preset encoder to enable the encoder to extract a hidden state conversion represented by the semantic vector sequence and input the hidden state conversion into a preset decoder; inputting the initial state vector corresponding to the current natural language instruction and the semantic vector sequence into the decoder to enable the decoder to combine the hidden state conversion, the semantic vector sequence, and the initial state vector corresponding to the current natural language instruction to generate an actual state transition matrix sequence; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after matrix transformation and joint execution instruction.

3. The robot state transition model construction method of claim 2, wherein, the convergence condition comprises a mean square error value less than a preset value; the determination of whether the current deep learning model meets the preset convergence condition according to the predictively generated state transition matrix sequence of the current deep learning model and the corresponding actual state transition matrix sequence, if yes, terminating the iterative training to obtain the trained deep learning model, if no, adjusting the model parameters of the current deep learning model for the next training, comprises the following steps: calculating a current final state vector according to the predictively generated state transition matrix sequence of the current deep learning model and the corresponding initial state vector; calculating a state deviation according to the current final state vector and a corresponding target final state vector; calculating a current mean square error value according to the state deviation; determining whether the current mean square error value is less than a preset value, if yes, terminating the iterative training to obtain the trained deep learning model, if no, adjusting the model parameters of the current deep learning model for the next training.

4. The robot state transition model construction method of claim 3, wherein, the convergence condition further comprises a deviation norm not greater than a preset feasibility threshold value; The state transition matrix sequence predicted by the current deep learning model and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets a preset convergence condition, and if yes, the iterative training is terminated to obtain the trained deep learning model, and if not, the model parameters of the current deep learning model are adjusted for the next training. The current final state vector is calculated according to the state transition matrix sequence predicted by the current deep learning model and the corresponding initial state vector; The state deviation is calculated according to the current final state vector and the corresponding target final state vector; The current deviation norm is calculated according to the state deviation; It is determined whether the current deviation norm is not greater than a preset feasibility threshold, If yes, the iterative training is terminated to obtain the trained deep learning model, and if not, the model parameters of the current deep learning model are adjusted for the next training.

5. A robot state transition model construction apparatus characterized by comprising: The data acquisition module is configured to acquire an action instruction sample set, wherein the action instruction sample set includes demonstration samples of different electric power operation scenarios, and each demonstration sample includes a natural language instruction, an initial state vector before a joint execution instruction, and an actual state transition matrix sequence. The training module is configured to perform iterative training on a preset deep learning model by taking the natural language instruction, the initial state vector, and the actual state transition matrix sequence in each demonstration sample as input data of the deep learning model and taking a predicted state transition matrix sequence as output data of the deep learning model, until a trained deep learning model is obtained. The model generation module is configured to generate the trained deep learning model as a robot state transition model. The generation process of the actual state transition matrix sequence includes:

6. The robot state transition model construction apparatus according to claim 5, wherein For each natural language instruction, the current natural language instruction is converted into a corresponding semantic vector sequence; The semantic vector sequence is input into a preset encoder to enable the encoder to extract a hidden state conversion represented by the semantic vector sequence and input the hidden state conversion into a preset decoder; The initial state vector corresponding to the current natural language instruction and the semantic vector sequence are input into the decoder to enable the decoder to combine the hidden state conversion, the semantic vector sequence, and the initial state vector corresponding to the current natural language instruction to generate an actual state transition matrix sequence; wherein the actual state transition matrix sequence is used to convert the current initial state vector into a target final state vector after a joint execution instruction through matrix transformation. The convergence condition includes a mean square error value less than a preset value.

7. The robot state transition model construction apparatus according to claim 5, wherein ​ The state transition matrix sequence generated according to the current deep learning model prediction and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets a preset convergence condition, if yes, the iterative training is terminated, and a trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted, and the next training is performed, including: The current final state vector is calculated according to the state transition matrix sequence generated according to the current deep learning model prediction and the corresponding initial state vector; The state deviation is calculated according to the current final state vector and the corresponding target final state vector; The current mean square error value is calculated according to the state deviation; It is judged whether the current mean square error value is less than a preset value, if yes, the iterative training is terminated, and a trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted, and the next training is performed.

8. The robot state transition model construction apparatus according to claim 7, wherein The convergence condition further includes that the deviation norm is not greater than a preset feasibility threshold; The state transition matrix sequence generated according to the current deep learning model prediction and the corresponding actual state transition matrix sequence are used to determine whether the current deep learning model meets a preset convergence condition, if yes, the iterative training is terminated, and a trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted, and the next training is performed, including: The current final state vector is calculated according to the state transition matrix sequence generated according to the current deep learning model prediction and the corresponding initial state vector; The state deviation is calculated according to the current final state vector and the corresponding target final state vector; The current deviation norm is calculated according to the state deviation; It is judged whether the current deviation norm is not greater than a preset feasibility threshold, if yes, the iterative training is terminated, and a trained deep learning model is obtained, if not, the model parameters of the current deep learning model are adjusted, and the next training is performed.

9. A terminal device, comprising: It includes: One or more processors; Memory coupled to the processor for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the robot state transition model construction method of the power system as claimed in claims 1-4.

10. A robot action execution state adjustment method characterized by comprising: It includes: Obtaining a target action instruction in a power operation scene input by a user; Identifying the initial unexecuted state of the required joint of the target action instruction; The target action instruction and the initial unexecuted state are input into the robot state transition model to obtain a target state transition matrix sequence output by the robot state transition model; wherein the robot state transition model is determined by the robot state transition model construction method of any one of claims 1-4; According to the target state transition matrix sequence and the preset state machine, the action execution state of the required joint is gradually adjusted.

Images