Robot Control Method, Device, Electronic Device, and Computer-Readable Medium

Abstract

Embodiments of the present disclosure disclose a robot control method, apparatus, electronic device, and computer-readable medium. A specific implementation of the method includes: receiving user instruction information; parsing and processing the user instruction information; in response to determining that the preset shortest distance first condition is satisfied and there is operation information that satisfies the preset movement condition, determining each operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition as the first operation information sequence; for each first operation information, determining the position information of the operation object information included; performing clustering processing on the first operation information sequence to obtain each first operation information group; generating a rearranged operation information sequence according to the operation information sequence and each first operation information group; for each operation information in the rearranged operation information sequence, controlling a target robot to perform a response operation. This implementation shortens the moving path length of the robot and saves the battery power resource of the robot.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and more particularly, to a robot control method, apparatus, electronic device, and computer-readable medium. Background Art

[0002] Robot control technology includes various control means for enabling a robot to complete various tasks and actions. In a home scenario, a user can issue instructions to the robot to make the robot perform corresponding operations. Currently, when controlling a robot to perform operations, the commonly adopted method is: pre-conducting task planning, perception and environmental modeling, and control strategy selection for the home scenario. When a user instruction requires the robot to perform multiple operations, the robot executes the operations completely in sequence.

[0003] However, the inventors have found that when the above method is adopted, the following technical problems often exist:

[0004] First, when the positions of multiple operations required to be performed by the robot are close but the execution orders are not adjacent, the robot needs to move back and forth in the scenario when executing the operations in sequence, resulting in a long moving path length and wasting the robot's power resources.

[0005] Second, when performing task planning, prior knowledge is not well utilized, resulting in a long time-consuming and inaccurate planned task, wasting computing resources and poor user experience.

[0006] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0007] The content part of the present disclosure is used to briefly introduce the concepts, which will be described in detail in the following detailed implementation part. The content part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0008] Some embodiments of the present disclosure propose a robot control method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the above background art section.

[0009] In a first aspect, some embodiments of the present disclosure provide a robot control method, the method comprising: receiving user instruction information corresponding to a target robot from a user; parsing and processing the user instruction information to obtain instruction parsing information, wherein the instruction parsing information includes an operation information sequence, each operation information in the operation information sequence includes operation object information and operation event information, and an operation type corresponds to the operation event information; in response to determining that the operation configuration information corresponding to the user for the target robot satisfies a preset shortest distance first condition, and there is operation information in the operation information sequence whose corresponding operation type satisfies a preset movement condition, determining each operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition as a first operation information sequence; for each first operation information in the first operation information sequence, determining position information corresponding to the operation object information included in the first operation information; performing clustering processing on the first operation information sequence according to the position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group; generating a rearranged operation information sequence according to the operation information sequence and each first operation information group; for each operation information in the rearranged operation information sequence, controlling the target robot to perform a response operation according to the operation object information and the operation event information included in the operation information.

[0010] Second aspect, some embodiments of the present disclosure provide a robot control device, the device comprising: a receiving unit configured to receive user instruction information of a user corresponding to a target robot; an analysis unit configured to analyze and process the user instruction information to obtain instruction analysis information, wherein the instruction analysis information includes an operation information sequence, each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type; a first determination unit configured to, in response to determining that the operation configuration information of the user corresponding to the target robot satisfies a preset shortest distance first condition, and there is operation information in the operation information sequence whose corresponding operation type satisfies a preset movement condition, determine each operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition as a first operation information sequence; a second determination unit configured to, for each first operation information in the first operation information sequence, determine the position information corresponding to the operation object information included in the first operation information; a clustering unit configured to perform clustering processing on the first operation information sequence according to the position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group; a generation unit configured to generate a rearranged operation information sequence according to the operation information sequence and each first operation information group; a control unit configured to, for each operation information in the rearranged operation information sequence, control the target robot to perform a response operation according to the operation object information and the operation event information included in the operation information.

[0011] Third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having stored thereon one or more programs, which when executed by the one or more processors cause the one or more processors to implement the method described in any implementation manner of the first aspect above.

[0012] Fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having stored thereon a computer program, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect above is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the robot control method of some embodiments of the present disclosure, the moving path length of the robot is shortened, and the power resource of the robot is saved. Specifically, the reason for the waste of the robot's power resource is that when the positions of multiple operations that the robot needs to execute are close but the execution order is not adjacent, when the robot executes the operations in order, it needs to move back and forth in the scene, resulting in a longer moving path length and causing waste of the robot's power resource. Based on this, in the robot control method of some embodiments of the present disclosure, first, user instruction information corresponding to the target robot is received from the user. Thus, the instruction issued by the user can be sensed. Then, the above user instruction information is parsed and processed to obtain instruction parsing information, wherein the above instruction parsing information includes an operation information sequence, and each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type. Thus, the instruction issued by the user can be parsed into the dimensions of the operation object and the operation event. Then, in response to determining that the operation configuration information corresponding to the user for the above target robot satisfies the preset shortest distance first condition, and there is an operation information in the above operation information sequence whose corresponding operation type satisfies the preset movement condition, each operation information in the above operation information sequence whose corresponding operation type satisfies the above preset movement condition is determined as the first operation information sequence. Thus, when the user configures the shortest distance strategy and the robot needs to move when executing the user instruction, the operation information that needs to be moved can be extracted for subsequent analysis. Secondly, for each first operation information in the above first operation information sequence, the position information corresponding to the operation object information included in the above first operation information is determined. Thus, the position of the operation object to be operated can be located. Next, according to the position information corresponding to each first operation information in the above first operation information sequence, clustering processing is performed on the above first operation information sequence to obtain each first operation information group. Thus, the extracted operation information can be clustered into multiple groups of data according to the position. Then, according to the above operation information sequence and the above each first operation information group, a rearranged operation information sequence is generated. Thus, under the constraint of the clustering result, each operation information in the operation information sequence is rearranged so that the operation information corresponding to the operation objects with a relatively close distance is arranged in adjacent positions. Finally, for each operation information in the above rearranged operation information sequence, according to the operation object information and operation event information included in the above operation information, the above target robot is controlled to execute the corresponding operation. Thus, the robot can be controlled to execute the operations in response to the user instruction according to the rearranged operation information sequence. Also, because the rearranged operation information sequence is sorted according to the distance characteristics of the operation objects, the operation information of the operation objects with a relatively close distance is close in the sequence, which can enable the robot to reach the position of the nearby operation object by moving a shorter distance without moving back and forth in the scene, thereby shortening the moving path length and saving the power resource of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of a robot control method according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a robot control device according to the present disclosure;

[0017] Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0019] In addition, it should be noted that, for the sake of convenience of description, only parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence relationship of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] For operations such as the collection, storage, and use of the user's personal information (such as user instruction information) involved in the present disclosure, before performing the corresponding operations, relevant organizations or individuals fulfill obligations including conducting a personal information security impact assessment, fulfilling the obligation of notification to the personal information subject, and obtaining the prior authorization and consent of the personal information subject.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 Flow 100 of some embodiments of a robot control method according to the present disclosure is shown. The robot control method includes the following steps:

[0026] Step 101, receiving user instruction information of the user corresponding to the target robot.

[0027] In some embodiments, the execution entity (such as a computing device) of the robot control method can receive the user instruction information of the user corresponding to the target robot through a wired connection method or a wireless connection method. Among them, the above execution entity can be the above target robot or an electronic device that can control the above target robot. The target robot can be a robot that can perform controllable operations in the current scene. Here, the robot can be, but is not limited to: a household robot, an industrial robot. The above user instruction information can be a voice instruction issued by the user or information used to control the robot sent by the user through a user terminal. For example, the user instruction information can be "Close the kitchen door, open the bedroom door, then turn off the kitchen light, and bring me the water cup in the bedroom". It should be noted that the above wireless connection method can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future-developed wireless connection methods.

[0028] Step 102, parsing and processing the user instruction information to obtain instruction parsing information.

[0029] In some embodiments, the above-mentioned execution entity may parse and process the above-mentioned user instruction information to obtain instruction parsing information. Among them, the above-mentioned instruction parsing information includes an operation information sequence. The operation information sequence may be each operation information parsed in the order in the user instruction information. Each operation information in the above-mentioned operation information sequence includes operation object information and operation event information. The operation object information may be the identifier of the object in the scenario that needs to be operated. For example, the operation object information may be "kitchen door". The operation event information may characterize the operation to be performed on the operation object. For example, the operation event information may be "close the door". The operation event information corresponds to an operation type. The operation type may characterize whether it is necessary to move to the position where the operation object is located when performing the operation event.

[0030] Optionally, the above-mentioned user instruction information may include at least one instruction sentence, and there is an arrangement order between the at least one instruction sentence. For example, the user instruction information "Close the kitchen door, open the bedroom door, then turn off the kitchen light, and bring me the water cup in the bedroom" may include four instruction sentences, which are "Close the kitchen door", "Open the bedroom door", "Then turn off the kitchen light", and "Bring me the water cup in the bedroom" in sequence.

[0031] In some alternative implementation manners of some embodiments, the above-mentioned execution entity may perform the following steps to parse and process the above-mentioned user instruction information to obtain instruction parsing information:

[0032] First step, for each instruction sentence included in the above-mentioned user instruction information, perform the following steps:

[0033] First sub-step, input the above-mentioned instruction sentence into the input layer of a pre-trained operation parsing model to obtain a set of token embedding vectors. Among them, the above-mentioned operation parsing model may be a named entity recognition model that takes text as input data and a set of annotation results as output. The above-mentioned operation parsing model may include an input layer, an encoding layer, and an output layer. The above-mentioned input layer may include a tokenization layer and an embedding layer. The above-mentioned output layer may include a sequence annotation layer. The above-mentioned tokenization layer may be used to tokenize the input user instruction information to obtain a set of tokens. The above-mentioned embedding layer may be used to generate token embedding vectors for each token in the set of tokens. The sequence annotation layer may be used to annotate each token with a serialized label. The sequence annotation layer may further include a linear layer and a softmax layer. The linear layer may be used to map the hidden state of each token to the serialized label space. The softmax layer may be used to calculate the probability distribution of each serialized label.

[0034] In the second sub-step, the above word segmentation embedding vector set is input into the above encoding layer to obtain a hidden state vector set. The above encoding layer can be used to output the hidden state vector of each word segmentation. The above encoding layer can include various encoder layers. Each encoder layer can include a multi-head attention mechanism and a feedforward neural network.

[0035] The third sub-step is to input the above hidden state vector set into the sequence annotation layer included in the above output layer to obtain an annotation result set, wherein each annotation result in the above annotation result set includes a segmentation, a serialization label corresponding to the segmentation, and a probability corresponding to the serialization label.

[0036] The fourth sub-step is to combine the annotation results whose corresponding serialized tags in the above annotation result set belong to the operation object type into an operation object annotation result set. The operation object type can be the type of the object that the characterization segmentation belongs to. For example, the serialized tags belonging to the operation object type may include but are not limited to: B-OBJ (Beginning of Object), I-OBJ (Inside of Object).

[0037] The fifth sub-step is to determine each word segment included in the above operation object annotation result set as a first word segmentation set.

[0038] The sixth sub-step is to serialize the first word segmentation set according to the serialization tags corresponding to the first word segmentation set to obtain the first word segmentation sequence. In practice, the execution subject may put the first word segmentation corresponding to the B-OBJ tag in the first place, and put the first word segmentations corresponding to the I-OBJ tag in the first place to obtain the first word segmentation sequence. For example, when the instruction sentence is "close the kitchen door", the word segmentation set may be: put, kitchen, door, close. Among them, the serialization tag corresponding to "kitchen" may be B-OBJ. The serialization tag corresponding to "door" may be I-OBJ. The first word segmentation sequence may be: "kitchen", "door".

[0039] The seventh sub-step is to sequentially combine the first segmented words in the first segmented word sequence into operation object information. For example, the combined operation object information may be "kitchen door".

[0040] In the eighth sub-step, the annotation results whose corresponding serialized tags in the above annotation result set belong to the operation event type are combined into an operation event annotation result set. The operation event type can be the type of the characterization segmentation belonging to the action. For example, the serialized tags belonging to the operation event type may include but are not limited to: B-ACT (Beginning of Action), I-ACT (Inside of Action).

[0041] The ninth sub-step is to determine each word segment included in the above operation event annotation result set as the second word segment set.

[0042] The tenth sub-step is to perform serialization processing on the above second word segment set according to the respective serialization tags corresponding to the above second word segment set to obtain a second word segment sequence. In practice, the above execution entity may rank the second word segments corresponding to the B-ACT tag first, and rank the respective second word segments corresponding to the I-ACT tag in sequence after the first one to obtain a second word segment sequence. For example, the serialization tag corresponding to "close" may be B-ACT. Since there are no more word segments with the I-ACT tag later, "close" can be directly determined as the second word segment sequence.

[0043] The eleventh sub-step is to sequentially combine each second word segment in the above second word segment sequence into operation event information.

[0044] The twelfth sub-step is to determine the above operation object information and the above operation event information as operation information.

[0045] The second step is to combine the obtained respective operation information into an operation information sequence.

[0046] The third step is to determine the above operation information sequence as instruction parsing information. Thus, the parsing of multiple consecutive instructions of the user can be realized at a finer granularity.

[0047] Step 103, in response to determining that the operation configuration information of the user corresponding to the target robot satisfies the preset shortest distance first condition, and there is operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition, determine the operation information whose corresponding operation type in the operation information sequence satisfies the preset movement condition as the first operation information sequence.

[0048] In some embodiments, the above execution entity may, in response to determining that the operation configuration information of the user corresponding to the target robot satisfies the preset shortest distance first condition, and there is operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition, determine the operation information whose corresponding operation type in the operation information sequence satisfies the above preset movement condition as the first operation information sequence. Among them, the above operation configuration information may be relevant information configured by the user for the execution strategy of the target robot. For example, the above operation configuration information may include "give priority to the shortest distance when moving". Another example is that the above operation configuration information may include "give priority to executing in the order of instructions when moving". The above preset shortest distance first condition may be "the operation configuration information includes giving priority to the shortest distance when moving". The preset movement condition may be that the operation type indicates that the execution of the operation event requires moving to the position where the operation object is located.

[0049] Step 104: For each first operation information in the first operation information sequence, determine the position information of the operation object information included in the corresponding first operation information.

[0050] In some embodiments, the above-mentioned execution entity may, for each first operation information in the above-mentioned first operation information sequence, determine the position information of the operation object information included in the corresponding first operation information. The above-mentioned position information may represent the position of the operation object corresponding to the operation object information in the current scene.

[0051] In some alternative implementation manners of some embodiments, the above-mentioned execution entity may determine the position information of the operation object information included in the corresponding first operation information through the following steps:

[0052] First step: Determine the three-dimensional scene model corresponding to the current position of the above-mentioned target robot. Among them, the three-dimensional scene model may be previously constructed by the SLAM algorithm. The three-dimensional scene model may include various scene element objects. The scene element object may be an object in the scene.

[0053] Second step: Select the scene element object that matches the above-mentioned operation object information from the above-mentioned three-dimensional scene model. In practice, the above-mentioned execution entity may select the scene element object whose corresponding object name is the same as the above-mentioned operation object information from the above-mentioned various scene element objects.

[0054] Third step: Determine the three-dimensional coordinates of the above-mentioned scene element object in the three-dimensional coordinate system corresponding to the above-mentioned three-dimensional scene model.

[0055] Fourth step: Determine the above-mentioned three-dimensional coordinates as the position information of the operation object information included in the corresponding first operation information.

[0056] Step 105: According to the position information corresponding to each first operation information in the first operation information sequence, perform clustering processing on the first operation information sequence to obtain each first operation information group.

[0057] In some embodiments, the above-mentioned execution entity may perform clustering processing on the above-mentioned first operation information sequence according to the position information corresponding to each first operation information in the above-mentioned first operation information sequence to obtain each first operation information group.

[0058] In some alternative implementation manners of some embodiments, the above-mentioned execution entity may perform clustering processing on the above-mentioned first operation information sequence according to the position information corresponding to each first operation information in the above-mentioned first operation information sequence through the following steps to obtain each first operation information group:

[0059] First step: For every two first operation information in the above-mentioned first operation information sequence, perform the following steps:

[0060] In the first sub-step, determine the position information corresponding to the above two first operation information respectively to obtain two pieces of position information.

[0061] In the second sub-step, determine the horizontal coordinates corresponding to the above two pieces of position information as the first horizontal coordinate and the second horizontal coordinate respectively. The horizontal coordinate can include the abscissa and the ordinate. The horizontal coordinate does not include the vertical coordinate.

[0062] In the third sub-step, determine the distance between the above first horizontal coordinate and the above second horizontal coordinate as the object distance.

[0063] In the second step, based on the determined object distances, cluster each first operation information in the above first operation information sequence to obtain each first operation information group. In practice, the above execution entity can cluster each first operation information with a corresponding object distance less than a preset threshold into a group of first operation information to obtain each first operation information group.

[0064] Step 106, generate a rearranged operation information sequence according to the operation information sequence and each first operation information group.

[0065] In some embodiments, the above execution entity can generate a rearranged operation information sequence according to the above operation information sequence and each first operation information group.

[0066] In some optional implementation manners of some embodiments, the above execution entity can generate a rearranged operation information sequence according to the above operation information sequence and each first operation information group through the following steps:

[0067] In the first step, for each first operation information group in the above first operation information groups, perform the following steps:

[0068] In the first sub-step, determine the arrangement order information of each first operation information in the above first operation information group in the above operation information sequence. Among them, the arrangement order information can be an arrangement serial number.

[0069] In the second sub-step, sort each first operation information in the above first operation information group according to the respective arrangement order information corresponding to each first operation information in the above first operation information group to obtain a second operation information sequence. In practice, the above execution entity can sort each first operation information in the above first operation information group in ascending order to obtain a second operation information sequence.

[0070] In the third sub-step, determine the first second operation information in the above second operation information sequence as the first second operation information.

[0071] Second step, for each determined first second operation information, based on the operation information sequence, perform the following steps:

[0072] First sub-step, determine the second operation information sequence corresponding to the above-mentioned first second operation information as the third operation information sequence.

[0073] Second sub-step, determine each third operation information arranged after the above-mentioned second operation information in the above-mentioned third operation information sequence as the fourth operation information sequence.

[0074] Third sub-step, move each operation information in the operation information sequence corresponding to the above-mentioned fourth operation information sequence after the above-mentioned first second operation information in the operation information sequence to update the operation information sequence. For example, the fourth operation information sequence may include the 5th operation information and the 6th operation information in the operation information sequence. The first second operation information may be the 2nd operation information in the operation information sequence. The above-mentioned execution entity may move the 5th operation information and the 6th operation information in the operation information sequence after the 2nd operation information.

[0075] Third step, determine the updated operation information sequence as the rearranged operation information sequence. Thus, the operation information with relatively close corresponding operation objects can be arranged together.

[0076] Step 107, for each operation information in the rearranged operation information sequence, control the target robot to perform a response operation according to the operation object information and operation event information included in the operation information.

[0077] In some embodiments, the above-mentioned execution entity may, for each operation information in the above-mentioned rearranged operation information sequence, control the above-mentioned target robot to perform a response operation according to the operation object information and operation event information included in the above-mentioned operation information. In practice, the above-mentioned execution entity may control the above-mentioned target robot to perform the operation event corresponding to the above-mentioned operation event information on the operation object corresponding to the above-mentioned operation object information.

[0078] In some optional implementation manners of some embodiments, the above-mentioned execution entity may control the above-mentioned target robot to perform a response operation according to the operation object information and operation event information included in the above-mentioned operation information through the following steps:

[0079] First step, determine whether there is an operation guidance file corresponding to the above-mentioned operation object information. The operation guidance file may be a file for guiding the operation of the operation object corresponding to the operation object information. For example, the operation guidance file may be a specification file. The operation guidance file may be stored in PDF format.

[0080] Second step, in response to determining that there is an operation guidance file corresponding to the above-mentioned operation object information, perform the following steps:

[0081] The first sub-step is to extract the operation guidance content that matches the above operation object information and the above operation event information from the above operation guidance document. In practice, the above execution entity can extract the operation guidance content that matches the above operation object information and the above operation event information from the above operation guidance document through a document matching algorithm. For example, the document matching algorithm can be, but is not limited to: TF-IDF, a long document matching algorithm based on theme extraction and aggregation, and a maximum matching algorithm.

[0082] The second sub-step is to input the above operation guidance content into the input layer of a pre-trained operation guidance parsing model to obtain a set of guided token embedding vectors. Among them, the above operation guidance parsing model can be a named entity recognition model that takes operation guidance content as input and outputs various serialized tags representing guidance types and their corresponding probabilities. The guidance type can be represented by a tag type. The guidance type can include, but is not limited to: operation location, operation method, operation time, operation height. The above operation guidance parsing model can include an input layer, an encoding layer, and an output layer. The above input layer includes a tokenization layer and an embedding layer. The above output layer includes a sequence annotation layer. The tokenization layer is used to tokenize the input operation guidance content to obtain a set of guided tokens. The embedding layer is used to generate guided token embedding vectors for each guided token in the set of guided tokens. The sequence annotation layer can be used to annotate serialized tags for each guided token. The sequence annotation layer can further include a linear layer and a softmax layer. The linear layer can be used to map the hidden state of each guided token to the serialized tag space. The softmax layer can be used to calculate the probability distribution of each serialized tag.

[0083] The third sub-step is to input the above set of guided token embedding vectors into the above encoding layer to obtain a set of guided hidden state vectors. Among them, the above encoding layer includes various encoder layers. Each encoder layer includes a multi-head attention mechanism and a feed-forward neural network.

[0084] The fourth sub-step is to input the above set of guided hidden state vectors into the sequence annotation layer included in the above output layer to obtain a set of operation guidance annotation results. Among them, each operation guidance annotation result in the above set of operation guidance annotation results includes a guided token, the serialized tag corresponding to the guided token, and the probability corresponding to the serialized tag.

[0085] The fifth sub-step is to determine, for each operation guidance annotation result in the above operation guidance annotation result set, the tag type corresponding to the serialized tag included in the above operation guidance annotation result. Among them, the tag type corresponding to each serialized tag can be the corresponding guidance type, which can be preset. For example, the tag type corresponding to the guidance type "operation location" can be "LOC". The serialized tags corresponding to the tag type "LOC" can include, but are not limited to: B-LOC (Beginning of Location), I-LOC (Inside of Location).

[0086] The sixth sub-step is to perform aggregation processing on each guided word segmentation included in the above operation guidance annotation result set according to the tag type and serialized tag corresponding to each operation guidance annotation result in the above operation guidance annotation result set, to obtain each guidance parsing information. Among them, each guidance parsing information corresponds to a tag type, and the above each guidance parsing information includes at least one of the following: operation location, operation method, operation time, operation height. For example, when the operation object is a fan and the operation event is closing, the corresponding guidance types can include, but are not limited to: operation location, operation method. The corresponding each guidance parsing information can include, but is not limited to: on the right side of the fan, press to close. In practice, the above execution entity can aggregate each guided word segmentation whose corresponding serialized tag belongs to the same tag type in sequence according to the order in the operation guidance content into a guidance parsing information, to obtain each guidance parsing information.

[0087] The seventh sub-step is to control the above target robot to perform a response operation according to the above each guidance parsing information, the above operation object information and the operation event information. In practice, the above execution entity can control the above target robot to perform the operation event corresponding to the operation event information for the operation object corresponding to the above operation object information according to the above each guidance parsing information.

[0088] The above first step - second step and their related content are an inventive point of the embodiments of the present disclosure, which solve the second technical problem mentioned in the background art, that is, "when performing task planning, the prior knowledge is not well utilized, resulting in a long time consumption and poor accuracy of the planning task, leading to waste of computing resources and poor user experience". The factors that lead to waste of computing resources and poor user experience are often as follows: when performing task planning, the prior knowledge is not well utilized, resulting in a long time consumption and poor accuracy of the planning task. If the above factors are solved, the effect of reducing waste of computing resources and improving user experience can be achieved. To achieve this effect, when there is an operation guidance file corresponding to the operation object in the present disclosure, specific operation guidance content can be matched according to the operation object information and operation event information. Then, the operation guidance content can be parsed, and the parsing result can be used for the operation guidance when the robot executes the operation event for the operation object. Thus, it is no longer necessary to plan detailed operation tasks for the operation event of the operation object, and the parsing result can be directly used to guide the operation process, thereby saving computing resources. Also, because the operation guidance is determined based on the operation guidance file, the accuracy of the operation can be improved, and thus the user experience can be enhanced.

[0089] Optionally, for each operation information in the above operation information sequence, the above execution entity performs the following steps:

[0090] First step, extract keywords from the operation event information included in the above operation information as event keywords. In practice, the above execution entity can extract keywords from the operation event information included in the above operation information as event keywords through a keyword extraction algorithm.

[0091] Second step, extract keywords from the operation object information included in the above operation information as object keywords. In practice, the above execution entity can extract keywords from the operation object information included in the above operation information as object keywords through a keyword extraction algorithm.

[0092] Third step, combine the above event keywords and the above object keywords into key information.

[0093] Fourth step, for each operation keyword in the preset operation keyword set, generate the similarity between the above key information and the above operation keyword as similarity information. Each operation keyword in the above operation keyword set corresponds to a movement label. The movement label indicates whether movement is required. Here, the similarity can be cosine similarity.

[0094] Fifth step, select, from the above operation keyword set, the operation keyword whose similarity information corresponding to the above key information meets the preset similarity condition as the target operation keyword. The above preset similarity condition can be that the similarity information corresponding to the operation keyword and the above key information is the largest.

[0095] Step 6: In response to determining that the movement label corresponding to the above target operation keyword indicates no movement, determine whether the operation object corresponding to the above operation information meets the automatic control network condition. Among them, the above automatic control network condition may be that the operation device corresponding to the above operation object and the above execution entity are in the same network. For example, when the operation object is a lamp, the corresponding operation device may be the switch of the lamp.

[0096] Step 7: In response to determining that the operation object corresponding to the above operation information does not meet the automatic control network condition, determine the movement label indicating the need for movement as the operation type corresponding to the above operation information. Thus, the operation type corresponding to the operation information can be automatically determined.

[0097] Optionally, after selecting, from the above operation keyword set, the operation keyword whose similar information corresponding to the above key information meets the preset similarity condition as the target operation keyword, the above execution entity may further perform the following steps:

[0098] Step 1: In response to determining that the movement label corresponding to the above target operation keyword indicates the need for movement, determine the movement label indicating the need for movement as the operation type corresponding to the above operation information.

[0099] Step 2: In response to determining that the operation object corresponding to the above operation information meets the above automatic control network condition, determine the movement label indicating no movement as the operation type corresponding to the above operation information.

[0100] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the robot control method of some embodiments of the present disclosure, the moving path length of the robot is shortened, and the power resource of the robot is saved. Specifically, the reason for the waste of the robot's power resource is that when the positions of multiple operations that the robot needs to execute are close but the execution order is not adjacent, the robot needs to move back and forth in the scene when executing the operations in order, resulting in a longer moving path length and causing waste of the robot's power resource. Based on this, in the robot control method of some embodiments of the present disclosure, first, user instruction information corresponding to the target robot is received from the user. Thus, the instruction issued by the user can be sensed. Then, the above user instruction information is parsed and processed to obtain instruction parsing information, where the above instruction parsing information includes an operation information sequence, and each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type. Thus, the instruction issued by the user can be parsed into the dimensions of the operation object and the operation event. Then, in response to determining that the operation configuration information corresponding to the user for the above target robot satisfies the preset shortest distance first condition, and there is an operation information in the above operation information sequence whose corresponding operation type satisfies the preset movement condition, each operation information in the above operation information sequence whose corresponding operation type satisfies the above preset movement condition is determined as the first operation information sequence. Thus, when the user configures the shortest distance strategy and the robot needs to move when executing the user instruction, the operation information that needs to be moved can be extracted for subsequent analysis. Secondly, for each first operation information in the above first operation information sequence, the position information corresponding to the operation object information included in the above first operation information is determined. Thus, the position of the operation object that needs to be operated can be located. Next, according to the position information corresponding to each first operation information in the above first operation information sequence, the above first operation information sequence is clustered to obtain each first operation information group. Thus, the extracted operation information can be clustered into multiple groups of data according to the position. Then, according to the above operation information sequence and the above each first operation information group, a rearranged operation information sequence is generated. Thus, under the constraint of the clustering result, each operation information in the operation information sequence is rearranged so that the operation information corresponding to the operation objects with closer distances is arranged in adjacent positions. Finally, for each operation information in the above rearranged operation information sequence, according to the operation object information and operation event information included in the above operation information, the above target robot is controlled to execute the corresponding operation. Thus, the robot can be controlled to execute the operation in response to the user instruction according to the rearranged operation information sequence. Also, because the rearranged operation information sequence is sorted according to the distance characteristics of the operation objects, the operation information of the operation objects with closer distances is close in the sequence, which can enable the robot to reach the positions of the nearby operation objects by moving a shorter distance without moving back and forth in the scene, thereby shortening the moving path length and saving the power resource of the robot.

[0101] Further reference is made to Figure 2 which, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a robot control device, and these device embodiments correspond to Figure 1 the method embodiments shown, and the device can be specifically applied to various electronic devices.

[0102] As shown in Figure 2 some embodiments of the robot control device 200 include: a receiving unit 201, an analyzing unit 202, a first determining unit 203, a second determining unit 204, a clustering unit 205, a generating unit 206, and a control unit 207. Among them, the receiving unit 201 is configured to receive user instruction information of a user corresponding to a target robot; the analyzing unit 202 is configured to perform parsing processing on the above user instruction information to obtain instruction parsing information, where the above instruction parsing information includes an operation information sequence, and each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type; the first determining unit 203 is configured to, in response to determining that the operation configuration information of the user corresponding to the target robot satisfies a preset shortest distance first condition, and there is an operation information in the operation information sequence whose corresponding operation type satisfies a preset movement condition, determine each operation information in the operation information sequence whose corresponding operation type satisfies the preset movement condition as a first operation information sequence; the second determining unit 204 is configured to, for each first operation information in the first operation information sequence, determine the position information corresponding to the operation object information included in the first operation information; the clustering unit 205 is configured to perform clustering processing on the first operation information sequence according to the position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group; the generating unit 206 is configured to generate a rearranged operation information sequence according to the operation information sequence and each first operation information group; the control unit 207 is configured to, for each operation information in the rearranged operation information sequence, control the target robot to perform a response operation according to the operation object information and operation event information included in the operation information.

[0103] It can be understood that the various units described in the device 200 correspond to the respective steps in the method described with reference to Figure 1 Accordingly, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units included therein, and will not be repeated here.

[0104] Next, reference is made to Figure 3 which shows a schematic structural diagram of an electronic device 300 (such as a terminal device or a robot) suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0105] As Figure 3 shown, the electronic device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0106] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 the electronic device 300 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included. Figure 3 Each block shown in

[0107] Specifically, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.

[0108] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0109] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0110] The above computer-readable medium may be included in the above electronic device; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: receive user instruction information corresponding to a target robot; parse and process the above user instruction information to obtain instruction parsing information, where the above instruction parsing information includes an operation information sequence, and each operation information in the above operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type; in response to determining that the operation configuration information corresponding to the user for the above target robot satisfies a preset shortest distance first condition, and there is operation information in the above operation information sequence whose corresponding operation type satisfies a preset movement condition, determine each operation information in the above operation information sequence whose corresponding operation type satisfies the above preset movement condition as a first operation information sequence; for each first operation information in the above first operation information sequence, determine the position information corresponding to the operation object information included in the above first operation information; perform clustering processing on the above first operation information sequence according to the position information corresponding to each first operation information in the above first operation information sequence to obtain each first operation information group; generate a rearranged operation information sequence according to the above operation information sequence and the above each first operation information group; for each operation information in the above rearranged operation information sequence, control the above target robot to perform a response operation according to the operation object information and operation event information included in the above operation information.

[0111] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0113] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a receiving unit, a parsing unit, a first determining unit, a second determining unit, a clustering unit, a generating unit, and a control unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the receiving unit can also be described as "the unit that receives the user instruction information corresponding to the target robot of the user".

[0114] The functions described above can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0115] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A robot control method, comprising: Receiving user instruction information corresponding to a target robot from a user, wherein the user instruction information includes at least one instruction sentence, and the at least one instruction sentence has an arrangement order; The user instruction information is parsed to obtain instruction parsed information, wherein the instruction parsed information includes an operation information sequence, each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type. The user instruction information is parsed to obtain instruction parsed information, including: For each instruction sentence included in the user instruction information, the following steps are performed: Input the instruction sentence into the input layer of a pre-trained operation parsing model to obtain a word segmentation embedding vector set, wherein the operation parsing model includes an input layer, an encoding layer, and an output layer, the input layer includes a word segmentation layer and an embedding layer, the output layer includes a sequence tagging layer, the word segmentation layer is used to perform word segmentation processing on the input user instruction information to obtain a word segmentation set, and the embedding layer is used to generate a word segmentation embedding vector for each word in the word segmentation set; Inputting the word embedding vector set into the encoding layer to obtain a hidden state vector set, wherein the encoding layer includes various encoder layers, each encoder layer includes a multi-head attention mechanism and a feedforward neural network; Inputting the hidden state vector set to the sequence labeling layer included in the output layer to obtain a labeling result set, wherein each labeling result in the labeling result set includes a segmentation, a serialization label corresponding to the segmentation, and a probability corresponding to the serialization label; Combining the annotation results in the annotation result set whose corresponding serialized tags belong to the operation object type into an operation object annotation result set; Determine each word segment included in the operation object annotation result set as a first word segment set; According to each serialization tag corresponding to the first word segmentation set, the first word segmentation set is serialized to obtain a first word segmentation sequence; Combining the first participles in the first participle sequence in sequence into operation object information; Combining the annotation results corresponding to the serialized tags in the annotation result set that belong to the operation event type into an operation event annotation result set; Determine each segmentation included in the operation event annotation result set as a second segmentation set; According to each serialization tag corresponding to the second word segmentation set, the second word segmentation set is serialized to obtain a second word segmentation sequence; Combining the second participles in the second participle sequence in sequence into operation event information; Determining the operation object information and the operation event information as operation information; Combining the obtained individual operation information into an operation information sequence; Determine the operation information sequence as instruction parsing information; In response to determining that the operation configuration information of the user corresponding to the target robot satisfies a preset shortest distance priority condition, and there is operation information of a corresponding operation type satisfying a preset movement condition in the operation information sequence, each operation information of a corresponding operation type satisfying the preset movement condition in the operation information sequence is determined as a first operation information sequence; For each first operation information in the first operation information sequence, determining position information of the operation object information included in the first operation information; performing clustering processing on the first operation information sequence according to position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group; Generating a rearranged operation information sequence according to the operation information sequence and the respective first operation information groups, wherein generating a rearranged operation information sequence according to the operation information sequence and the respective first operation information groups comprises: For each of the first operation information groups, perform the following steps: Determine arrangement order information of each first operation information in the first operation information group in the operation information sequence; Sort each first operation information in the first operation information group according to each arrangement sequence information corresponding to each first operation information in the first operation information group to obtain a second operation information sequence; Determine the first second operation information in the second operation information sequence as the first second operation information; For each determined first second operation information, based on the operation information sequence, perform the following steps: Determine the second operation information sequence corresponding to the first second operation information as the third operation information sequence; determining each piece of third operation information arranged after the second operation information in the third operation information sequence as a fourth operation information sequence; Moving each operation information corresponding to the fourth operation information sequence in the operation information sequence to the first second operation information in the operation information sequence, so as to update the operation information sequence; determining the updated operation information sequence as a rearrangement operation information sequence; For each piece of operation information in the rearranged operation information sequence, the target robot is controlled to perform a response operation according to the operation object information and the operation event information included in the operation information.

2. The method according to claim 1, wherein: The method further comprises: For each operation information in the operation information sequence, perform the following steps: extracting keywords from the operation event information included in the operation information as event keywords; extracting keywords from the operation object information included in the operation information as object keywords; combining the event keyword and the object keyword into key information; For each operation keyword in the preset operation keyword set, a similarity between the key information and the operation keyword is generated as similarity information, wherein each operation keyword in the operation keyword set corresponds to a movement tag, and the movement tag indicates whether movement is required; Selecting, from the operation keyword set, an operation keyword whose similar information corresponding to the key information meets a preset similarity condition as a target operation keyword; In response to determining that the mobile tag corresponding to the target operation keyword indicates that no movement is required, determining whether the operation object corresponding to the operation information meets the automatic control network condition; In response to determining that the operation object corresponding to the operation information does not meet the automatic control network condition, a mobile tag indicating the need to move is determined as an operation type corresponding to the operation information.

3. The method according to claim 2, wherein: After selecting, from the operation keyword set, an operation keyword whose similar information corresponding to the key information meets a preset similarity condition as a target operation keyword, the method further includes: In response to determining that the mobile tag corresponding to the target operation keyword indicates that movement is required, determining the mobile tag indicating that movement is required as an operation type corresponding to the operation information; In response to determining that the operation object corresponding to the operation information meets the automatic control network condition, a mobile tag indicating that it does not need to move is determined as an operation type corresponding to the operation information.

4. The method according to claim 1, wherein: The determining of the location information of the operation object information included in the first operation information includes: Determining a three-dimensional scene model corresponding to the current position of the target robot; Selecting a scene element object that matches the operation object information from the three-dimensional scene model; Determine the three-dimensional coordinates of the scene element object in the three-dimensional coordinate system corresponding to the three-dimensional scene model; The three-dimensional coordinates are determined as position information corresponding to the operation object information included in the first operation information.

5. The method according to claim 1, wherein: The clustering process is performed on the first operation information sequence according to the position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group, including: For every two pieces of first operation information in the first operation information sequence, perform the following steps: respectively determining the position information corresponding to the two pieces of first operation information to obtain two pieces of position information; Determine the horizontal coordinates corresponding to the two pieces of position information as the first horizontal coordinate and the second horizontal coordinate respectively; determining a distance between the first horizontal coordinate and the second horizontal coordinate as an object distance; Based on the determined distances between the objects, the first operation information in the first operation information sequence is clustered to obtain first operation information groups.

6. A robot control device, comprising: A receiving unit is configured to receive user instruction information of a user corresponding to a target robot, wherein the user instruction information includes at least one instruction sentence, and the at least one instruction sentence has an arrangement order; The parsing unit is configured to parse the user instruction information to obtain instruction parsing information, wherein the instruction parsing information includes an operation information sequence, each operation information in the operation information sequence includes operation object information and operation event information, and the operation event information corresponds to an operation type. The parsing unit parses the user instruction information to obtain the instruction parsing information, including: for each instruction sentence included in the user instruction information, performing the following steps: inputting the instruction sentence into the input layer of a pre-trained operation parsing model to obtain a word segmentation embedding vector set, wherein the operation parsing model includes an input layer, an encoding layer and an output layer, wherein the input layer includes a word segmentation layer and an embedding layer, and the output layer includes a sequence annotation layer, wherein the word segmentation layer is used to perform word segmentation processing on the input user instruction information to obtain a word segmentation set, and the embedding layer is used to generate a word segmentation embedding vector for each word in the word segmentation set; the word segmentation embedding vector set is input into the encoding layer to obtain a hidden state vector set, wherein the encoding layer includes various encoder layers, and each encoder layer includes a multi-head attention mechanism and a feedforward neural network; the hidden state vector set is input into the sequence annotation layer included in the output layer to obtain a tagging result set, wherein the Each annotation result in the annotation result set includes a segmentation, a serialization tag corresponding to the segmentation, and a probability corresponding to the serialization tag; the individual annotation results whose corresponding serialization tags in the annotation result set belong to the operation object type are combined into an operation object annotation result set; the individual segmentations included in the operation object annotation result set are determined as a first segmentation set; according to the individual serialization tags corresponding to the first segmentation set, the first segmentation set is serialized to obtain a first segmentation sequence; the individual first segmentations in the first segmentation sequence are sequentially combined into operation object information; the individual annotation results whose corresponding serialization tags in the annotation result set belong to the operation event type are combined into an operation event annotation result set; the individual segmentations included in the operation event annotation result set are determined as a second segmentation set; according to the individual serialization tags corresponding to the second segmentation set, the second segmentation set is serialized to obtain a second segmentation sequence; the individual second segmentations in the second segmentation sequence are sequentially combined into operation event information; the operation object information and the operation event information are determined as operation information; the individual operation information obtained are combined into an operation information sequence; the operation information sequence is determined as instruction parsing information; A first determining unit is configured to, in response to determining that the operation configuration information of the user corresponding to the target robot satisfies a preset shortest distance priority condition, and there is operation information of a corresponding operation type satisfying a preset movement condition in the operation information sequence, determine each operation information of a corresponding operation type satisfying the preset movement condition in the operation information sequence as a first operation information sequence; a second determining unit, configured to determine, for each first operation information in the first operation information sequence, position information of the operation object information included in the first operation information; a clustering unit configured to perform clustering processing on the first operation information sequence according to position information corresponding to each first operation information in the first operation information sequence to obtain each first operation information group; A generating unit is configured to generate a rearranged operation information sequence according to the operation information sequence and the respective first operation information groups, wherein the generating of the rearranged operation information sequence according to the operation information sequence and the respective first operation information groups comprises: for each first operation information group in the respective first operation information groups, performing the following steps: determining the arrangement order information of each first operation information in the first operation information group in the operation information sequence; sorting the respective first operation information in the first operation information group according to the respective arrangement order information corresponding to the respective first operation information in the first operation information group to obtain a second operation information sequence; sorting the The first second operation information in the second operation information sequence is determined as the first second operation information; for each determined first second operation information, based on the operation information sequence, the following steps are performed: the second operation information sequence corresponding to the first second operation information is determined as the third operation information sequence; each third operation information arranged after the second operation information in the third operation information sequence is determined as the fourth operation information sequence; each operation information corresponding to the fourth operation information sequence in the operation information sequence is moved to after the first second operation information in the operation information sequence, so as to update the operation information sequence; and the updated operation information sequence is determined as the rearranged operation information sequence; The control unit is configured to control the target robot to perform a response operation for each operation information in the rearranged operation information sequence according to the operation object information and the operation event information included in the operation information.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images