Robot optimization learning method and device for demonstration learning, equipment and medium

Abstract

The embodiment of the invention discloses a robot optimization learning method and device for demonstration learning, equipment and a medium. A specific embodiment of the method comprises the following steps: generating a motion image sequence, a point cloud data sequence and a multi-modal feature information set corresponding to voice; performing feature semantic alignment processing on the multi-modal feature information set to generate an aligned multi-modal feature information set; inputting the aligned multi-modal feature information set into a multi-modal demonstration learning model; recording the action image sequence, the point cloud data sequence, the voice, the aligned multi-modal feature information set and the multi-modal demonstration learning result into a memory of the intelligent robot, and controlling the intelligent robot to execute a target demonstration task; and in response to determining that the execution of the target demonstration task is completed, optimizing the autonomous learning model of the intelligent robot according to the collected task data corresponding to the target demonstration task. According to the embodiment, the autonomous learning and execution capability of the robot system in complex tasks is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computers, and in particular to a robot optimization learning method, device, equipment and medium for demonstration learning. Background Art

[0002] Existing robot systems mainly rely on programming and preset actions to complete tasks, and are difficult to adapt to dynamic and complex environmental changes. Although machine learning and artificial intelligence have made some progress, they often require a large amount of data and a complex training process. However, existing robot systems require programming knowledge to set the robot's actions, which is difficult for ordinary users to operate; robots cannot respond flexibly to changes and require human intervention and reprogramming; the learning and training process is cumbersome and time-consuming, making them unsuitable for home applications.

[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure propose a robot optimization learning method, device, electronic device and computer-readable medium for demonstration learning to solve one or more of the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a robot optimization learning method for demonstration learning, the method comprising: in response to receiving a user demonstration instruction, controlling the depth camera of the above-mentioned intelligent robot to collect the action image sequence and point cloud data sequence of the target user, and controlling the voice acquisition device of the above-mentioned intelligent robot to collect the voice of the above-mentioned target user in real time, wherein the action image in the above-mentioned action image sequence corresponds to the point cloud data in the above-mentioned point cloud data sequence; generating a multimodal feature information set corresponding to the above-mentioned action image sequence, the point cloud data sequence and the above-mentioned voice, wherein the above-mentioned multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and voice modal feature information corresponding to the voice. information; performing feature semantic alignment processing on the multimodal feature information set to generate an aligned multimodal feature information set; inputting the aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result; recording the action image sequence, the point cloud data sequence, the speech, the aligned multimodal feature information set and the multimodal demonstration learning result into the memory of the intelligent robot, and controlling the intelligent robot to perform a target demonstration task, wherein the target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result; in response to determining that the target demonstration task is completed, optimizing the autonomous learning model of the intelligent robot according to the collected task data corresponding to the target demonstration task.

[0007] In a second aspect, some embodiments of the present disclosure provide a robot optimization learning device for demonstration learning, the device comprising: a control unit, configured to control the depth camera of the above-mentioned intelligent robot to collect the action image sequence and point cloud data sequence of the target user in response to receiving a user demonstration instruction, and control the voice acquisition device of the above-mentioned intelligent robot to collect the voice of the above-mentioned target user in real time, wherein the action image in the above-mentioned action image sequence corresponds to the point cloud data in the above-mentioned point cloud data sequence; a generation unit, configured to generate a multimodal feature information set corresponding to the above-mentioned action image sequence, the point cloud data sequence and the above-mentioned voice, wherein the above-mentioned multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and voice modal feature information corresponding to the voice; an alignment unit, configured to The multimodal feature information set is subjected to feature semantic alignment processing to generate an aligned multimodal feature information set; an input unit is configured to input the aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result; a recording unit is configured to record the action image sequence, the point cloud data sequence, the voice, the aligned multimodal feature information set and the multimodal demonstration learning result into the memory of the intelligent robot, and control the intelligent robot to perform a target demonstration task, wherein the target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result; an optimization unit is configured to optimize the autonomous learning model of the intelligent robot in response to determining that the target demonstration task is completed according to the collected task data corresponding to the target demonstration task.

[0008] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0009] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0010] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the robot optimization learning method for demonstration learning of some embodiments of the present disclosure, the autonomous learning and execution ability of the robot system in complex tasks is improved. It can not only efficiently imitate and execute the user's operation, but also self-optimize in the process of continuous feedback and improvement, and adapt to various complex and dynamic task environments. In the future, the system is expected to be widely used in many fields, such as intelligent manufacturing, home services, medical care, etc., to provide strong technical support for the automation and intelligent development of various industries. First, in response to receiving the user's demonstration instruction, the depth camera of the above-mentioned intelligent robot is controlled to collect the action image sequence and point cloud data sequence of the target user, and the voice acquisition device of the above-mentioned intelligent robot is controlled to collect the voice of the above-mentioned target user in real time, wherein the action image in the above-mentioned action image sequence corresponds to the point cloud data in the above-mentioned point cloud data sequence. Thus, according to the specific operation of the user, the dynamic changes of the environment during the operation and the user's voice instructions are also covered to ensure the comprehensiveness and accuracy of the record. Secondly, generate a multimodal feature information set corresponding to the above-mentioned action image sequence, point cloud data sequence and the above-mentioned voice, wherein the above-mentioned multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and voice modal feature information corresponding to the voice. Then, the above-mentioned multimodal feature information set is subjected to feature semantic alignment processing to generate an aligned multimodal feature information set. Afterwards, the above-mentioned aligned multimodal feature information set is input into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result. Thus, the commonalities and laws in different demonstrations can be identified and generalized through a multimodal large model algorithm, thereby deeply understanding the essence of the task. Then, the above-mentioned action image sequence, point cloud data sequence, the above-mentioned voice, the above-mentioned aligned multimodal feature information set and the above-mentioned multimodal demonstration learning result are recorded in the memory of the above-mentioned intelligent robot, and the above-mentioned intelligent robot is controlled to perform a target demonstration task, wherein the above-mentioned target demonstration task is a demonstration task corresponding to the above-mentioned multimodal demonstration learning result. Finally, in response to determining that the above target demonstration task is completed, the autonomous learning model of the above intelligent robot is optimized according to the collected task data corresponding to the above target demonstration task. Thus, dynamic adjustment and optimization can be performed according to real-time feedback to ensure the smooth completion and efficient execution of the task. This improves the autonomous learning and execution capabilities of the robot system in complex tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar figure labels represent the same or similar elements. It should be understood that the drawings are schematic and that the components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a flow chart of some embodiments of a robot optimization learning method for demonstration learning according to the present disclosure;

[0013] Figure 2 The example shows a schematic diagram of the steps for the intelligent robot to optimize the driving route;

[0014] Figure 3 is a flow chart of some embodiments of a robot optimized learning apparatus for demonstration learning according to the present disclosure;

[0015] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Figure 11 is a flowchart of some embodiments of the robot optimization learning method for demonstration learning according to the present disclosure. A process 100 of some embodiments of the robot optimization learning method for demonstration learning according to the present disclosure is shown. The robot optimization learning method for demonstration learning is applied to an intelligent robot and includes the following steps:

[0023] Step 101, in response to receiving a user demonstration instruction, controlling the depth camera of the above-mentioned intelligent robot to collect the action image sequence and point cloud data sequence of the target user, and controlling the voice acquisition device of the above-mentioned intelligent robot to collect the voice of the above-mentioned target user in real time.

[0024] In some embodiments, the execution subject (e.g., a computing device) of the robot optimization learning method for demonstration learning can control the depth camera of the above-mentioned intelligent robot to collect the action image sequence and point cloud data sequence of the target user in response to receiving the user demonstration instruction, and control the voice acquisition device of the above-mentioned intelligent robot to collect the voice of the above-mentioned target user in real time. Among them, the action image in the above-mentioned action image sequence corresponds to the point cloud data in the above-mentioned point cloud data sequence. The user demonstration instruction may refer to an instruction for the user to demonstrate the action, and the intelligent robot records the user's action steps, operation details and environmental changes. The intelligent robot may be an intelligent robot with a mechanical back that can imitate user actions. The target user may refer to a demonstration user who demonstrates the operation task several times through manual operation, voice instructions or other forms. For example, the voice acquisition device may be a microphone.

[0025] The intelligent robot uses advanced sensing technology and uses a depth camera to record the RGB and depth point cloud information of the operation, and also records the relevant voice information. These data not only include the user's specific operations, but also cover the dynamic changes of the environment during the operation and the user's voice commands, ensuring the comprehensiveness and accuracy of the records.

[0026] Step 102: Generate a multimodal feature information set corresponding to the action image sequence, point cloud data sequence and the speech.

[0027] In some embodiments, the execution subject may generate a multimodal feature information set corresponding to the action image sequence, the point cloud data sequence and the speech. The multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and speech modal feature information corresponding to the speech. The multimodal feature information may characterize the feature semantic content of the object information under the corresponding modality. The multimodal feature information may be information in vector form.

[0028] As an example, the above-mentioned execution subject can use the corresponding multimodal feature extraction model under multimodality to extract feature semantic content from the object information (action image sequence, point cloud data sequence and voice) under each modality to generate multimodal feature information and obtain a multimodal feature information set. The modal feature extraction model in the multimodal feature extraction model has a one-to-one correspondence with the modality in the multimodality. The modal feature extraction model can be a neural network model that extracts the feature semantic content under the corresponding modality. The modal feature extraction model can have different corresponding model network structures depending on the corresponding modality. Multimodality can represent action image modality, point cloud data modality and voice modality. For example, the multimodal feature extraction model can include but is not limited to: image feature extraction model, point cloud feature extraction model, voice feature extraction model.

[0029] Step 103: perform feature semantic alignment processing on the multimodal feature information set to generate an aligned multimodal feature information set.

[0030] In some embodiments, the execution subject may perform feature semantic alignment processing on the multimodal feature information set to generate an aligned multimodal feature information set. The multimodal feature information set may be subjected to feature semantic alignment processing using an LLaVA (Large Language and Vision Assistant) model to generate an aligned multimodal feature information set.

[0031] Step 104: input the above-mentioned aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result.

[0032] In some embodiments, the above-mentioned execution subject may input the above-mentioned aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result. The multimodal demonstration learning model may refer to a multi-model self-learning model, which may include: a type A multimodal model, a generative adversarial network (GAN), a variational autoencoder (VAE), a flow-based generative model, etc. The type A multimodal model is a deep fusion model based on standard cross-attention, which uses a standard Transformer model and adds a standard cross-attention layer to the internal layer of the model to achieve deep fusion of input multimodal information. This model requires a large amount of multimodal training data and has high computing resource requirements, but has the advantage of fine control of multimodal information. Typical type A models include Flamingo and OpenFlamingo, which can process image and text data. The generative adversarial network (GAN) performs adversarial training through a generative model and a discriminative model. The generative model attempts to deceive the discriminative model so that it cannot distinguish between real data and false data. The variational autoencoder (VAE) is a generative model that maps the input to the probability distribution of the hidden space instead of a fixed encoding. The flow-based generative model gradually adds noise from the real data distribution by defining forward and reverse processes, and then learns the process of recovering the real data from the noise. Multimodal learning is a method of learning using data from different senses or interaction modes, which may include text, images, audio, video, etc. Multimodal learning trains models by fusing multiple data modalities, thereby improving the perception and understanding capabilities of the model and achieving cross-modal information interaction and fusion. Typical modal representation methods include the pre-trained model BERT for text, the convolutional neural network (CNN) for images, and the deep neural network (DNN) for sound. The multimodal demonstration learning result can refer to the result of self-learning for aligning multimodal feature information sets.

[0033] The multimodal demonstration learning model can be trained by the following steps:

[0034] The first step is to obtain a multimodal feature information training sample set, wherein the multimodal feature information training samples in the multimodal feature information training sample set include: sample action image feature information, sample point cloud data feature information, sample speech feature information and sample labels.

[0035] The second step is to determine the initial multimodal demonstration learning model. Among them, the above-mentioned initial multimodal demonstration learning model includes: an initial action image demonstration learning network, an initial point cloud data demonstration learning network, and an initial voice demonstration learning network. The above-mentioned initial voice demonstration learning network includes: an initial voice feature extraction network, an initial multi-level feature autoencoding network, and an initial voice recognition network. The initial action image demonstration learning network may refer to a self-learning network model for self-learning action features in action images. The initial point cloud data demonstration learning network may refer to a self-learning network model for self-learning action features in point cloud data. The initial voice demonstration learning network may refer to a self-learning network model for self-learning voice features in voice. The initial voice feature extraction network may be an untrained neural network for extracting voice features. For example, the initial voice feature extraction network may be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc. The above-mentioned initial multi-level feature autoencoder network includes: an initial encoding network (encoder) and an initial decoding network (decoder), the initial encoding network includes: an initial speech feature decoupling network (including an autoencoder network, an LSTM network) and an initial speech feature fusion network (VAE (Variational AutoEncoder, variational encoder), the initial speech feature decoupling network includes: an initial static speech feature decoupling network, an initial dynamic speech feature decoupling network, and the initial speech feature fusion network includes: an initial fusion network and an initial reparameterization network. The initial static speech feature decoupling network can be an Auto-Encoder (autoencoder network) model. The initial dynamic speech feature decoupling network can be an LSTM (Long Short-Term Memory, long short-term memory network) model. The initial fusion network can be a feature pyramid network (Feature Pyramid Network, FPN) or a self-attention mechanism (Self-Attention Mechanism) network. The initial reparameterization network can include: DyRep (Dynamic Re-parameterization) and ACNet (Asymmetric Convolution Net). The initial speech recognition network can include: a rule-based model, a statistics-based model, and a deep learning-based model.

[0036] The third step is to select a target multimodal feature information training sample from the multimodal feature information training sample set. A multimodal feature information training sample can be randomly selected from the multimodal feature information training sample set as the target multimodal feature information training sample.

[0037] The fourth step is to train the initial multimodal demonstration learning model based on the target multimodal feature information training samples to obtain a trained multimodal demonstration learning model.

[0038] The fourth step may include the following sub-steps:

[0039] In the first sub-step, the sample action image feature information included in the target multimodal feature information training sample is input into the above-mentioned initial action image demonstration learning network to obtain the initial action image demonstration learning result.

[0040] In the second sub-step, the sample point cloud data feature information included in the target multimodal feature information training sample is input into the above-mentioned initial point cloud data demonstration learning network to obtain the initial point cloud data demonstration learning result.

[0041] The third sub-step is to input the sample speech feature information included in the target multimodal feature information training sample into the above-mentioned initial speech feature extraction network to obtain initial speech feature extraction information.

[0042] In the fourth sub-step, the initial speech feature extraction information is input into the initial multi-level feature autoencoder network to obtain the initial multi-level speech feature information.

[0043] The fifth sub-step is to input the above-mentioned initial multi-level speech feature information into the initial speech recognition network to obtain initial speech recognition information.

[0044] The sixth sub-step is to combine the above-mentioned initial action image demonstration learning results, the above-mentioned initial point cloud data demonstration learning results and the above-mentioned initial speech recognition information into initial multimodal demonstration learning information.

[0045] The seventh sub-step is to determine the loss value between the initial multimodal demonstration learning information and the corresponding sample label. The loss value between the initial multimodal demonstration learning information and the corresponding sample label can be determined by a cross entropy loss function or a hinge loss function.

[0046] An eighth sub-step, in response to determining that the above loss value is less than or equal to a preset loss value, determines the initial multimodal demonstration learning model as a trained multimodal demonstration learning model.

[0047] Step 105, recording the above-mentioned action image sequence, point cloud data sequence, the above-mentioned voice, the above-mentioned aligned multimodal feature information set and the above-mentioned multimodal demonstration learning result into the memory of the above-mentioned intelligent robot, and controlling the above-mentioned intelligent robot to perform the target demonstration task.

[0048] In some embodiments, the execution subject may record the action image sequence, the point cloud data sequence, the voice, the aligned multimodal feature information set and the multimodal demonstration learning result into the memory of the intelligent robot, and control the intelligent robot to perform a target demonstration task. The target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result. The target demonstration task may be a task set to instruct the intelligent robot to perform action and voice feature learning based on the action image sequence and point cloud data sequence of the target user.

[0049] Step 106, in response to determining that the above-mentioned target demonstration task is completed, the autonomous learning model of the above-mentioned intelligent robot is optimized according to the collected task data corresponding to the above-mentioned target demonstration task.

[0050] In some embodiments, the execution subject may optimize the autonomous learning model of the intelligent robot in response to determining that the target demonstration task has been completed, according to the collected task data corresponding to the target demonstration task. Task data may refer to the task operation data generated by the intelligent robot in the process of executing the target demonstration task. The autonomous learning model may refer to a self-learning model set in the intelligent robot, or it may be a multimodal self-learning model. A model of the operation task (autonomous learning model) is established in the intelligent robot, including action sequences, conditions and expected results. It can also identify and generalize the commonalities and laws in different demonstrations through a multimodal large model algorithm, so as to deeply understand the essence of the task.

[0051] As an example, first, the difference between the standard task data and the task data that are pre-set to correspond to the above-mentioned target demonstration task can be determined. Secondly, the model parameters of the self-learning model in the intelligent robot can be adjusted by gradient descent, so that continuous learning and improvement can be made to continuously improve the autonomy and accuracy of the robot. For example, users can provide more sample data to the robot through additional demonstration operations to help it better understand the details of the task. The robot will integrate these new data with the existing model and gradually improve and optimize the operation strategy. In addition, the self-learning function of the robot enables it to continuously accumulate experience during use and improve execution efficiency and accuracy.

[0052] Optionally, obtain driving learning task requirement information of the intelligent robot corresponding to the target area.

[0053] In some embodiments, the execution subject may obtain the driving learning task requirement information of the intelligent robot corresponding to the target area, wherein the driving learning task requirement information includes: a driving starting point, a driving end point, and environmental information, and the environmental information includes: at least one obstacle information. The at least one obstacle information may refer to the obstacle information between the driving starting point and the driving end point, and may include the obstacle position / obstacle volume. The driving starting point may be the starting point of the intelligent robot's driving. The driving end point may be the end point of the intelligent robot's driving.

[0054] Optionally, based on the driving learning task requirement information, an initial driving route between the driving starting point and the driving end point is generated.

[0055] In some embodiments, the execution subject may generate an initial driving route between the driving starting point and the driving end point according to the driving learning task requirement information. The initial driving route for the driving starting point and the driving end point may be generated by a target heuristic search algorithm. The initial driving route may be a driving path with the driving starting point as the starting point and the driving end point as the ending point. In practice, the target heuristic search algorithm may be an A* (AStar) algorithm.

[0056] Optionally, the node routes between the various driving nodes in the above-mentioned initial driving route are optimized to generate an optimized driving route.

[0057] In some embodiments, the execution entity may optimize the node routes between the various driving nodes in the initial driving route to generate an optimized driving route.

[0058] In practice, the execution entity may optimize the node routes between the various driving nodes in the initial driving route by following the steps below:

[0059] In the first step, the distance matrix and route matrix corresponding to the driving starting point and the driving end point are determined by using the interpolation method. The interpolation method can be the Floyd-Warshall algorithm. The distance matrix can represent the node distance between each driving node between the driving starting point and the driving end point. The route matrix can represent the route information between each driving node between the driving starting point and the driving end point. Each driving node can be a pre-set landmark route node, or can be a route node set according to the driving time or driving distance.

[0060] In the second step, based on the distance matrix and the route matrix, node route optimization is performed on the node routes between the various driving nodes in the initial driving route to generate an optimized driving route.

[0061] The second step may include the following sub-steps:

[0062] The first sub-step is to determine the driving node sequence corresponding to the initial driving route. The driving node sequence may include: a driving starting point, a driving end point, and various driving nodes. The various driving nodes in the driving node sequence are arranged in a driving order.

[0063] The second sub-step is to divide each travel node in the travel node sequence to generate a travel node group sequence. For example, every two adjacent travel nodes in the travel node sequence are determined as a travel node sub-group to obtain a travel node group sequence.

[0064] The third sub-step is to perform the following processing steps for each travel node group in the travel node group sequence:

[0065] 1. According to the above distance matrix and the above route matrix, determine the first node distance and the first node route corresponding to the above travel node group. The first node distance can represent the node distance between each travel node in the travel node group. The first node route can represent the node connection route between each travel node in the travel node group. Each travel node in the travel node group is adjacent. The above execution entity can filter out the node distance corresponding to the travel node group from the distance matrix as the first node distance, and filter out the node route corresponding to the travel node group from the route matrix as the first node route.

[0066] 2. Determine a second node route for the driving node group in the initial driving route, and determine a second node distance corresponding to the second node route. The second node distance may represent the node distance between each driving node in the initial driving route. The second node route may represent the node connection route between each driving node in the initial driving route. The node distance corresponding to the driving node group may be screened out from the distance matrix as the second node distance, and the node route corresponding to the driving node group may be screened out from the route matrix as the second node route.

[0067] 3. In response to determining that the first node distance is less than or equal to the second node distance, determining the first node route as an alternative node route.

[0068] 4. In response to determining that the first node distance is greater than the second node distance, determining the second node route as an alternative node route.

[0069] The fourth sub-step is to combine the obtained candidate node routes into an optimized driving route.

[0070] Further references Figure 2, a schematic diagram of the step scenario of the intelligent robot performing driving route optimization is illustrated, including: 201 represents the intelligent robot, 202 represents the driving starting point A, 203 represents the driving end point B, 204 represents the initial driving route (A→C→D→B) after the driving starting point A and the driving end point B are searched by the algorithm; 205 represents the route after the initial driving route is optimized (A→E→D→B).

[0071] Optionally, based on the above-mentioned optimized driving route, the above-mentioned intelligent robot is controlled to perform a driving learning task corresponding to the target area.

[0072] In some embodiments, the execution subject may control the intelligent robot to perform the driving learning task corresponding to the target area according to the optimized driving route. For example, a driving learning instruction may be sent to the intelligent robot so that the intelligent robot automatically drives in the target area according to the optimized driving route to adapt to the environment of the target area.

[0073] This allows the robot to adjust its motion path when faced with unexpected obstacles or adjust its perception strategy based on changes in ambient lighting conditions. This adaptive capability significantly enhances the robot’s operational performance in complex environments.

[0074] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a robot optimization learning device for demonstration learning. These embodiments of the robot optimization learning device for demonstration learning are similar to Figure 1 Corresponding to the method embodiments shown, the robot optimization learning device for demonstration learning can be specifically applied to various electronic devices.

[0075] like Figure 3As shown, some embodiments of the robot optimization learning device 300 for demonstration learning include: a control unit 301, a generation unit 302, an alignment unit 303, an input unit 304, a recording unit 305 and an optimization unit 306. The control unit 301 is configured to control the depth camera of the intelligent robot to collect the target user's action image sequence and point cloud data sequence in response to receiving the user's demonstration instruction, and control the voice acquisition device of the intelligent robot to collect the target user's voice in real time, wherein the action image in the action image sequence corresponds to the point cloud data in the point cloud data sequence; the generation unit 302 is configured to generate a multimodal feature information set corresponding to the action image sequence, the point cloud data sequence and the voice, wherein the multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and voice modal feature information corresponding to the voice; the alignment unit 303 is configured to perform feature semantic alignment on the multimodal feature information set. Processing to generate an aligned multimodal feature information set; an input unit 304 is configured to input the above-mentioned aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result; a recording unit 305 is configured to record the above-mentioned action image sequence, the point cloud data sequence, the above-mentioned voice, the above-mentioned aligned multimodal feature information set and the above-mentioned multimodal demonstration learning result into the memory of the above-mentioned intelligent robot, and control the above-mentioned intelligent robot to perform a target demonstration task, wherein the above-mentioned target demonstration task is a demonstration task corresponding to the above-mentioned multimodal demonstration learning result; an optimization unit 306 is configured to optimize the autonomous learning model of the above-mentioned intelligent robot in response to determining that the above-mentioned target demonstration task is executed and completed according to the collected task data corresponding to the above-mentioned target demonstration task.

[0076] It is understandable that the units recorded in the robot optimization learning device 300 for demonstration learning are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the robot optimization learning device 300 for demonstration learning and the units contained therein, and will not be described in detail here.

[0077] Reference below Figure 4 , which shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 400 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0078] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0079] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0080] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0081] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0082] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0083] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to receiving a user demonstration instruction, controls the depth camera of the above-mentioned intelligent robot to collect the action image sequence and point cloud data sequence of the target user, and controls the voice acquisition device of the above-mentioned intelligent robot to collect the voice of the above-mentioned target user in real time, wherein the action image in the above-mentioned action image sequence corresponds to the point cloud data in the above-mentioned point cloud data sequence; generates a multimodal feature information set corresponding to the above-mentioned action image sequence, point cloud data sequence and the above-mentioned voice, wherein the above-mentioned multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and voice modal feature information corresponding to the voice. The method comprises the steps of: collecting the motion image sequence, the point cloud data sequence, the speech, the aligned multimodal feature information set and the multimodal demonstration learning result into the memory of the intelligent robot, and controlling the intelligent robot to perform a target demonstration task, wherein the target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result; in response to determining that the execution of the target demonstration task is completed, optimizing the autonomous learning model of the intelligent robot according to the collected task data corresponding to the target demonstration task.

[0084] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising: a control unit, a generation unit, an alignment unit, an input unit, a recording unit, and an optimization unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances, for example, the optimization unit may also be described as "a unit that optimizes the autonomous learning model of the above-mentioned intelligent robot according to the collected task data corresponding to the above-mentioned target demonstration task in response to determining that the above-mentioned target demonstration task has been executed".

[0087] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0088] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A robot optimization learning method for demonstration learning, applied to intelligent robots, comprising: In response to receiving a user demonstration instruction, controlling the depth camera of the intelligent robot to collect an action image sequence and a point cloud data sequence of a target user, and controlling the voice acquisition device of the intelligent robot to collect the voice of the target user in real time, wherein the action image in the action image sequence corresponds to the point cloud data in the point cloud data sequence; Generate a multimodal feature information set corresponding to the action image sequence, the point cloud data sequence and the speech, wherein the multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and speech modal feature information corresponding to the speech; Performing feature semantic alignment processing on the multimodal feature information set to generate an aligned multimodal feature information set; Inputting the aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result; Recording the action image sequence, the point cloud data sequence, the voice, the aligned multimodal feature information set and the multimodal demonstration learning result into the memory of the intelligent robot, and controlling the intelligent robot to perform a target demonstration task, wherein the target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result; In response to determining that the target demonstration task is completed, the autonomous learning model of the intelligent robot is optimized according to the collected task data corresponding to the target demonstration task.

2. The method according to claim 1, wherein: Before inputting the aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result, the method further includes: Acquire a multimodal feature information training sample set, wherein the multimodal feature information training samples in the multimodal feature information training sample set include: sample action image feature information, sample point cloud data feature information, sample voice feature information and sample labels; Determine an initial multimodal demonstration learning model, wherein the initial multimodal demonstration learning model includes: an initial action image demonstration learning network, an initial point cloud data demonstration learning network, and an initial voice demonstration learning network, and the initial voice demonstration learning network includes: an initial voice feature extraction network, an initial multi-level feature autoencoder network, and an initial voice recognition network; Selecting a target multimodal feature information training sample from the multimodal feature information training sample set; Based on the target multimodal feature information training samples, the initial multimodal demonstration learning model is trained to obtain a trained multimodal demonstration learning model.

3. The method according to claim 2, wherein: The method of training the initial multimodal demonstration learning model based on the target multimodal feature information training sample to obtain a trained multimodal demonstration learning model includes: Inputting sample action image feature information included in the target multimodal feature information training sample into the initial action image demonstration learning network to obtain an initial action image demonstration learning result; Inputting the sample point cloud data feature information included in the target multimodal feature information training sample into the initial point cloud data demonstration learning network to obtain the initial point cloud data demonstration learning result; Inputting the sample speech feature information included in the target multimodal feature information training sample into the initial speech feature extraction network to obtain initial speech feature extraction information; Inputting the initial speech feature extraction information into the initial multi-level feature autoencoder network to obtain the initial multi-level speech feature information; Inputting the initial multi-level speech feature information into an initial speech recognition network to obtain initial speech recognition information; Combining the initial action image demonstration learning result, the initial point cloud data demonstration learning result and the initial speech recognition information into initial multimodal demonstration learning information; Determining a loss value between the initial multimodal demonstration learning information and the corresponding sample label; In response to determining that the loss value is less than or equal to the preset loss value, the initial multimodal demonstration learning model is determined as the trained multimodal demonstration learning model.

4. The method according to claim 1, wherein: The method further comprises: Acquire driving learning task requirement information of the intelligent robot corresponding to the target area, wherein the driving learning task requirement information includes: a driving starting point, a driving end point, and environmental information, and the environmental information includes: at least one obstacle information; generating an initial driving route between the driving starting point and the driving end point according to the driving learning task requirement information; Optimizing the node routes between the driving nodes in the initial driving route to generate an optimized driving route; According to the optimized driving route, the intelligent robot is controlled to execute the driving learning task corresponding to the target area.

5. The method according to claim 4, wherein: The step of optimizing the node routes between the driving nodes in the initial driving route to generate an optimized driving route includes: Using the interpolation method, determine the distance matrix and route matrix corresponding to the driving start point and the driving end point; According to the distance matrix and the route matrix, node route optimization is performed on the node routes between the respective driving nodes in the initial driving route to generate an optimized driving route.

6. A robot optimization learning device for demonstration learning, applied to an intelligent robot, comprising: A control unit is configured to, in response to receiving a user demonstration instruction, control the depth camera of the intelligent robot to collect a target user's action image sequence and a point cloud data sequence, and control the voice acquisition device of the intelligent robot to collect the target user's voice in real time, wherein the action image in the action image sequence corresponds to the point cloud data in the point cloud data sequence; A generating unit is configured to generate a multimodal feature information set corresponding to the action image sequence, the point cloud data sequence and the speech, wherein the multimodal feature information set includes: action image modal feature information corresponding to the action image, point cloud data modal feature information corresponding to the point cloud data, and speech modal feature information corresponding to the speech; an alignment unit, configured to perform feature semantic alignment processing on the multimodal feature information set to generate an aligned multimodal feature information set; An input unit is configured to input the aligned multimodal feature information set into a pre-trained multimodal demonstration learning model to obtain a multimodal demonstration learning result; a recording unit configured to record the action image sequence, the point cloud data sequence, the voice, the aligned multimodal feature information set and the multimodal demonstration learning result into a memory of the intelligent robot, and control the intelligent robot to perform a target demonstration task, wherein the target demonstration task is a demonstration task corresponding to the multimodal demonstration learning result; The optimization unit is configured to optimize the autonomous learning model of the intelligent robot according to the collected task data corresponding to the target demonstration task in response to determining that the target demonstration task is completed.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images