167 results about "Imitation learning" patented technology

The invention provides a method for selecting reward functions in adversarial imitation learning. The method comprises the following steps: constructing a strategy network with a parameter theta, a discrimination network with a parameter w and at least two reward functions; obtaining teaching data under an expert strategy and storing the teaching data into an expert data buffer containing an expert track, wherein the input of the control strategy network is a state returned by the simulation environment, and the output is a decision action; enabling the discrimination network to update parameters by using the state action pair under the expert strategy and the state action pair of the strategy network; in the award calculation stage, judging that the input of the network is a state actionpair of the strategy network, and judging that the output value is an award value obtained by award function calculation; selecting a reward function of the current task according to the sizes of theperformance indexes of different reward functions; and storing the parameters of the strategy network corresponding to the selected reward function. And the intelligent agent learns under the guidanceof different reward functions, and then selects an optimal reward function in a specific task scene according to the performance evaluation index.

The invention discloses a high-speed railway train automatic driving system based on an artificial intelligence technology. The system is a novel high-speed railway train automatic driving system based on deep reinforcement learning and imitation learning. By constructing comprehensive train operation condition evaluation indexes and supplementing and enhancing acquisition of big data related to train operation, a train control strategy model which is based on deep reinforcement learning and imitation learning and can perform comprehensive perception and make correct and reasonable decisions is trained, and the comprehensive quality of train operation is further improved.

The invention discloses a mechanical arm autonomous grabbing method based on deep reinforcement learning and dynamic movement primitives. The mechanical arm autonomous grabbing method includes the following steps that firstly, a camera image assembly is installed, it is ensured that the recognition area is not shielded, grabbing target area images are preprocessed and sent to a deep reinforcementlearning intelligent agent as state information; secondly, a local strategy near-end optimization training model is established on the basis of the state and deep reinforcement leaning principle; thirdly, a new mixed movement primitive model is established by fusing the dynamic movement primitives and imitation learning; and fourthly, a mechanical arm is trained to autonomously grab objects on thebasis of the models. By means of the mechanical arm autonomous grabbing method, the problem that the mechanical arm joint movement based on traditional deep reinforcement learning is unsmooth can beeffectively solved, the learning problem of primitive parameters is converted into the reinforcement learning problem through combination with the dynamic movement primitive algorithm, and by means ofthe training method of deep reinforcement learning, the mechanical arm can complete the autonomous grabbing task.

The present invention relates to a method for controlling motions of a robot using evolutionary computation, the method including constructing a database by collecting patterns of human motion, evolving the database using a genetic operator that is based upon PCA and dynamics-based optimization, and creating motion of a robot in real time using the evolved database. According to the present invention, with the evolved database, a robot may learn human motions and control optimized motions in real time.

The invention relates to a Chinese character writing learning method based on track imitation, wherein the method belongs to the fields of artificial intelligent and robot learning. According to the method, imitation learning based on track matching is introduced into studying of a robot writing skill; demonstration data are coded through a Gaussian mixture model; track characteristics are extracted; data reconstruction is performed through Gaussian mixture regression; a generalized output of the track is obtained; and furthermore learning of a track-continuous Chinese character writing skill can be realized. An interference problem in the writing process is processed in a method of multiple demonstrations, and noise tolerance of the method is improved. According to the method, multitask expansion is based on the basic Gaussian mixture model; a complicated Chinese character is divided into a plurality of parts; track coding and reconstruction are performed on each divided part; and the method is applied for generating discrete tracks, thereby realizing writing of track-discontinuous Chinese characters. The Chinese character writing learning method realizes high Chinese character writing generalization effect.

The invention discloses a robot navigation method, comprising robot imitation learning and autonomous navigation steps. The robot imitation learning step comprises: recording each frame video image obtained by a camera and corresponding control commands; extracting a visual feature vector corresponding to each frame video image; employing the visual feature vectors and control commands as a training data set of a classifier; and calculating an optimal vector of the classifier. The robot autonomous navigation step comprises: extracting the visual feature vector of each frame video image; and the classifier calculating and outputting the control commands according to the visual feature vectors of the video images and the optimal vectors to realize autonomous navigation. The method does not need to establish complex environment models or robot motion control models; robot detection to environments mainly depends on visual feature information, and is suitable for indoor navigation and outdoor navigation; the method can be applied to mobile robots, humanoid robots, mechanical arms, unmanned aerial vehicles, etc.

The invention discloses a remote machine two-arm system with an imitation learning mechanism and a method. The remote machine two-arm system comprises a demonstrator teaching module, a self-designed action executing module composed of two humanoid machine arms and a digital rudder controller, an XBOX360 body sensor perception module, a remote upper computer communication module and an imitation learning algorithm study module. The module independent power supply mode is adopted, demonstrators do demonstration actions in front of an XBOX360 sensor, the XBOX360 is used for collecting action dataof the demonstrators, and data processing is conducted on a local upper computer (server side); learning is performed through an imitation learning algorithm, and real-time communication is achievedthrough the server side and a remote upper computer (client side); and then the remote client side sends the data to the controller in real time through a serial port, and the controller receiving thedata is used for guiding the two machine arms to imitate and learn the teaching actions of the demonstrators. By means of the remote machine two-arm system with the imitation learning mechanism and the method, machine arm operation intelligence is improved; and meanwhile, operation efficiency of the machine arms in a dangerous space is greatly improved and actual application significance is realized.

The invention discloses a vision-based deep imitation reinforcement learning driving strategy training method. The method comprises the steps of constructing an imitation learning network; training the imitation learning network; performing network splitting on the trained imitation learning network to obtain a sensing module; constructing a DDPG network to obtain a control module; completing theconstruction of a deep imitation reinforcement learning model through the sensing module and the control module; and training the deep imitation reinforcement learning model. An imitation learning network comprises five convolution layers and four full connection layers, the convolution layers are used for extracting features, and the full connection layers are used for predicting a steering angle, an accelerator and a brake opening degree; in addition, a reward function is set in the training process of the deep imitation reinforcement learning model, and comfort and safety of curve driving are guaranteed.

The invention brings forward an imitation display apparatus capable of dynamically performing calligraphy or painting imitation, and a display method thereof. A holographic screen and a micro projection technology are employed, dynamic imitation of calligraphy or paintings can be realized through dynamic demonstration of projection, the problem that an imitation learning apparatus can only realize static imitation in the prior art is solved, holographic screen imaging is employed, the sharpness, the contrast and the color reduction degree are quite high, and the display effect is quite good. Besides, a desk body is different from the shape of a conventional desk body, the desk body employs a structure with a rhombus-shaped side surface, projection light is enabled to maintain a certain angle with the visual angle of a user, direct light is avoided, and human eyes are protected. Besides, the invention also brings forward an imitation projection display method. User imitation learning information is compared with preset standard information. A learning result of a user is evaluated by use of an intelligent apparatus, and learning and improvement of the user can be facilitated.

The invention provides a mobile robot navigation method based on imitation learning and deep reinforcement learning. The mobile robot navigation method comprises the following steps: step 1, establishing an environment model of a mobile robot; 2, constructing a navigation control framework based on imitation learning and deep reinforcement learning algorithm coupling, and training the mobile robotmodel by using the coupled navigation framework; and 3, realizing a navigation task by utilizing the trained model.

Embodiments of the invention disclose a shooting processing method and a terminal. The method comprises the steps of obtaining a preview picture in a shooting preview interface, and determining at least one picture description tag corresponding to the preview picture; determining at least one target reference image corresponding to the picture description tag in at least one reference image stored in a cloud server, wherein the target reference image comprises at least one target image description tag, and the similarity obtained by performing similarity comparison on the picture description tag with the target image description tag exceeds a preset threshold; and generating and displaying prompt information about the target reference image, wherein the prompt information is used for prompting whether the target reference image is viewed or not. According to the embodiments of the method and the terminal, a user can be assisted to shoot a perfect image by gradually improving shooting skills through continuous imitation learning.

The invention discloses a mechanical arm action learning method and system based on third-person imitation learning. The method and system are used for automatic control of a mechanical arm so that the mechanical arm can automatically learn how to complete a corresponding control task by watching a third-party demonstration. According to the method and system, samples exist in a video form, and the situation that a large number of sensors are needed to be used obtaining state information is avoided; an image difference method is used in a discriminator module so that the discriminator module can ignore the appearance and the environment background of a learning object, and then third-party demonstration data can be used for imitation learning; the sample acquisition cost is greatly reduced; a variational discriminator bottleneck is used in the discriminator module to restrain the discriminating accuracy of a discriminator on demonstration generated by the mechanical arm, and the training process of the discriminator module and a control strategy module is better balanced; and the demonstration action of a user can be quickly simulated, operation is simple and flexible, and the requirements for the environment and demonstrators are low.

The invention discloses an imitation learning social navigation method based on a feature map fused with pedestrian information. According to the imitation learning social navigation method, a robot is guided to imitate the movement habits of experts by introducing an imitation learning method, the navigation method conforming to the social specifications is planned, the planning efficiency is improved, the problem that the robot is locked is solved, and the robot is helped to be better integrated into a man-machine co-integration environment. The method comprises the steps of: acquiring a time sequence motion state of a pedestrian through pedestrian detection and tracking in a sequence RGB image and three-dimensional point cloud alignment; then, in combination with two-dimensional laser data and a social force model, acquiring a local feature map marked with pedestrian dynamic information; and finally, establishing a deep network with the local feature map, a current speed of the robot and a relative position of a target as input and a robot control instruction as output, training with expert teaching data as supervision, and acquiring a navigation strategy conforming to social specifications.

The invention discloses a deep reinforcement learning strategy optimization defense method and device based on imitation learning, and the method comprises the steps: building an agent automatic driving simulation environment of deep reinforcement learning, constructing a target agent based on a deep Q network in reinforcement learning, and carrying out the reinforcement learning of the target agent to optimize the parameters of the deep Q network; utilizing the parameter-optimized deep Q network to generate a state action pair sequence of the target agent at T moments as expert data, wherein an action value in a state action pair corresponds to an action with a minimum Q value; constructing an adversarial agent based on the generative adversarial network, and performing imitation learning on the adversarial agent, that is, taking the state in the expert data as the input of the generative adversarial network, and taking the expert data as a label to supervise and optimize the parameters of the generative adversarial network; and performing adversarial training on the target agent based on the state generated by the adversarial agent, and then optimizing parameters of the deep Q network to achieve deep reinforcement learning strategy optimization defense.

The invention provides an Internet of Vehicles mobile edge computing task unloading method and system based on learning pruning, belongs to the technical field of Internet of Vehicles communication, and the method comprising the following steps: calculating vehicle parameters in an Internet of Vehicles mobile edge computing scene, calculating unloading parameters according to the vehicle parameters; constructing a task unloading utility model according to the unloading parameters; and solving an optimal solution of the task unloading utility model by using a branch and bound algorithm in combination with an imitation learning method, thereby selecting a task unloading mode in a utility optimization manner and determining computing resources obtained by bidding. On the premise of considering the mobility of the vehicle, a vehicle utility function in an Internet of Vehicles scene is established, so that the selection of the unloading mode by the vehicle and the calculation resource obtained by bidding are carried out in a utility optimization mode; when the task is selected to be unloaded to the service type vehicle, the vehicle running in different directions is selected, so that the transmission rate is increased; a branch and bound method is utilized, a pruning strategy based on learning is combined to accelerate the branch pruning process, and the complexity is reduced.

The invention discloses a drug molecule generation method based on adversarial imitation learning, drug molecules are generated based on adversarial imitation learning and multi-task reinforcement learning, and the method comprises the following steps: constructing an effective drug molecule library; establishing an improved drug molecule generation model: designing and realizing a multi-task reinforcement learning module, and designing and realizing an adversarial imitation learning module; pre-training the model; executing a drug molecule generation process; and generating a candidate drug molecule result. By adopting the technical scheme provided by the invention, optimization of biochemical properties of drug molecules can be effectively promoted, the stability of model training is improved, and better drug molecules are obtained.

The invention discloses a mobile robot path planning method based on deep reinforcement learning. The method comprises the steps of S1 determining state information according to a motion scene of a mobile robot; S2 initializing basic parameters of deep reinforcement learning, and pre-training a state value network weight through imitation learning; S3 carrying out forward transmission on the state information through a state value network, and using an epsilon-greedy strategy for guiding the robot to act; S4 obtaining rewards through a comprehensive reward function; S5 continuously updating the weight through the target value network, and updating related parameters; and S6 recording related data and a model which is finally trained in the training process to obtain an optimal navigation strategy of the robot. The method has a path planning scene aiming at the pedestrian environment in the public service field; and a state value network is designed by utilizing an artificial potential field method and an attention mechanism, so that the state information of the robot and a pedestrian is effectively expanded, and the state information interaction is promoted.

The invention belongs to the technical field of artificial intelligence. The invention discloses an autonomous learning method and system of an agent for man-machine cooperative work. The method comprises the steps of obtaining a cooperation data set, training a cooperation agent and a simulation agent according to the cooperation data set; and assessing whether the cooperation agent and the simulation agent meet assessment requirements or not according to the obtained assessment data generated by cooperation of the trained cooperation agent and the simulation agent in the environment, judgingwhether the trained simulation agent needs new imitation learning or not if the assessment requirements are met, and ending autonomous learning of the trained cooperation agent if the assessment requirements are not met. The system comprises a cooperation agent, a simulation agent and a server. Through the scheme, the dynamic change of the environment can be adapted, the same performance effect can be obtained in the similar environment, the demonstration behaviors of different teaches can be simulated, so that the trained intelligent agent can adapt to the dynamic change of the teaches, andthe teaches with different operation levels can also achieve the same cooperation effect.

The invention discloses a free floating target capturing method based on 3D vision and imitation learning. The method mainly comprises the following three parts: firstly, acquiring the position and posture of a target capture point in real time by using a depth camera; secondly, estimating a motion state by using Kalman filtering according to historical data, and predicting a track of free floating in a future period of time according to motion information; and finally, determining the capture opportunity and the pose of the capture point at the moment, collecting capture data of the person, establishing a skill model, and migrating the skill model to the robot for capture. Pose estimation, trajectory prediction and mechanical arm trajectory planning technologies in the process are completed on the basis of vision and imitation learning, autonomous capture of a free floating target is achieved, and capture requirements can be well met.