VR-based processing method and related device for manipulator training data

Abstract

The present application provides a method and related device for processing manipulator training data based on VR, the method comprising: determining training object information according to the training information of the user in the current training; generating virtual image information for instructing the virtual reality device to generate a virtual image according to the user's historical training information and training object information; when the virtual reality device generates a virtual image according to the virtual image information, determining the first coordinate information of the manipulator in the virtual coordinate system according to the target image including the manipulator collected by the virtual reality device; determining the target training object contacted by the manipulator according to the first coordinate information and the second coordinate information of at least one training object displayed by the virtual image; controlling the manipulator to perform the training action corresponding to the target training object, and sending a deletion instruction to the virtual reality device to instruct the deletion of the target training object. The present application can flexibly generate personalized virtual images and improve the interactivity and fun in training.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method for processing manipulator training data based on VR and related devices. Background Art

[0002] In order to improve the training efficiency of users with weakened or impaired limb abilities, relevant engineers have designed various auxiliary mechanical devices to assist users in limb ability training. For example, when training hands, users can wear a robot arm to train. However, existing robots only perform repetitive training movements, resulting in a monotonous training process, which causes users to have low intention to continue training and affects the rehabilitation effect. Summary of the invention

[0003] The present application provides a VR-based method for processing manipulator training data and related devices, which flexibly generate personalized virtual images through training object information and historical training information, improve the interactivity and fun of the training process, enhance the user's willingness to train, and thus improve the user's rehabilitation efficiency.

[0004] In a first aspect, an embodiment of the present application provides a method for processing manipulator training data based on VR, which is applied to a server of a manipulator system, wherein the manipulator system includes the server, a manipulator connected to the server, and a virtual reality device, and the method includes:

[0005] Determine training object information according to training information of the user, the training information including information of at least one training action and the number of training times corresponding to each training action, the training object information including information of at least one type of training object corresponding to the at least one training action and the number of generated training objects corresponding to each type of training object;

[0006] Generate virtual image information according to the total number of training actions performed by the manipulator in the historical training information of the user, the action completion frequency and the training object information, wherein the virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object, and the action completion frequency represents the ratio of the total number to the total number of training actions planned to be completed in the historical period;

[0007] Sending the virtual image information to the virtual reality device;

[0008] When the virtual reality device generates the virtual image according to the virtual image information, first coordinate information of the manipulator in a virtual coordinate system is determined according to a target image including the manipulator acquired by the virtual reality device, and the virtual coordinate system represents a coordinate system of the virtual image;

[0009] determining a target training object contacted by the manipulator according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image;

[0010] The manipulator is controlled to execute a training action corresponding to the target training object, and a deletion instruction is sent to the virtual reality device to instruct the deletion of the target training object.

[0011] In a second aspect, an embodiment of the present application provides a method for processing manipulator training data based on VR, which is applied to a virtual reality device, and the method includes:

[0012] receiving virtual image information, the virtual image information being used to instruct the virtual reality device to generate a virtual image including a training object, the virtual image information being information generated according to the total number of training actions performed by the manipulator in the user's historical training information, the frequency of action completion, and training object information, the training object information being information determined according to the training information of the user in the current training, the training information including information of at least one training action and the number of training times corresponding to each training action, the training object information including information of at least one type of training object corresponding to the at least one training action and the number of generated training objects corresponding to each type of training object, and the action completion frequency representing the ratio of the total number to the total number of training actions planned to be completed in the historical period;

[0013] generating the virtual image according to the virtual image information;

[0014] When generating the virtual image, a target image including the manipulator is collected, the manipulator is connected to a server, and the server is connected to the virtual reality device;

[0015] Sending the target image to the server, so that the server determines first coordinate information of the manipulator in a virtual coordinate system according to the target image, wherein the virtual coordinate system represents virtual coordinates of the virtual image;

[0016] receiving a deletion instruction instructing to delete the target training object when the server controls the manipulator to perform a training action corresponding to a target training object, the target training object being a training object contacted by the manipulator determined according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image;

[0017] The target training object in the virtual image is deleted according to the deletion instruction.

[0018] In a third aspect, an embodiment of the present application provides a processing device for manipulator training data based on VR, which is applied to a server of a manipulator system, wherein the manipulator system includes the server, a manipulator connected to the server, and a virtual reality device, and the device includes:

[0019] a first determining unit, determining training object information according to training information of a user, wherein the training information includes information of at least one training action and a number of training times corresponding to each training action, and the training object information includes information of at least one type of training object corresponding to the at least one training action and a number of generated training objects corresponding to each type of training object;

[0020] a generating unit, generating virtual image information according to the total number of training actions performed by the manipulator in the historical training information of the user, the action completion frequency and the training object information, wherein the virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object, and the action completion frequency represents a ratio of the total number to the total number of training actions planned to be completed in the historical period;

[0021] A sending unit, configured to send the virtual image information to the virtual reality device;

[0022] a second determining unit, which determines first coordinate information of the manipulator in a virtual coordinate system according to a target image including the manipulator acquired by the virtual reality device when the virtual reality device generates the virtual image according to the virtual image information, wherein the virtual coordinate system represents a coordinate system of the virtual image;

[0023] a third determining unit, configured to determine a target training object contacted by the manipulator according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image;

[0024] The control unit is used to control the manipulator to execute the training action corresponding to the target training object, and send a deletion instruction to the virtual reality device to instruct to delete the target training object.

[0025] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising at least one processor, a communication interface and a memory, wherein the communication interface is used to send and / or receive data, the memory is used to store a computer program, and the at least one processor is used to call the computer program stored in the memory to implement any method as described in the first aspect of the present application.

[0026] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a communication module, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method of the first aspect of the present application.

[0027] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium has a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes part or all of the steps described in the steps of any method in the first aspect of the embodiment of the present application.

[0028] In a seventh aspect, the present application provides a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application, or in any method of the second aspect. The computer program may be a software installation package.

[0029] It can be seen that in the embodiment of the present application, the training object information is determined according to the training information of the user in the current training, wherein the training information includes information of at least one training action and the number of training times corresponding to each training action, and the training object information includes information of at least one type of training object and the number of generated training objects corresponding to each type of training object. The training object information is determined by the training information, so that the determined training object information is more in line with the training needs of the user. Virtual image information is generated according to the user's historical training information and training object information, so that the virtual image generated according to the virtual image information is more personalized. The virtual image information is sent to the virtual reality device, and when the virtual reality device generates a virtual image according to the virtual image information, the first coordinate information of the manipulator in the virtual coordinate system is determined according to the target image including the manipulator collected by the virtual reality device. The target training object contacted by the manipulator is determined according to the first coordinate information and the second coordinate information of at least one training object displayed by the virtual image, the manipulator is controlled to perform the training action corresponding to the target training object, and a deletion instruction indicating the deletion of the target training object is sent to the virtual reality device. In the present application, personalized virtual images are flexibly generated according to the user's training information and historical training information, the interactivity and fun in the training are improved, the user's training willingness is improved, and the user's rehabilitation efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A schematic diagram of the structure of a manipulator system provided in an embodiment of the present application;

[0032] Figure 2 A schematic diagram of the structure of a manipulator provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0034] Figure 4 A flowchart of a method for processing manipulator training data based on VR provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of a virtual image generated by a virtual reality device provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of a display interface of a server provided in an embodiment of the present application;

[0037] Figure 7 A block diagram of the functional units of a device for processing VR-based manipulator training data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a manipulator system provided in an embodiment of the present application. Figure 1 As shown, the manipulator system includes a server, a virtual reality device and a manipulator. The server is connected to the manipulator and the virtual reality device respectively, so as to facilitate data exchange between the server and the manipulator, and between the server and the virtual reality device. It is understandable that the number of manipulators connected to the server can be set according to actual needs, for example, two or more can be connected, and there is no specific limitation here.

[0042] The server in this application can be a terminal device such as a smart phone, tablet computer, laptop computer, desktop computer, wearable device, head-mounted device, vehicle-mounted terminal, etc., which is installed with a control application (i.e., application client). The type of terminal device is not limited here. It should be understood that when the application client runs on the server, it can communicate with Figure 1 Data interaction is performed between the robot arm and the virtual reality device shown.

[0043] The virtual reality device includes a collection module that can obtain video information or image information. For example, the collection module can collect images including the manipulator. The server in the present application can also be replaced by other control devices, which are not specifically limited here.

[0044] See also Figure 2 , Figure 2 A schematic diagram of the structure of a manipulator provided in an embodiment of the present application. Figure 2As shown, the manipulator includes a manipulator palm 20, a finger joint drive module 21, a Bluetooth module, a finger fixing belt 22, a palm fixing belt 23, a power module 24 and a switch button 25. The power module 24 is partially arranged in the manipulator palm 20 for storing electrical energy and supplying energy to the manipulator. The switch button 25 is arranged on the manipulator palm 20 for controlling the start and stop of the manipulator. The finger fixing belt 22 and the palm fixing belt 23 are arranged on the manipulator palm 20, and the finger fixing belt 22 is located between the palm fixing belt 23 and the finger joint drive module 21. The manipulator palm 20 is connected to the finger joint drive module 21, and the degree of the joint of the finger joint drive module 21 can be adjusted according to demand. The Bluetooth module is arranged in the manipulator palm 20, and the manipulator can exchange data with the server through the Bluetooth module. Each finger of the manipulator corresponds to at least one joint driver, and the joint driver outputs drive information so that the corresponding finger on the manipulator moves, that is, each joint of each finger can be controlled separately so that the manipulator can make training movements. When the user uses the manipulator for training, the hand is bound to the manipulator through the palm fixing band 23 and the finger fixing band 22, and then the manipulator performs training actions based on the training instructions, such as grasping, holding, pinching, etc., and drives the user's hand to perform corresponding training actions, thereby realizing the user's passive training.

[0045] See also Figure 3 , Figure 3 A schematic diagram of the structure of a server provided in an embodiment of the present application. Figure 3 As shown, the server includes a processor 301, a memory 303, a communication module 302, a program 304 and a Bluetooth module 305, and the processor 301 is connected to the memory 303, the communication module 302 and the Bluetooth module 305 through an internal communication bus. It is understood that the number of processors 301 can be set according to actual needs, for example, one, two or more, and the number of processors 301 is not specifically limited here. The number of programs 304 can be set according to actual needs, for example, one, two or more, and the number of programs 304 is not specifically limited here.

[0046] The program 304 is stored in the memory 303 and executed by the processor 301. The program 304 includes instructions for executing any step in the following method embodiments.

[0047] The processor 301 may be, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, units and circuits described in conjunction with the disclosure of the present application. The processor 301 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication unit may be a communication module 302, a transceiver, a transceiver circuit, and the like, and the storage unit may be a memory 303.

[0048] The memory 303 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM).

[0049] The server may respectively exchange data with the manipulator and the virtual reality device through the communication module 302 , or may respectively exchange data with the manipulator and the virtual reality device through the Bluetooth module 305 , and no specific limitation is made here.

[0050] Please combine the following Figure 4 The method in the embodiment of the present application is described in detail:

[0051] See also Figure 4 , Figure 4 A flowchart of a method for processing manipulator training data based on VR provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, a method for processing manipulator training data based on VR is applied to a server of a manipulator system, wherein the manipulator system includes the server, a manipulator connected to the server, and a virtual reality device, and the method includes:

[0052] Step S401: determining training object information according to the user's training information.

[0053] Wherein, the training information includes information of at least one training action and the number of training times corresponding to each training action, and the training object information includes information of at least one type of training object corresponding to the at least one training action and the number of generated objects corresponding to each type of training object. In a specific example, the user training plan can be determined based on the training actions used by people in the same training stage and based on the advice of professionals, and the training information of the current training in the user training plan can be obtained. After obtaining the training information, the training object information is determined according to the training information. Wherein, the training information includes information of at least one training action and the number of training times corresponding to each training action. The training action can be a grasping, holding and pinching action, which is not specifically limited here. The training object is determined according to the training action, and the corresponding number of generated objects of the training object is determined according to the number of training times of each training action. Each training action corresponds to at least one type of training object, that is, one training action can correspond to one type of training object, or to multiple types of training objects. When one training action corresponds to multiple types of training objects, the sum of the multiple numbers of generated objects corresponding to the multiple types of training objects is the same as the value of the number of training times corresponding to the one training action. For example, a first training action corresponds to a first type of training object and a second type of training object, the first training action is trained 10 times, the first type of training object corresponds to a first generation number, the second type of training object corresponds to a second generation number, and the sum of the first generation number and the second generation number is 10.

[0054] In a possible example, if a training action corresponds to multiple types of training objects, the numerical value of the number of training times corresponding to a training action and the total number of types of the corresponding training objects are obtained, and the number of training objects of each type is determined according to the quotient of the total number divided by the numerical value. Alternatively, the number of training objects of each type is randomly generated according to the numerical value. For example, the first training action corresponds to the first type of training object and the second type of training object. The number of training times corresponding to the first training action is 10 times. The first type of training object corresponds to the first generation number, and the second type of training object corresponds to the second generation number. The first generation number and the second generation number can be randomly generated based on the numerical value of the number of training times, 10, that is, the first generation number can be 2, and the second generation number can be 8. The method of generating the number of training objects of each type can be set according to actual needs, and there is no specific limitation here.

[0055] Step S402 : generating virtual image information according to the total number of training actions performed by the robot arm in the historical training information of the user, the action completion frequency, and the training object information.

[0056] The virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object, and the action completion frequency represents the ratio of the total number to the total number of training actions planned to be completed in the historical period. In a specific example, the virtual image information is generated according to the user's historical training information and training object information, so that the virtual reality device generates a more personalized virtual image according to the virtual image information, thereby making the virtual image more in line with the user's training needs and improving the intelligence of the manipulator system.

[0057] Step S403: sending the virtual image information to the virtual reality device.

[0058] Among them, the virtual image information is sent to the virtual reality device, so that the virtual reality device generates a virtual image to provide data support for the manipulator to touch the training object.

[0059] Step S404, when the virtual reality device generates the virtual image according to the virtual image information, first coordinate information of the manipulator in the virtual coordinate system is determined according to the target image including the manipulator acquired by the virtual reality device.

[0060] The virtual coordinate system represents the coordinate system of the virtual image. The target image including the manipulator is collected by the virtual reality device to improve the image collection efficiency. The first coordinate information of the manipulator in the virtual coordinate system is determined according to the target image to provide data support for determining the training object in the virtual image that the manipulator contacts.

[0061] Step S405 : determining the target training object that the manipulator contacts according to the first coordinate information and second coordinate information of at least one training object displayed in the virtual image.

[0062] Please combine Figure 5 , Figure 5 A schematic diagram of a virtual image generated by a virtual reality device provided in an embodiment of the present application, such as Figure 5 As shown, the virtual image generated by the virtual reality device according to the virtual image information may include multiple different types of training objects, so it is necessary to determine which training object in the virtual image the manipulator is in contact with, so as to determine the training action to be performed. The second coordinate information of at least one training object displayed in the virtual image at the current time point is obtained, and the coordinate information of the first coordinate information and the second coordinate information that overlap is determined based on the obtained first coordinate information, and the training object corresponding to the overlapping coordinate information is set as the target training object to be determined to be in contact with the manipulator, so as to improve the positioning accuracy and thus improve the user experience.

[0063] Step S406: Control the manipulator to execute the training action corresponding to the target training object, and send a deletion instruction to the virtual reality device to instruct to delete the target training object.

[0064] After determining the target training object that the manipulator is touching, the manipulator is controlled to perform the training action corresponding to the target training object, thereby driving the user to perform the training action. A deletion instruction is sent to the virtual reality device, thereby instructing the virtual reality device to delete the target training object, making the interaction more flexible.

[0065] It can be seen that in this example, the training object information is determined by the user's training information, and then the virtual image information is generated based on the training object information, so that the virtual image generated according to the virtual image information meets the training requirements and is more personalized. When the manipulator contacts the corresponding training object, the manipulator performs the training action corresponding to the contacted training object, which improves the fun of the training process and thus improves the user's rehabilitation efficiency.

[0066] In a possible example, the historical training information includes the total number of training actions performed by the robot in a historical period and the frequency of action completion, the action completion frequency representing the ratio of the total number to the total number of training actions planned to be completed in the historical period, and the generating of virtual image information based on the total number of training actions performed by the robot in the historical training information of the user, the action completion frequency and the training object information includes: determining the training level of the user according to the total number and the action completion frequency in the historical training information; obtaining a first set corresponding to the training level, the first set including different display priorities and different initial display durations of different types of reference training objects, the display priority representing the order in which the reference training objects are displayed; searching the first set for the target initial display duration and target display priority of each type of the training object, and obtaining a second set consisting of the target initial display duration and target display priority of each type of the training object included in the training object information; and generating the virtual image information according to the training object information and the second set.

[0067] In a specific example, the user's historical training information includes the total number of training actions performed by the robot in the historical period and the frequency of action completion corresponding to the user. The action completion frequency represents the ratio of the total number to the total number of training actions planned to be completed in the historical period. The historical period is the period before the current time point. Among them, the higher the numerical value of the total number and the numerical value of the action completion frequency, the higher the corresponding training level. Different training levels correspond to different sets. The sets include different display priorities and different initial display durations for different types of reference training objects. The display priority represents the order in which the reference training objects are displayed in the generated virtual image, and the initial display duration represents the duration of the reference training objects displayed in the virtual image. The higher the training level, the shorter the initial display duration in the corresponding set, so as to meet the user's training needs and improve the fun of the user's training process.

[0068] After obtaining the total number and action completion frequency in the user's historical training information, determine the user's training level based on the total number and action completion frequency. Obtain a first set corresponding to the training level, then search for reference training objects that are the same as each type of training object included in the training object information from the first set, obtain the target initial display duration and target display priority corresponding to the same reference training object, and so on, obtain the target initial display duration and target display priority of each type of training object. The target initial display duration and target display priority of each type of training object included in the found training object information form a second set. Determine the image of each type of training object and the number of training objects of each type to be generated based on the training object information, and determine the order of appearance and display duration of different types of training objects in the generated virtual image based on the target initial display duration and target display priority corresponding to different types of training objects in the second set, so that the time point and display duration of the training objects in the virtual image are more in line with the user's training needs.

[0069] For example, if the display priority of the preset training object is higher, the order in which it appears in the virtual image is later. The initial display duration corresponding to the first type of training object is 5 minutes, and the display priority is level 2; the initial display duration corresponding to the second type of training object is 3 minutes, and the display priority is level 2; the initial display duration corresponding to the third type of training object is 5 minutes, and the display priority is level 4. Since the display priority of the first type of training object and the second type of training object is lower than the display priority of the third type of training object, the first type of training object and the second type of training object are displayed preferentially, and the two types of training objects can be displayed at the same time point. That is, in the generated virtual image information, the first type of training object and the second type of training object are set to be displayed at the same time point, and the initial display duration of the first type of training object is 5 minutes, and the initial display duration of the second type of training object is 3 minutes. After any type of training object displayed in the virtual image disappears, the training object with the next display priority begins to be displayed, that is, the third type of training object begins to be displayed.

[0070] It can be seen that in this example, the generated virtual image information is determined in combination with the user's historical training information, so that the virtual image generated according to the virtual image information better meets the user's training needs, improves the intelligence of the training process, and thus improves the user experience.

[0071] In a possible example, after determining the display order of different types of training objects according to the display priority level. The display interval duration between the last displayed training object and the next displayed training object can also be determined, and the display interval duration can be set according to actual needs. For example, if the last displayed training object is a training object of the first type, and the next displayed training object is a training object of the second type. The initial display duration of the first type of training object is 8 minutes, and the set display interval duration is 5 minutes. After the first type of training object is displayed for 5 minutes, there are still remaining training objects of the first type. The second type of training object and the remaining first type of training objects are still displayed in the virtual image at the same time, thereby improving the user's training enthusiasm.

[0072] In a possible example, if there are multiple training objects of the same type, after the manipulator contacts a preset number of training objects of the same type, the training intensity of the training movements corresponding to the remaining training objects of the same type is increased to achieve a better training effect. For example, the training objects belonging to the same type include the first training object, the second training object and the third training object. The preset number is set to 2, and it is determined that the training intensity is increased after the manipulator contacts two training objects of the same type. That is, after the manipulator has contacted any two training objects among the first training object, the second training object and the third training object, and then contacts the remaining one training object among the first training object, the second training object and the third training object, the training intensity of the training movement corresponding to the remaining one training object is increased. The training intensity represents the range value of the joint adjustment degree when the manipulator performs the training action. For example, the joint adjustment range is 30°~50°. The greater the training intensity, the larger the joint adjustment range. The size of the training intensity is determined according to the joint adjustment range, and the training intensity is visualized to make the control more accurate.

[0073] In a possible example, when the virtual reality device generates the virtual image according to the virtual image information, determining the first coordinate information of the manipulator in the virtual coordinate system according to the target image including the manipulator acquired by the virtual reality device includes: when the virtual reality device generates the virtual image according to the virtual image information, acquiring the target image including the manipulator through the virtual reality device; acquiring the preset object size and preset object position information in the target image; establishing a physical coordinate system according to the target image, the preset object size and the preset object position information; determining the physical coordinate information of the manipulator based on the target image and the physical coordinate system; calculating the mapping relationship between the physical coordinate system and the virtual coordinate system; and determining the first coordinate information according to the physical coordinate information and the mapping relationship.

[0074] In a specific example, when the virtual reality device generates a virtual image according to the virtual image information, the target image including the manipulator is collected by the virtual reality device, the preset object size and the preset object position information in the target image are obtained, and the physical coordinate system is established according to the target image, the preset object size and the preset object position information. It can be understood that the acquired target image may include multiple ones, and the preset object in the target image may be a manipulator, or an object that exists for a long time and fixedly in the space where the user is located. The specific type of the preset object is not limited here, and the preset object size and the preset object position information are used as a reference to establish a physical coordinate system. After obtaining the physical coordinate system, the position and size of the manipulator relative to the preset object in the target image are obtained, so as to determine the physical coordinate information of the manipulator in the physical coordinate system. The mapping relationship between the physical coordinate system and the virtual coordinate system is calculated, and then the first coordinate information of the manipulator in the virtual coordinate system is determined according to the physical coordinate information and the mapping relationship.

[0075] It can be seen that in this example, the physical coordinate information of the robot is virtualized to obtain the first coordinate information of the robot in the virtual coordinate system, so that the robot can interact with the training object in the virtual image, thereby improving the accuracy of the interaction and allowing users to have a richer usage experience.

[0076] In a possible example, after controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to delete the target training object, the method includes: if it is detected that the virtual image also includes a training object of the same type as the target training object, obtaining body data of the user collected by the manipulator, the body data being the body data of the user when the manipulator performs the training action corresponding to the target training object; if it is determined according to the body data that the user has a preset situation, determining a first delay duration corresponding to the preset situation; generating delay information according to the first delay duration, the delay information representing information instructing the virtual reality device to extend the display time of the same type of training object in the virtual image by the first delay duration; and sending the delay information to the virtual reality device.

[0077] In a specific example, after controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device, it is detected whether there is still a training object of the same type as the target training object in the virtual image. If so, the body data of the user collected by the manipulator is obtained, and the body data represents the body data of the user when the manipulator performs the training action corresponding to the target training object. If it is determined based on the body data that the user has a preset condition, the first delay duration corresponding to the preset condition is determined. Among them, the preset condition represents whether the user's hand has a spasm. After determining the first delay duration, delay information is generated based on the first delay duration, and then the delay information is sent to the virtual reality device, so that the virtual reality device extends the display time of the same type of training object in the virtual image by the first delay duration.

[0078] It can be seen that in this example, after the robot leads the user to perform the training action corresponding to the target training object, if the user encounters a preset situation, the display time of the same type of training object corresponding to the target training object will be extended, thereby improving the intelligence of the robot system and further improving the user experience.

[0079] In a possible example, after controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to instruct the deletion of the target training object, the method includes: if it is detected that the virtual image also includes a training object of the same type as the target training object, obtaining body data of the user collected by the manipulator, the body data being the body data of the user when the manipulator performs the training action corresponding to the target training object; if it is determined according to the body data that the user has a preset condition, determining an abnormality level of the user's body according to the body data; obtaining a deletion quantity value corresponding to the abnormality level; generating deletion information according to the deletion quantity value, the deletion information being used to instruct the virtual reality device to delete the training objects of the same type of the deletion quantity value in the virtual image; and sending the deletion information to the virtual reality device.

[0080] In a specific example, after controlling the manipulator to drive the user to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device, it is detected whether there are still training objects of the same type as the target training object in the virtual image. If there are, the body data of the user collected by the manipulator is obtained. According to the body data, it is determined that the user has a preset situation, and the abnormal level of the user's body is further determined according to the body data. Among them, the preset situation represents whether the user's hand has a spasm. Specifically, the automatic contraction degree and automatic contraction duration of the hand muscles in the body data can be obtained, and the abnormal level is determined according to the automatic contraction degree and automatic contraction duration. It is understandable that the physical conditions such as the amount of sweating of the user's hands or the number of heartbeats of the user in the body data can also be obtained to determine the abnormal level, and the specific physical conditions used to determine the abnormal level are not limited here. After determining the abnormal level, the deletion quantity value corresponding to the abnormal level is obtained; the deletion information is generated according to the deletion quantity value, and then the deletion information is sent to the virtual reality device, so that the virtual reality device deletes the same type of training objects of the deletion quantity value in the virtual image.

[0081] It can be seen that in this example, after the robot leads the user to perform the training action corresponding to the target training object, if the user encounters a preset situation, the abnormality level of the user's body is determined based on the physical data, and the deletion quantity value is determined based on the abnormality level to delete the same type of training objects in the virtual image, thereby improving the intelligence of the robot system and then improving the user experience.

[0082] In a possible example, after controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to delete the target training object, the method includes: if it is detected that the virtual image also includes a training object of the same type as the target training object, counting the display quantity value of the same type of training objects in the virtual image; obtaining a second delay duration corresponding to the display quantity value; generating delay information according to the second delay duration, the delay information representing information instructing the virtual reality device to extend the display time of the same type of training object in the virtual image by the second delay duration; and sending the delay information to the virtual reality device.

[0083] In a specific example, after controlling the manipulator to drive the user to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device, it is detected whether there are still training objects of the same type as the target training object in the virtual image. If so, the display quantity value of the same type of training objects in the virtual image is counted. After the manipulator contacts the target training object, the target training object will be deleted, and the display quantity value of the remaining training objects of the same type as the target training object in the virtual image is obtained, and then the second delay duration corresponding to the display quantity value is obtained. Delay information is generated according to the second delay duration, and then the delay information is sent to the virtual reality device. This instructs the virtual reality device to extend the display duration of the same type of training object in the virtual image by the second delay duration.

[0084] It can be seen that in this example, after the robot leads the user to perform the training action corresponding to the target training object, it also includes training objects of the same type as the target training object, then the display time of the same type of training objects corresponding to the target training object is extended, thereby improving the intelligence of the robot system and further improving the user experience.

[0085] In a possible example, when the virtual reality device generates the virtual image according to the virtual image information, after determining the first coordinate information of the manipulator in the virtual coordinate system according to the target image including the manipulator acquired by the virtual reality device, the method includes: if it is determined according to the first coordinate information and the second coordinate information that the manipulator does not touch the training object in the virtual image, then calculating a first duration from the start time point of generating the virtual image to the current time point; obtaining a target initial display duration corresponding to each training object in the at least one training object from the second set, and obtaining an initial display duration set consisting of the target initial display duration corresponding to each training object in the at least one training object; determining a training object corresponding to a target initial display duration less than or equal to the first duration in the initial display duration set as an eliminated training object; obtaining a display image of the eliminated training object in the virtual image; and sending the display image to the virtual reality device if it is detected that the display of the virtual image is complete.

[0086] In a specific example, after determining the first coordinate information of the manipulator in the virtual coordinate system according to the target image including the manipulator collected by the virtual reality device, if it is determined according to the first coordinate information and the second coordinate information that the manipulator has not touched the training object in the virtual image, it is necessary to determine whether the duration of the training object in the virtual image from the appearance time point to the current time point reaches the display duration. Specifically, calculate the first duration from the start time point of generating the virtual image to the current time point. Determine an initial display duration set consisting of the target initial display duration corresponding to each training object in at least one training object in the virtual image. Search for an initial display duration less than or equal to the first duration from the initial display duration set, and determine the training object corresponding to the initial display duration less than or equal to the first duration as the elimination training object. Obtain the display image of the eliminated training object in the virtual image, and if it is detected that the virtual image has been displayed, send the display image to the virtual reality device.

[0087] It can be seen that in this example, to determine that the manipulator has not touched the training object in the virtual image, it is necessary to detect whether the training object in the virtual image has reached the display time, and to eliminate the training object that has reached the display time in time, making the interaction more interesting and improving the user's enthusiasm for interaction. At the end of the training, the training object that the manipulator has not touched is displayed again to improve the training completion, thereby improving the rehabilitation efficiency.

[0088] The following is a detailed description of another method in the embodiment of the present application:

[0089] A method for processing manipulator training data based on VR, applied to virtual reality equipment, the method comprising:

[0090] receiving virtual image information, the virtual image information being used to instruct the virtual reality device to generate a virtual image including at least one training object, the virtual image information being information generated based on the total number of training actions performed by the manipulator in the user's historical training information, the frequency of action completion, and training object information, the training object information being information determined based on the training information of the user in the current training, the training information including information on at least one training action and the number of training times corresponding to each training action, the training object information including information on at least one type of training object corresponding to the at least one training action and the number of generated training objects corresponding to each type of training object, the action completion frequency representing the ratio of the total number to the total number of training actions planned to be completed in the historical period; and generating a virtual image including at least one training object corresponding to the at least one training action according to the virtual image information. The virtual image is generated by using image information; when generating the virtual image, a target image including the manipulator is collected, the manipulator is connected to a server, and the server is connected to the virtual reality device; the target image is sent to the server, so that the server determines the first coordinate information of the manipulator in a virtual coordinate system according to the target image, and the virtual coordinate system represents the virtual coordinates of the virtual image; when the server controls the manipulator to perform a training action corresponding to a target training object, a deletion instruction instructing to delete the target training object is received, and the target training object is a training object contacted by the manipulator determined according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image; and the target training object in the virtual image is deleted according to the deletion instruction.

[0091] Wherein, the virtual reality device is connected to the server so that the virtual reality device and the server can exchange data, and the server is connected to the manipulator to control the manipulator to perform training actions. The virtual reality device receives virtual image information, which is information generated by the server based on the user's historical training information and training object information. Wherein, the training information includes information about at least one training action and the number of training times corresponding to each training action. The training action can be actions such as grasping, holding and pinching, which are not specifically limited here. The training object is determined according to the training action, and the corresponding number of generation of the training object is determined according to the number of training times of each training action. Each training action corresponds to at least one type of training object, that is, one training action can correspond to one type of training object, or, to multiple types of training objects. When one training action corresponds to multiple types of training objects, the sum of the multiple numbers of generation corresponding to multiple types of training objects is the same as the numerical value of the number of training times corresponding to one training action.

[0092] After receiving the virtual image information, a virtual image is generated according to the virtual image information. The virtual image generated by the virtual reality device according to the virtual image information may include multiple different types of training objects. When generating the virtual image, a target image including the manipulator is collected, and then the target image is sent to the server so that the server determines the first coordinate information of the manipulator in the virtual coordinate system according to the target image, wherein the virtual coordinate system represents the virtual coordinates of the virtual image. The server determines the target training object contacted by the manipulator according to the first coordinate information and the second coordinate information of at least one training object displayed by the virtual image, and then controls the manipulator to drive the user to perform the training action corresponding to the target training object, and sends a deletion instruction to the virtual reality device to instruct the deletion of the target training object. The virtual reality device receives the deletion instruction instructing the deletion of the target training object, and then deletes the target training object in the virtual image according to the deletion instruction.

[0093] It can be seen that in this example, the virtual reality device generates a virtual image based on the virtual image information, and the virtual image information is generated based on the user's training information and historical training information, making the virtual image more personalized. In addition, the virtual reality device collects the target image including the manipulator and sends it to the server to determine the training object contacted by the manipulator, thereby improving the accuracy of the determination result. After determining the target training object contacted by the manipulator, the target training object in the virtual image is deleted in a timely manner to improve interactivity and thereby improve the fun of the training process.

[0094] In a possible example, the server includes a display module, and the display interface of the display module can be used to display the user's training report so that the user can adjust the training in time. Figure 6 , Figure 6 A schematic diagram of a display interface of a server provided in an embodiment of the present application, such as Figure 6 As shown, the display interface includes a first display bar 601, a second display bar 602, a third display bar 603 and a fourth display bar 604. Among them, the first display bar 601 is used to display user information, which may specifically include name: Xiaoxiao, age: 35 years old. The second display bar 602 is used to display the statistical results of data statistics, which may specifically include training mode: passive training. The third display bar 603 is used to display the maximum range of motion of the training action, for example, it can be 0 degrees to 60 degrees. The fourth display bar 604 is used to display training action parameter information, for example, the training action includes grasping and fisting, and the number of training times corresponding to each training action, that is, the number of training times corresponding to grasping is 5, and the number of training times corresponding to fisting is 10. It can be understood that the content displayed in the display bar and the number of display bars can be set according to needs, and there is no specific restriction here.

[0095] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0096] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0097] In the case of dividing each functional module according to each function, the following is combined Figure 7 A processing device for manipulator training data based on VR in an embodiment of the present application is described in detail. Figure 7 A functional unit composition block diagram of a VR-based manipulator training data processing device provided in an embodiment of the present application is applied to a server of a manipulator system, wherein the manipulator system includes the server, a manipulator connected to the server, and a virtual reality device, and the device includes:

[0098] A first determining unit 710 determines training object information according to training information of a user, wherein the training information includes information of at least one training action and a number of training times corresponding to each training action, and the training object information includes information of at least one type of training object corresponding to the at least one training action and a number of generated training objects corresponding to each type of training object;

[0099] A generating unit 720 generates virtual image information according to the total number of training actions performed by the manipulator in the historical training information of the user, the action completion frequency, and the training object information, wherein the virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object, and the action completion frequency represents a ratio of the total number to the total number of training actions planned to be completed in the historical period;

[0100] A sending unit 730, configured to send the virtual image information to the virtual reality device;

[0101] a second determining unit 740, which determines first coordinate information of the manipulator in a virtual coordinate system according to a target image including the manipulator acquired by the virtual reality device when the virtual reality device generates the virtual image according to the virtual image information, wherein the virtual coordinate system represents a coordinate system of the virtual image;

[0102] A third determining unit 750 determines a target training object contacted by the manipulator according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image;

[0103] The control unit 760 is used to control the manipulator to execute the training action corresponding to the target training object, and send a deletion instruction to the virtual reality device to instruct to delete the target training object.

[0104] In a possible example, the historical training information includes the total number of training actions performed by the robot in a historical period and the frequency of action completion, the action completion frequency representing the ratio of the total number to the total number of training actions planned to be completed in the historical period, and the generation unit 720 is also used to determine the training level of the user based on the total number and the action completion frequency in the historical training information; and obtain a first set corresponding to the training level, the first set including different display priorities and different initial display durations of different types of reference training objects, the display priority representing the order in which the reference training objects are displayed; and search for the target initial display duration and target display priority of each type of the training object in the first set to obtain a second set consisting of the target initial display duration and target display priority of each type of the training object included in the training object information; and generate the virtual image information based on the training object information and the second set.

[0105] In a possible example, the second determination unit 740 is further used to collect the target image including the manipulator through the virtual reality device when the virtual reality device generates the virtual image according to the virtual image information; and obtain the preset object size and preset object position information in the target image; and establish a physical coordinate system according to the target image, the preset object size and the preset object position information; and determine the physical coordinate information of the manipulator based on the target image and the physical coordinate system; and calculate the mapping relationship between the physical coordinate system and the virtual coordinate system; and determine the first coordinate information according to the physical coordinate information and the mapping relationship.

[0106] In a possible example, the device also includes a first acquisition unit, and the first acquisition unit is used to acquire the body data of the user collected by the manipulator if it is detected that the virtual image also includes a training object of the same type as the target training object, the body data being the body data of the user when the manipulator performs a training action corresponding to the target training object; and if it is determined according to the body data that the user has a preset situation, then determine a first delay duration corresponding to the preset situation; and generate delay information according to the first delay duration, the delay information representing information instructing the virtual reality device to extend the display time of the same type of training object in the virtual image by the first delay duration; and send the delay information to the virtual reality device.

[0107] In a possible example, the device also includes a second acquisition unit, and the second acquisition unit is used to obtain the body data of the user collected by the manipulator if it is detected that the virtual image also includes a training object of the same type as the target training object, and the body data is the body data of the user when the manipulator performs the training action corresponding to the target training object; and if it is determined according to the body data that the user has a preset situation, determine the abnormality level of the user's body according to the body data; and obtain a deletion quantity value corresponding to the abnormality level; and generate deletion information according to the deletion quantity value, the deletion information is used to instruct the virtual reality device to delete the training objects of the same type of the deletion quantity value in the virtual image; and send the deletion information to the virtual reality device.

[0108] In a possible example, the apparatus further includes a statistical unit, wherein if it is detected that the virtual image also includes a training object of the same type as the target training object, the statistical unit is used to count the display quantity value of the training objects of the same type in the virtual image; and obtain a second delay duration corresponding to the display quantity value; and generate delay information based on the second delay duration, wherein the delay information represents information instructing the virtual reality device to extend the display time of the training object of the same type in the virtual image by the second delay duration; and send the delay information to the virtual reality device.

[0109] In a possible example, the device also includes a calculation unit, which is used to calculate a first duration from the start time point of generating the virtual image to the current time point if it is determined based on the first coordinate information and the second coordinate information that the manipulator has not touched the training object in the virtual image; and obtain the target initial display duration corresponding to each training object in the at least one training object from the second set, and obtain an initial display duration set consisting of the target initial display duration corresponding to each training object in the at least one training object; and determine the training object corresponding to the target initial display duration less than or equal to the first duration in the initial display duration set as an elimination training object; and obtain a display image of the elimination training object in the virtual image; and if it is detected that the virtual image is displayed, send the display image to the virtual reality device.

[0110] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0111] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes an electronic device.

[0112] An embodiment of the present application further provides a computer program product, which includes a computer program. The computer program can be operated to enable a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0113] The computer program product may be a software installation package, and the computer includes an electronic device.

[0114] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0118] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0119] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions without departing from the spirit and scope of the present invention, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including software and hardware implementation methods, all of which are within the scope of protection of the present invention.

Claims

1. A method for processing manipulator training data based on VR, characterized in that: A server applied to a manipulator system, wherein the manipulator system comprises the server, a manipulator connected to the server, and a virtual reality device, wherein the method comprises: Determine training object information according to training information of the user, the training information including information of at least one training action and the number of training times corresponding to each training action, the training object information including information of at least one type of training object corresponding to the at least one training action and the number of generated training objects corresponding to each type of training object; Generating virtual image information according to the historical training information of the user and the training object information includes: Determine the training level of the user according to the total number and the action completion frequency in the historical training information, wherein the total number and the action completion frequency are the total number and the action completion frequency of the training actions performed by the manipulator in the historical period in the historical training information; Acquire a first set corresponding to the training level, the first set including different display priorities and different initial display durations of different types of reference training objects, the display priorities representing the order in which the reference training objects are displayed; searching the target initial display duration and the target display priority of each type of the training object in the first set, and obtaining a second set consisting of the target initial display duration and the target display priority of each type of the training object included in the training object information; generating the virtual image information according to the training object information and the second set, wherein the virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object; Sending the virtual image information to the virtual reality device; When the virtual reality device generates the virtual image according to the virtual image information, first coordinate information of the manipulator in a virtual coordinate system is determined according to a target image including the manipulator acquired by the virtual reality device, and the virtual coordinate system represents a coordinate system of the virtual image; determining a target training object contacted by the manipulator according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image; Controlling the manipulator to perform a training action corresponding to the target training object, and sending a deletion instruction to the virtual reality device to instruct to delete the target training object; If it is determined according to the first coordinate information and the second coordinate information that the manipulator does not touch the training object in the virtual image, then calculating a first duration from the start time point of generating the virtual image to the current time point; Acquire a target initial display duration corresponding to each training object in the at least one training object from the second set, and obtain an initial display duration set consisting of the target initial display duration corresponding to each training object in the at least one training object; Determine the training object corresponding to the target initial display duration that is less than or equal to the first duration in the initial display duration set as the eliminated training object; Acquire a display image of the elimination training object in the virtual image; If it is detected that the display of the virtual image is completed, the displayed image is sent to the virtual reality device.

2. The method according to claim 1, characterized in that When the virtual reality device generates the virtual image according to the virtual image information, determining the first coordinate information of the manipulator in the virtual coordinate system according to the target image including the manipulator acquired by the virtual reality device includes: When the virtual reality device generates the virtual image according to the virtual image information, the target image including the manipulator is collected by the virtual reality device; Acquire the preset object size and preset object position information in the target image; Establishing a physical coordinate system according to the target image, the preset object size and the preset object position information; Determining physical coordinate information of the manipulator based on the target image and the physical coordinate system; Calculating a mapping relationship between the physical coordinate system and the virtual coordinate system; The first coordinate information is determined according to the entity coordinate information and the mapping relationship.

3. The method according to claim 1 or 2, characterized in that: After controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to instruct to delete the target training object, the method includes: If it is detected that the virtual image also includes a training object of the same type as the target training object, obtaining the body data of the user collected by the manipulator, the body data being the body data of the user when the manipulator performs a training action corresponding to the target training object; If it is determined according to the body data that the user has a preset condition, determining a first delay duration corresponding to the preset condition; generating delay information according to the first delay duration, wherein the delay information represents information instructing the virtual reality device to extend the display duration of the same type of training object in the virtual image by the first delay duration; The delay information is sent to the virtual reality device.

4. The method according to claim 1 or 2, characterized in that: After controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to instruct to delete the target training object, the method includes: If it is detected that the virtual image also includes a training object of the same type as the target training object, obtaining the body data of the user collected by the manipulator, the body data being the body data of the user when the manipulator performs a training action corresponding to the target training object; If it is determined according to the body data that the user has a preset condition, determining the abnormality level of the user's body according to the body data; Obtain the deletion quantity value corresponding to the abnormal level; generating deletion information according to the deletion quantity value, wherein the deletion information is used to instruct the virtual reality device to delete the same type of training objects of the deletion quantity value in the virtual image; The deletion information is sent to the virtual reality device.

5. The method according to claim 1 or 2, characterized in that: After controlling the manipulator to perform the training action corresponding to the target training object and sending a deletion instruction to the virtual reality device to instruct to delete the target training object, the method includes: If it is detected that the virtual image also includes a training object of the same type as the target training object, counting the displayed number of the training objects of the same type in the virtual image; Obtaining a second delay duration corresponding to the displayed quantity value; generating delay information according to the second delay duration, wherein the delay information represents information instructing the virtual reality device to extend the display duration of the same type of training object in the virtual image by the second delay duration; The delay information is sent to the virtual reality device.

6. A method for processing manipulator training data based on VR, characterized in that: Applied to virtual reality equipment, the method comprises: Receive virtual image information, the virtual image information is used to instruct the virtual reality device to generate a virtual image including a training object, the virtual image information is information generated according to the training object information and a second set, the second set is a set consisting of a target initial display duration and a target display priority of each type of the training object found in the first set, the first set is a set determined by the training level of the user determined according to the total number and the action completion frequency in the historical training information, the total number and the action completion frequency are the total number and the action completion frequency of the training actions performed by the manipulator in the historical period in the historical training information, the first set includes different display priorities and different initial display durations of different types of reference training objects, the display priority represents the order of display of the reference training objects, the training object information is information determined according to the training information of the user in the current training, the training information includes information of at least one training action and the number of training times corresponding to each training action, and the training object information includes information of at least one type of the training object corresponding to the at least one training action and the number of generated objects corresponding to each type of the training object; generating the virtual image according to the virtual image information; When generating the virtual image, a target image including a manipulator is collected, the manipulator is connected to a server, and the server is connected to the virtual reality device; Sending the target image to the server, so that the server determines first coordinate information of the manipulator in a virtual coordinate system according to the target image, wherein the virtual coordinate system represents virtual coordinates of the virtual image; receiving a deletion instruction instructing to delete the target training object when the server controls the manipulator to perform a training action corresponding to a target training object, the target training object being a training object contacted by the manipulator determined according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image; Deleting the target training object in the virtual image according to the deletion instruction; If it is determined based on the first coordinate information and the second coordinate information that the manipulator has not touched the training object in the virtual image, then a display image sent by the server is received, wherein the display image is an image sent by the server after detecting that the virtual image is displayed. The display image is an image in which the training object in the virtual image is eliminated. The eliminated training object is a training object corresponding to a target initial display time that is less than or equal to a first time in an initial display time set. The first time is the time from the start time point of generating the virtual image to the current time point. The initial display time set is a set of target initial display times corresponding to each training object in the at least one training object obtained in the second set.

7. A processing device for manipulator training data based on VR, characterized in that: A server applied to a manipulator system, the manipulator system comprising the server, a manipulator connected to the server, and a virtual reality device, the device comprising: a first determining unit, determining training object information according to training information of a user, wherein the training information includes information of at least one training action and a number of training times corresponding to each training action, and the training object information includes information of at least one type of training object corresponding to the at least one training action and a number of generated training objects corresponding to each type of training object; a generating unit, generating virtual image information according to the historical training information of the user and the training object information; The generating unit is further used to determine the training level of the user according to the total number and the action completion frequency in the historical training information, wherein the total number and the action completion frequency are the total number and the action completion frequency of the training actions performed by the manipulator in the historical period in the historical training information; Acquire a first set corresponding to the training level, the first set including different display priorities and different initial display durations of different types of reference training objects, the display priorities representing the order in which the reference training objects are displayed; searching the target initial display duration and the target display priority of each type of the training object in the first set, and obtaining a second set consisting of the target initial display duration and the target display priority of each type of the training object included in the training object information; generating the virtual image information according to the training object information and the second set, wherein the virtual image information is used to instruct the virtual reality device to generate a virtual image including the training object; A sending unit, configured to send the virtual image information to the virtual reality device; a second determining unit, which determines first coordinate information of the manipulator in a virtual coordinate system according to a target image including the manipulator acquired by the virtual reality device when the virtual reality device generates the virtual image according to the virtual image information, wherein the virtual coordinate system represents a coordinate system of the virtual image; a third determining unit, configured to determine a target training object contacted by the manipulator according to the first coordinate information and second coordinate information of at least one training object displayed by the virtual image; A control unit, configured to control the manipulator to execute a training action corresponding to the target training object, and to send a deletion instruction to the virtual reality device to instruct the deletion of the target training object; a calculation unit, configured to calculate a first duration from a start time point of generating the virtual image to a current time point if it is determined according to the first coordinate information and the second coordinate information that the manipulator does not contact the training object in the virtual image; Acquire a target initial display duration corresponding to each training object in the at least one training object from the second set, and obtain an initial display duration set consisting of the target initial display duration corresponding to each training object in the at least one training object; Determine the training object corresponding to the target initial display duration that is less than or equal to the first duration in the initial display duration set as the eliminated training object; Acquire a display image of the elimination training object in the virtual image; If it is detected that the display of the virtual image is completed, the displayed image is sent to the virtual reality device.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 5.

Images