Training system, method and program

Abstract

To enable highly efficient training using VR technology, with regard to a farm crop cultivation technology.SOLUTION: Provided is a cultivation technology education and training system for the farm crop displayed in a virtual space which is the object of training. This system comprises: a training scenario storage unit for storing, in association with a work procedure, a work object region on the farm crop displayed in the virtual space and work means which is a tool used for the work object region; a three-dimensional display control unit for causing a display device to display the farm crop and work means in a three-dimensional display mode along with the progress of the work procedure; a detection signal output unit for detecting the position or motion of a user or an operating device and outputting a detection signal; an operation detection unit for causing the work means displayed in the virtual space to be changed on the basis of the detection signal; and an operation assessment unit for assessing the result of operation on the basis of the coordinates of the work object region in the virtual space and the coordinates and operation content of the work means. The three-dimensional display control unit causes the farm crop to be changed on the basis of the assessment result of the operation assessment unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a training system, method, and program that uses virtual reality (hereinafter referred to as "VR") technology to provide training in mastering cultivation techniques for agricultural crops such as grapes, such as thinning grape berries, thinning clusters, and pruning. [Background technology]

[0002] Conventionally, patent proposals have been made regarding methods for cultivating agricultural crops, such as grapes (see, for example, Patent Documents 1 and 2).

[0003] Generally, agricultural cultivation often requires skill, and traditionally workers have acquired the skills by working in the field. However, this requires waiting until the time when practical training is possible, and there is also a limit to the amount and number of agricultural crops that can be trained.

[0004] In response to this problem, Patent Document 3 proposes that users (trainees) have a simulated experience of farming using virtual images, rather than actual farming experience. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 4418863 [Patent Document 2] Patent No. 5206925 [Patent Document 3] JP 2009-87309 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, because there are a wide variety of techniques for cultivating agricultural crops, it is important to consider how to build and train a system according to the type and characteristics of the crop. For example, Patent Document 3 does not disclose anything about what kind of system should be used to train workers in the thinning process, which is necessary for cultivating grapes, and how to implement the training.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a training system, method, and program that uses VR technology to enable highly effective training in agricultural crop cultivation techniques, such as grape thinning. [Means for solving the problem]

[0008] In order to achieve the above object, the training system of the present invention is a training system for educating and training a training subject on a cultivation technique of a crop displayed in a virtual space of a display device, a training scenario storage unit that stores one or more work target areas for the crop displayed in a virtual space in association with a work procedure, and stores a work means that is a tool to be used for the work target areas; a three-dimensional display control unit that causes a display device to display images of the crops and the work means in three dimensions as the work procedure progresses; a detection signal output unit that detects a position or movement of a user or an operating device and outputs a detection signal; an operation detection unit that changes an image of the working tool displayed in a virtual space based on the detection signal; an operation determination unit that determines a result of an operation based on coordinates of the work target portion in a virtual space, and on coordinates and operation content of the working means; The three-dimensional display control unit changes the image of the crops based on a result of the determination by the operation determination unit.

[0009] In the present invention, the image of the working means in the virtual space is moved based on the movement of the user or the operation device such as a controller in the real space. Then, the result of the worker is judged based on the coordinates of the working means and the operation content and the coordinates of the work target part in the virtual space, and the image of the crop in the virtual space is changed based on the judgment result. In this way, it is possible to assign an appropriate working means in the virtual space according to the crops to be trained, and the work result by the working means can be efficiently judged.

[0010] Preferably, the device further includes an operation judgment output unit that generates and outputs an image of the growth process of the crops after the operation, which has changed based on the judgment result of the operation judgment unit, so that the user can easily know how the result of the operation will affect the subsequent growth of the crops.

[0011] When the crop is grapes, it is preferable to have at least two or more nodes for each bunch connected in order toward the end, that is, a data structure having a connection relationship and directionality, with each bunch being a node, which is a cluster, a stem, a stem, a branch, and a fruit. In this case, the work target part is any one of the nodes, and the work means is a pair of scissors.

[0012] In addition, the training system according to the present invention is characterized in that a first virtual object for collision detection that is not displayed on the display device can be set to at least one of the nodes, and a second virtual object for collision detection that is not displayed on the display device can be set between the blades of the scissors, and when the operation determination unit detects a collision (including overlap) between the first virtual object and the second virtual object, it distinguishes and displays the nodes connected to the node where the collision was detected from the node toward the end (fruit or grain side), and further, when the scissors perform a predetermined operation (for example, a closing operation), it determines that the distinguished node has been cut. This allows the user to check the fruit to be cut and perform the cutting operation.

[0013] The training method according to the present invention is a method for educating and training a training subject on a cultivation technique of a crop displayed in a virtual space of a display device, a process of storing one or more work target areas for the agricultural crop in association with a work procedure, and storing a work means which is a tool to be used for the work target areas; a three-dimensional display control process for displaying images of the crops and the working means in three dimensions on the display device as the work procedure progresses; an operation detection process for detecting a position or movement of a user or an operation device and outputting a detection signal; and an operation detection process for changing an image of the working means displayed in a virtual space based on the detection signal; an operation determination process for determining a result of an operation based on coordinates of the work target portion in a virtual space, and on coordinates and operation content of the working means; A first virtual object for collision detection that is not displayed on the display device is set for the crop, and a second virtual object for collision detection that is not displayed on the display device is set for the work means. 2 The virtual object is set, The operation determination process further includes: a first process of detecting a collision between the first virtual object and the second virtual object in a virtual space and thereby changing a part or all of an image of the crop; a second process of detecting an operation of the working means in a state in which a collision between the first virtual object and the second virtual object in a virtual space has been detected after the first process, and further changing the image portion changed by the first process; The present invention is characterized by comprising:

[0014] The program according to the present invention is a program for educating and training a person on the cultivation technique of a training target crop displayed in a virtual space, a process for storing one or more work target areas for the crop displayed in a virtual space in association with a work procedure, and storing a work means which is a tool to be used for the work target areas; and a three-dimensional display control process for displaying the crop and the work means in three dimensions on a display device as the work procedure progresses. executes an operation detection process for detecting a position or movement of a user or an operation device and outputting a detection signal, an operation detection process for changing the working means displayed in a virtual space based on the detection signal, and an operation determination process for determining a result of the operation based on the coordinates of the work target portion in the virtual space, the coordinates of the working means, and the operation content, The three-dimensional display control process changes the crop based on a determination result of the operation determination process. 、 Furthermore, an operation determination output process is executed to generate and output an image of the growth process of the crops after the operation, which has changed based on the determination result of the operation determination process. It is characterized by: Effect of the Invention

[0015] In the present invention, VR technology is used to enable highly effective training in crop cultivation techniques, such as thinning of grains. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a functional block diagram of a training system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is an explanatory diagram of an apparatus configuration and a training object according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a data structure diagram of the training scenario storage unit in FIG. 1. [Figure 4] 4 is a data configuration diagram of the operation determination storage unit of FIG. 1, in which FIG. 4(a) shows operation determination records of individual users, and FIG. 4(b) shows statistically processed operation records of multiple users. [Diagram 5] FIG. 2 is an explanatory diagram of the operation of the training system for grape thinning according to the present embodiment. [Figure 6] This is an overall conceptual diagram of the hierarchical structure of the grape model. [Figure 7] FIG. 2 is an explanatory diagram of the hierarchical structure of the grape model. [Figure 8] FIG. 2 is an explanatory diagram of a coordinate system of a grape model according to the present embodiment. [Figure 9] 11 is an explanatory diagram of setting virtual objects for a grape model and scissors. FIG. [Figure 10] 11 is an explanatory diagram of setting virtual objects for a hierarchical grape model. FIG. [Figure 11] FIG. 2 is an explanatory diagram of a controller and a coordinate system according to the present embodiment. [Figure 12] 4 is a flowchart showing a processing procedure of an operation determination unit in FIG. 1; [Figure 13] These are explanatory diagrams of the evaluation method for the thinning process according to this embodiment, where Figure 13(a) shows the state of a bunch after scores have been assigned to all berries, both those that are subject to thinning and those that are not, and Figure 13(b) shows the state of a bunch after the berries that are subject to thinning have been thinned. [Figure 14] FIG. 2 is an explanatory diagram showing an example of a data structure of a grape model. [Figure 15] FIG. 2 is an explanatory diagram of a method for evaluating the grain picking operation according to the present embodiment. [Figure 16] FIG. 1 is an explanatory diagram of the growth process after plucking. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of a training system according to the present invention will be described with reference to the drawings. In Fig. 1, the training system 1 is composed of a virtual content display device 10, an operation device 50, and a training scenario / operation record judgment device (hereinafter simply referred to as "judgment device") 60. For example, a head mounted display (hereinafter simply referred to as "HMD") can be used as the virtual content display device 10. The following describes an example in which this HMD is used.

[0018] The determination device 60 and the HMD 10 are connected via a wired or wireless communication line. The determination device 60 can be realized by a general-purpose computer, but may also be realized as an integrated part of the HMD 10. The operation device 50 can also be realized by using the functions of the HMD 10.

[0019] The determination device 60 stores three-dimensional data of the training object to be transmitted to the HMD 10, operation procedure data of the training object, etc. The determination device 60 also performs determination and statistical analysis on the user's operation content based on the operation information transmitted from the HMD 10. Then, the determination device 60 changes the training scenario, such as the operation procedure and determination criteria, based on the results of the statistical analysis.

[0020] The operation device 50 can be configured as a controller held or worn by the user, or hand tracking that detects the movement of the user's fingers, etc. This operation device 50 is sufficient as long as it can detect the movement of the user, and if it is possible to operate the training object using the user's line of sight or a detected specific object (for example, an image of the user's fingers), it also includes a function to detect the line of sight or a specific object.

[0021] The HMD 10 displays the training object in a three-dimensional virtual space based on the three-dimensional data of the training object sent from the determination device 60, and obtains operation information such as the position of the means for operating the training object (e.g., scissors, which will be described later) and the operation content (e.g., movement of the scissors, opening and closing operation, etc.) based on the detection signal sent from the operation device 50. This operation information is sent to the determination device 60, which determines the operation content.

[0022] More specifically, the judgment device 60 includes a 3D model memory unit 61 that stores three-dimensional data of the training object, a training scenario memory unit 62 that stores scenarios (training scenarios) for training the user, such as operation procedures, and an operation judgment memory unit 63 that stores the results of operation judgment.

[0023] The judgment device 60 also includes a 3D-CG display control command unit 71 that outputs control information for virtually displaying information such as a training object and operation instructions according to a training scenario on the HMD 10 based on data stored in the 3D model storage unit 61 and the training scenario storage unit 62, an operation judgment unit 72 that evaluates the operation content of the user based on the operation information sent from the HMD 10 and writes the evaluation result in the operation judgment storage unit 63, an operation judgment output unit 73 that outputs the evaluation result stored in the operation judgment storage unit 63, a statistical analysis unit 74 that performs statistics and analysis based on the evaluation result, and a scenario modification editing unit 75 that edits the data of the training scenario based on the result of this statistical analysis. The scenario modification editing unit 75 may edit the training scenario manually using a user interface (not shown), or may automatically perform editing processing based on the result of the statistical analysis.

[0024] The HMD 10 includes a 3D-CG display unit 11 that displays a 3D object in the virtual space of a display device (not shown) based on control information sent from a 3D-CG display control command unit 71 of the determination device 60, and an operation detection unit 12 that is set specific to an application program and detects coordinate acquisition in the virtual space in accordance with the operation of the operation device 50 and operation contents specific to the application program based on a detection signal passed from the operation device 50. Specifically, based on a detection signal such as the movement of the operation device 50 such as a controller or the pressing of a button, the operation detection unit 12 detects three-dimensional coordinates and movement information in the virtual space of a predetermined working means such as scissors described later, or operation contents (for example, opening and closing of scissors, etc.). The information detected by the operation detection unit 12 is passed to the operation determination unit 72 as operation coordinates (operation coordinate information and operation content information).

[0025] The operation device 50 includes a detection signal output unit 51 that detects data used for operation determination, such as a user's motion or operation, and passes the data to the operation detection unit 12 of the HMD 10. This detection signal output unit 51 automatically acquires the position of the operation device 50 and the like when the HMD 10 is started up. Here, each of the means 11, 12, 51, and 71 to 75 can be realized by a program as a function of a computer.

[0026] FIG. 2 shows an example of the device configuration of the training system 1 and an image of a training object displayed on the HMD 10. In the virtual space of the HMD 10 worn by the user, a right-hand object 80a, a left-hand object 80b, a working means 81 which is a tool for agricultural work, and a training object 82 are displayed. In the example of FIG. 2, the right-hand object 80a and scissors 81 which are a working means held by the object move in conjunction with the movement of the controller 50 operated by the user. For example, when a button on the right-hand controller 50 is pressed, this information is passed from the detection signal output unit 51 to the operation detection unit 12, and operation information (closing operation of the scissors 81) generated by the operation detection unit 12 is passed to the operation determination unit 72 and the 3D-CG display control command unit 71. The 3D-CG display control command unit 71 generates control information for a state change (closing operation of the scissors 81 and an image of the right-hand object 80a linked thereto) based on this operation information, and displays it on the display device via the 3D-CG display unit 11. As a result, the images of the right hand object 80a and scissors 81 change in the virtual space. The left hand object 80b moves in the virtual space using hand tracking technology, but it may also be moved by a left hand controller. Note that known methods such as head tracking, position tracking, hand tracking, and eye tracking can be used to track the user's movements.

[0027] When training in picking grapes as an agricultural crop, a bunch of grapes with multiple grapes is displayed in three dimensions as a training object 82 on the HMD 10. In the example of Fig. 2, due to the positional relationship between the grapes 82 as the training object and the left hand object 80b, the bunch of grapes 82 moves and rotates in response to the movement of the user's left hand. In addition, due to the positional relationship in the virtual space between the grapes 82 and the scissors 81, grapes at a predetermined position are picked as the scissors 81 are opened and closed.

[0028] Next, an example of data in the training scenario storage unit 62 is shown in Fig. 3. In the training scenario, the system operation content, operation conditions, guidance content, etc. are set for each item (work item) of the training procedure. Multiple training scenarios are provided for each training subject. The training scenario may be branched depending on the judgment result of each item.

[0029] In FIG. 3, the work items are stored in the order of the training procedure, and for each work item, it is possible to set the presence or absence of guidance display, the presence or absence of judgment, the work means (identification information), the work object (identification information), the behavior during the work, the time limit for the work, the enable / disable of scoring, and setting data for guidance display. For example, if a flag is set in the guidance display field, the guidance set in the setting data is displayed as the scenario progresses. If a flag is set in the judgment field, a judgment program is started when the user advances to that item. Therefore, appropriate training can be performed by changing the flag setting according to the user's level, such as the user's work ability and proficiency. Note that these flags may be set manually, but they may also be set automatically by the scenario change editing unit 75 based on the identification information of the user to be trained and the user's level.

[0030] The work means is a tool used for work, which moves in the virtual space in conjunction with the user's operation. The work object is the agricultural crop to be worked on. In the training scenario data of this embodiment, different work means can be set according to the work object for each work item of the training scenario.

[0031] Fig. 4(a) is an example of the data configuration of the operation judgment storage unit 63. The judgment results of each work item are stored for each user ID and training scenario ID. Fig. 4(b) is an example of statistical data for each group. The 3D model storage unit 61 stores three-dimensional coordinates for displaying a 3D model of a training object.

[0032] Below, an overview of the operation of this embodiment will be described using an example in which the training object is grapes and the work item is picking work. In the following description, it is assumed that the user holds controllers 50 in the right and left hands, operates scissors (working means) 81 in the virtual space with the controller in the right hand, and rotates and tilts a bunch of grapes 82 with the controller in the left hand.

[0033] In the grape picking operation, as shown in Fig. 5, when scissors 81 come into contact with a grape to be picked in the virtual space, the color of the grape to be picked changes. When the user presses the trigger button on the controller 50 in the right hand, the grape is cut. The bunch can also be rotated by operating the controller 50 in the left hand. When the left hand controller 50 is moved to bring the left hand object 80b in the virtual space closer to a bunch of grapes 82, a rotation mark is displayed at the base of the bunch. In this state, the bunch 82 rotates when the user rotates the left hand controller 50 in the direction of the rotation mark.

[0034] A bunch of grapes with multiple grapes can be represented by the 3D model shown in Figure 6. In Figure 6, the data structure of grapes is managed as a unit of "bunch," with the parts that make up the bunch from the base to the fruit at the end being managed as nodes, such as "stalk," "grain wheel," "internal branches," and "fruit." Each part (node) has information (identification information) to identify the part, such as S: bunch, E: stalk, C: grain wheel, K: internal branches, and T: fruit.

[0035] As shown in FIG. 6, for example, object 1 is given attribute information (hereinafter referred to as "3D object name") indicating the route through which it is connected, such as "S01_E01_C01_T14". This 3D object name can be constructed by attaching a serial number to the alphabet for identifying each part, and connecting each part with a symbol such as an underscore. In this case, the underscore functions as information (branch) indicating the connection relationship of each part (node). By connecting the identification information of each part (node) in order from the base of the tassel to the corresponding node, the order of connection and branching of each node can be effectively expressed.

[0036] The grape model is not limited to the above, and it is sufficient if the training scenario includes as nodes the parts on which the working means 81 acts with respect to the work on the grapes 82. For example, as shown in Fig. 7, the "fruit" of a grape may be further divided into a "stalk" and a "grain," and the working means 81 may act on the "grain."

[0037] Although not shown, each part (node) has a 3D coordinate. The 3D coordinate can be set as an absolute coordinate in the virtual space, but can also be expressed as a relative coordinate with the base of the bunch as the reference point and the stem as the reference axis (e.g., y-axis) of the bunch, as shown in Fig. 8. Then, the image of the grape bunch 82 is rotated in conjunction with the movement of the controller 50 by a process to be described later.

[0038] As shown in Figs. 9 and 10, a virtual object 1 can be set to each node constituting the grapes, and a virtual object 2 can be set to the working means. The virtual objects 1 and 2 are transparent objects of specific shapes that are not displayed in the virtual space. Whether or not the virtual object 1 and the virtual object 2 collide is determined by processing of an operation determination unit 72, which will be described later. Depending on the result of this determination, different processing is executed for the work target. The virtual object can be represented as a cube or a rectangular parallelepiped as shown in Fig. 9, but is not limited thereto and can be represented as a sphere or other shapes.

[0039] The operation of this training system 1 for the task of thinning grapes will be described. In this task, the user holds controller 1 (50) and controller 2 (50) in their right and left hands, respectively, in real space. Controller 1 (50) operates scissors 81 in virtual space, and controller 2 (50) rotates and displaces the bunch of grapes that is the subject of the task in virtual space. The allocation of the operations of controllers 1 and 2 (50) and the task means in virtual space can be arbitrarily set in the training scenario shown in FIG. 3.

[0040] When the training system 1 is started, the 3D-CG display control command unit 71 accesses the training scenario storage unit 62 and the 3D model storage unit 61, and outputs control information to the HMD 10 for displaying 3D objects such as the training subject and working means of the training scenario to be executed in the virtual space. The 3D-CG display unit 11 of the HMD 10 displays an image of the virtual space in the user's field of view on a display device (not shown) based on the control information received from the 3D-CG display control command unit 71. Specifically, as shown in Fig. 2, the user's hands 80a and 80b, a pair of scissors 81 for picking grapes as the working means, and a bunch of grapes 82 as the training subject are each displayed as a 3D object.

[0041] While viewing the field of view image displayed on the display device, the user moves the controller 2 (50) until the left hand in the virtual space is near the bunch of grapes. When the user presses the trigger button of the controller 2 (50), the operation detection unit 12 of the training system 1 detects this by a detection signal sent from the controller 2 (50) and displays an image of the left hand 80b holding the bunch of grapes.

[0042] In this state, the user rotates the controller 2 (50). In response, the operation detection unit 12 rotates the bunch of grapes in the virtual space based on the detection signal sent from the detection signal output unit 51 of the controller 2 (50).

[0043] The tuft rotation process will now be described. The operation detection unit 12 sets the axis value according to the angle of the controller 2 when the user grabs the tassel (see FIG. 11).

[0044] Step 1) The user presses the button on the left hand (controller 2 (50)) and saves the quaternions that represent the rotation of the tassel and the hand at the moment the tassel is grasped (tassel: Q_f1, hand: Q_h1). Step 2) Then, let the current hand quaternion after moving the left hand be Q_h2. Step 3) Calculate the rotation difference Q_h3 between the current time and the time when the gripping started. (Q_h3 = Q_h2 * Inverse(Q_h1)) Step 4) Q_h3 is multiplied by the correction for the angle of the tassel at the moment it is grasped to obtain Q_f2. (Q_f2 = Q_h3 * Q_f1) Step 5) Apply Q_f2 as the rotation of the current tuft.

[0045] The above is a process for rotating the tassel using the controller 2 (50), but similar processes can be used to rotate the tassel using other techniques such as hand gestures.

[0046] Each part belonging to the tassel is set as a relative coordinate with respect to the reference axis of the tassel, so that each part belonging to the tassel is displaced in the same manner as the axis of the tassel is displaced. This link between the controller 2 (50) and the tassel is released by a specified action of the user on the controller 2 (50) (for example, releasing or pressing the button again).

[0047] When the operation detection unit 12 detects the position and orientation (direction) of the controller 1 (50) in the right hand, it displays the right hand 80a and scissors 81 held in the right hand in the virtual space based on this position and direction information. Then, when the position of the scissors 81 in the virtual space comes close to a bunch of grapes 82, the bunch is identified and displayed to indicate that it has been selected as the task object.

[0048] (Decision making process) Hereinafter, the procedure of the determination process of the operation determination unit 72 regarding the grain picking operation will be described with reference to FIG. When a flag is set in the judgment column during the progress of the training scenario, the operation judgment unit 72 executes judgment processing corresponding to the task item. For example, in the case of a grain picking task, the following processing is executed.

[0049] The operation determination unit 72 determines whether or not the virtual object 1 set to the grape grain and the virtual object 2 set to the scissors 81 overlap (i.e., whether or not they collide) in the virtual space (S101). This overlap is determined based on the 3D coordinates in the virtual space sent from the operation detection unit 12. Then, the node in the hierarchy under the node where the collision was detected is determined as the cut target (S102), and the cut target is displayed in a different color via the 3D-CG display control command unit 71 (S103). For example, in FIG. 10, when the virtual object 1 set to the grain X in the bottom layer overlaps with the virtual object 2 of the scissors 1, only the grain X is displayed in a different color. If the virtual object 1 set to the car C01 overlaps with the virtual object 2 of the scissors 1, all parts (nodes) under the car C01 (C, K, T, M, X) are displayed in a different color.

[0050] When the user confirms that the color of the node that the user wants to pluck has changed, he or she presses the trigger button of the controller 1 (50). Then, the operation detection unit 12 detects the closing action of the scissors 81 as an action in the virtual space corresponding to the pressing of the trigger button of the controller 1 (50). This closing action of the scissors 81 is sent from the operation detection unit 12 to the operation determination unit 72 as operation information.

[0051] When the operation content determination unit 72 detects from this operation information that the scissors 81 have been closed (S104), it separates the object to be cut, which has been displayed in a different color in step S103, from the other nodes (S105). Specifically, a cutting flag (not shown) is set for the node in Fig. 10, and via the 3D-CG display control command unit 71, the image is switched to one in which both blades of the scissors 81 are closed, and an image is displayed in which the object to be cut is separated from the bunch of grapes and falls.

[0052] The operation determination unit 72 then uses the name of the object to be cut as a key to obtain a score from the data on the number of particles to be cut for each particle shown in Figures 13 and 14, and adds the score to the scoring result (S106). Details of this scoring process will be described later. On the other hand, if the closing operation of the scissors 81 is not detected within a certain period of time in step S104, the node is returned to its original color.

[0053] Then, in step S107, it is determined whether or not the pulping work has been completed, and if not, the process returns to step S101 and the subsequent steps are repeated. Note that whether or not the work has been completed can be determined arbitrarily, for example, by setting the next item in the training scenario displayed in the virtual space, or by ending the work after a certain period of time has elapsed.

[0054] When the task is completed, the scoring result is stored in the operation judgment storage unit 63 (S108). The task result can be output by the operation judgment output unit 73 to the display device of the HMD 10 or an output device (not shown) of the judgment device 60 (S109). Through the above process, the operation for the grain picking work is judged.

[0055] (Scoring and evaluation processing) In this embodiment, a plurality of tassel objects having a structure as shown in FIG. 14 are held. The tassel object name can be the name of the tassel as a 3D model. The tassel object has car objects in the form of an array. The car object has a character string type object name as a 3D model, an integer score assigned to the car, and a plurality of real objects in the form of an array. The real object has a character string type object name as a 3D model and an integer score. As shown in FIG. 13, for example, a positive score is assigned to a portion that should be cut (target for pruning), and a negative score is assigned to a portion that should be left (non-target for pruning). In this way, if the total score for the cut kernels is equal to or greater than a certain reference value, or the total score for the remaining kernels is equal to or less than a reference value, a judgment such as pass can be made. The above scoring can be applied to calculate the proficiency of the pruning work.

[0056] The preferred method of scoring is to assign a score to each grain based on the reason for removal and the location. The score should be the score that would be given if the target grain were removed. Reasons for removal include small grain / deformed / inward / protruding, etc. Locations include inside / outside and shoulder / body / tip (see Figures 15 and 13).

[0057] In this way, by storing the grain thinning proficiency in an external parameter file, the proficiency can be adjusted without modifying the program.

[0058] (Growth treatment after fruit thinning) The 3D-CG display control command unit 71 enlarges the size of the fruit after the berry thinning operation at a constant rate over time (see FIG. 16). By providing a simulation function for this growth process, the user can confirm the results of berry thinning and cluster thinning. In addition, the condition can be judged from the attributes of the kernels remaining after plucking. Specifically, the growth shape after plucking can be judged from the part and position. For example, the remaining kernel standard values ​​for the shoulder, body, and tip, and the balance of the outer / inner kernels for each part can be used as indicators.

[0059] In this way, by creating an external parameter file for the determination of the results of fruit thinning, the determination of the results of fruit thinning can be adjusted without modifying the program, making it possible to simply determine the growth pattern without using AI, etc.

[0060] (Compared to the craftsman's route; how to approach a certain layout) By saving the 6DoF sensor values ​​and operation logs, it is possible to reproduce the movements of a skilled worker (master craftsman) and intuitively learn the skilled work procedures by tracing the master's copper wire. The type of event such as granulation, the timing of the event, and the value of the six-axis sensor at that time are stored. The DoF sensor values ​​include the date and time of the event, the event content, the HMD position, the HMD rotation, the left hand position, the left hand rotation, the right hand position, and the right hand rotation. The operation log records the operation history of the controller 50 by the user when the training scenario is executed.

[0061] Using these data, the standing position, the rotation angle of the bunch, the insertion angle of the scissors 81, the rotation angle of the hand when picking the fruit, the order in which the fruit is picked, etc. can be used as indicators of the artisan techniques when picking the fruit. Also, the position when pruning, the movement line created from the time, the order of pruning, the shape confirmation (check position), etc. can be used as indicators of the artisan techniques when pruning.

[0062] The above processing makes it possible to construct an efficient training system for thinning grapes and bunches. In the training, bunches of different shapes may be prepared, and multiple connection points for bunches may be provided on the grape vine, and each bunch may be assigned to a connection point for bunches according to a predetermined rule such as randomly or according to the level of difficulty for each training.

[0063] As described above, according to this embodiment, it is possible to use VR technology to provide highly effective training in agricultural crop cultivation techniques, such as grape thinning. The present invention is not limited to this embodiment and can be realized in various modifications. For example, the present invention can be applied to any agricultural crop represented by a hierarchical structure of nodes, and it is clear that the present invention can be applied to fruit thinning of cherries, La France, apples, watermelons, and the like. In addition to fruit thinning, tasks such as bunch thinning and pruning can also be performed. For example, bunch thinning involves picking the bunches themselves, and a virtual object can be set in association with the bunch node.

[0064] In addition, in the above explanation, the operation information detected by the operation detection unit 12 is passed to the operation judgment unit 72, and the image of the work means in the virtual space is changed via the 3D-CG display control command unit 71 and the 3D-CG display unit 11. However, the image of the work means displayed in the virtual space of the display device may also be changed directly from the operation detection unit 12 or via the 3D-CG display unit 11. [Explanation of symbols]

[0065] 1. Training System 2. Communication lines 10 Virtual Content Display Device (HMD) 11 3D-CG display section 12 Operation detection unit 50 Operating equipment 51 Detection signal output section 60 Training scenario / operation record judgment device (judgment device) 61 3D model storage section 62 Training scenario memory section 63 Operation judgment storage unit 71 3D-CG display control command unit (three-dimensional display control unit) 72 Operation judgment section 73 Operation judgment output section 74 Statistical Analysis Department 75 Scenario Change Editorial Department 80 Hand Objects 80a Right Hand Object 80b Left Hand Object 81 Work means identification image 82 Training Objects

Claims

1. A training system for educating and training a training subject on a cultivation technique of a crop displayed in a virtual space of a display device, a training scenario storage unit that stores one or more work target portions for the agricultural crop in association with a work procedure, and stores a work means that is a tool to be used for the work target portions; a three-dimensional display control unit that causes the display device to display images of the crops and the work means in three dimensions as the work procedure progresses; a detection signal output unit that detects a position or movement of a user or an operating device and outputs a detection signal; an operation detection unit that changes an image of the working tool displayed in a virtual space based on the detection signal; an operation determination unit that determines a result of an operation based on coordinates of the work target portion in a virtual space, and on coordinates and operation content of the working means; the three-dimensional display control unit changes the image of the crop based on a determination result of the operation determination unit; The training system further comprises an operation judgment output unit that generates and outputs an image of the growth process after the operation for the crops that have changed based on the judgment result of the operation judgment unit.

2. the training scenario storage unit stores a judgment criterion for a task as an external parameter; 2. The training system according to claim 1, wherein the operation judgment output unit calculates a level of proficiency in the task based on the judgment criterion and a judgment result from the operation judgment unit.

3. The training system according to claim 1 or 2, characterized in that the agricultural crop is a crop that can be displayed using hierarchical data in which at least two of the following are connected in order as nodes: clusters, stems, cartilage, branches, fruits, calyxes, and grains.

4. 4. The training system according to claim 3, wherein the work target portion is any one of the nodes, and the working means is a cutting means having a blade.

5. a first virtual object for collision detection that is not displayed on the display device can be set to any of the nodes, and a second virtual object for collision detection that is not displayed on the display device can be set in the vicinity of a blade of the cutting means, 5. The training system according to claim 4, wherein when a collision between the first virtual object and the second virtual object is detected in the virtual space, the operation determination unit distinguishably displays nodes connected to the node at which the collision was detected toward the lower end, and further determines that the distinguished nodes have been disconnected at the timing when the disconnection means performs a predetermined operation.

6. The training system described in claim 5, characterized in that the three-dimensional display control unit displays the node determined to have been disconnected by the operation determination unit separately from other nodes that constitute the crop, or displays it not.

7. The training system described in claim 5, characterized in that the three-dimensional display control unit displays the growth of the crops by enlarging and displaying all or some of the nodes other than the node determined to have been cut by the operation determination unit over time.

8. A method for educating and training a training subject in cultivation techniques of a crop displayed in a virtual space of a display device, comprising: a process of storing one or more work target areas for the agricultural crop in association with a work procedure, and storing a work means which is a tool to be used for the work target areas; a three-dimensional display control process for displaying images of the crops and the working means in three dimensions on the display device as the work procedure progresses; a detection signal output process for detecting a position or movement of a user or an operating device and outputting a detection signal; and an operation detection process for changing an image of the working means displayed in a virtual space based on the detection signal. an operation determination process for determining a result of an operation based on coordinates of the work target portion in a virtual space, and on coordinates and operation content of the working means; a first virtual object for collision detection that is not displayed on the display device is set for the farm crop, and a second virtual object for collision detection that is not displayed on the display device is set for the work means, The operation determination process further includes: a first process of detecting a collision between the first virtual object and the second virtual object in a virtual space and thereby changing a part or an entire image of the farm crop; a second process of detecting an operation of the working means in a state in which a collision between the first virtual object and the second virtual object in a virtual space has been detected after the first process, and further changing the image portion changed by the first process; A training method comprising:

9. A program for providing education and training in cultivation techniques for a training target crop displayed in a virtual space, the program including a process for storing one or more work target areas for the crop displayed in the virtual space in association with a work procedure, and storing a work means which is a tool to be used for the work target areas; and a three-dimensional display control process for displaying images of the crop and the work means on a display device in three dimensions as the work procedure progresses. a detection signal output process for detecting a position or movement of a user or an operating device and outputting a detection signal; and an operation detection process for changing an image of the working means displayed in a virtual space based on the detection signal. an operation determination process for determining a result of an operation based on coordinates of the work target portion in a virtual space, and on coordinates and operation content of the working means; the three-dimensional display control process changes the image of the crops based on a determination result of the operation determination process; The computer-executable program further comprises an operation determination output process for generating and outputting an image of the growth process after the operation for the crops that have changed based on the determination result of the operation determination process.

Images