Model data determination method, model data determination device and model data determination program

Abstract

To provide a model data determination method, model data determination device, and model data determination program that can appropriately reflect movements of a performer in real space onto model data in virtual space using proximity sensors installed on a musical instrument.SOLUTION: A model data determination method is executed by a computer. In the model data determination method, detection results from proximity sensors 220 installed on a musical instrument are acquired. Further, based on the acquired detection results from the proximity sensors 220, a relative position between the musical instrument and at least a part of the performer's body is determined. Additionally, based on the determined relative positions, a posture of the performer's model data in the virtual space is determined.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a model data determination method, a model data determination device, and a model data determination program for determining model data. [Background technology]

[0002] A sensor is sometimes provided on a musical instrument, and the detection results of the sensor are used in performance. For example, Patent Document 1 describes a stringed instrument in which a detection sensor is attached to the instrument body. The detection sensor outputs a detection signal corresponding to the force applied to the instrument body. Based on the detection signal output from the detection sensor, the operation of an effect imparting unit that imparts an effect to the string vibration signal is controlled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-151642 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, ensembles have been held in virtual spaces using model data (avatars) of musicians. In a real-space ensemble, each musician plays in sync with not only the sounds played by fellow musicians but also their preparatory movements. Note that preparatory movements are movements that lead to the actual musical movements that produce sound, but are not movements that actually produce sound. A specific example of a preparatory movement is shaking the hands or head in time with the rhythm.

[0005] In order to realize a comfortable ensemble performance in a virtual space, it is necessary to appropriately reflect the movements of the performers, including preparatory movements, in the model data. However, it is difficult to appropriately reflect the movements of the performers in the model data using the technology disclosed in Patent Document 1.

[0006] An object of the present invention is to provide a model data determination method, a model data determination device, and a model data determination program that can appropriately reflect the movements of a performer in real space in model data in virtual space using a proximity sensor attached to a musical instrument. [Means for solving the problem]

[0007] A model data determination method according to one aspect of the present invention is executed by a computer, which acquires detection results from a proximity sensor provided on a musical instrument, determines a relative position between the musical instrument and at least a part of the player's body based on the acquired detection results from the proximity sensor, and determines a posture of the player's model data in a virtual space based on the determined relative position.

[0008] A model data determination device according to another aspect of the present invention includes a detection result acquisition unit that acquires detection results from a proximity sensor provided on a musical instrument, a relative position determination unit that determines the relative position between the musical instrument and at least a part of the player's body based on the detection results of the proximity sensor acquired by the detection result acquisition unit, and a posture determination unit that determines the posture of the player's model data in a virtual space based on the relative position determined by the relative position determination unit.

[0009] A model data determination program according to yet another aspect of the present invention causes a computer to execute the following processes: acquiring the detection results of a proximity sensor provided on a musical instrument; determining the relative position of the musical instrument and at least a part of the player's body based on the acquired detection results of the proximity sensor; and determining the posture of the player's model data in a virtual space based on the determined relative position. [Effects of the Invention]

[0010] According to the present invention, the movements of a player in real space can be appropriately reflected in model data in virtual space using a proximity sensor provided on a musical instrument. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a model data determination system including a model data determination device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating the configuration of a musical instrument. [Figure 3] FIG. 2 is a block diagram illustrating a configuration of a functional unit of the model data determination device. [Figure 4] FIG. 10 is a diagram for explaining a procedure for determining a relative attitude. [Figure 5] FIG. 10 is a diagram for explaining a procedure for determining the posture of the right arm. [Figure 6] FIG. 10 is a diagram for explaining a procedure for determining the posture of the right arm. [Figure 7] 10 is a flowchart illustrating an example of a model data determination process performed by the model data determination device. [Figure 8] 8 is a flowchart showing an example of a procedure for determining a relative attitude in the model data determination process of FIG. 7. [Figure 9] 8 is a flowchart showing an example of a procedure for determining an input value for the left arm in the model data determination process of FIG. 7. [Figure 10] 8 is a flowchart showing an example of a procedure for determining an input value for the right arm in the model data determination process of FIG. 7. [Figure 11] FIG. 10 is a block diagram illustrating the configuration of functional units of a model data determination device according to another embodiment. [Figure 12] FIG. 10 is a diagram showing the configuration of another musical instrument. [Figure 13] FIG. 10 is a diagram showing the configuration of another musical instrument. [Figure 14] FIG. 10 is a diagram showing the configuration of another musical instrument. DETAILED DESCRIPTION OF THE INVENTION

[0012] 1.Configuration of the model data determination system A model data determination method, a model data determination device, and a model data determination program according to an embodiment of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a model data determination system including a model data determination device according to an embodiment of the present invention. As shown in FIG. 1, a model data determination system 300 includes a model data determination device 100 and a musical instrument 200. Unique identification information is assigned to the musical instrument 200.

[0013] The model data determination device 100 includes a storage device 110, a CPU (Central Processing Unit) 120, a RAM (Random Access Memory) 130, a ROM (Read Only Memory) 140, a display device 150, an operation unit 160, a communication I / F (Interface) 170, and a bus 180. The model data determination device 100 may be realized by an information processing device such as a personal computer (PC), a cloud server, a smart device, or a tablet device. The model data determination device 100 may also be realized by two or more information processing devices. Therefore, some of the components of the model data determination device 100 may be realized by a cloud server or the like, and other components of the model data determination device 100 may be realized by a smart device or the like. The storage device 110, the CPU 120, the RAM 130, the ROM 140, the display device 150, the operation unit 160, and the communication I / F 170 are connected to the bus 180. A computer-readable storage medium 190, such as an SD memory card or a CD (Compact Disc)-ROM, can be connected to the communication I / F 170.

[0014] The storage device 110 includes a storage medium such as a hard disk, optical disk, magnetic disk, or memory card, and stores a model data determination program in advance. The CPU 120 executes the model data determination program stored in the storage device 110 to perform a model data determination process. Details of the model data determination process will be described later. The RAM 130 is, for example, a volatile memory, and is used as a work area for the CPU 120, and also temporarily stores various data. The ROM 140 is, for example, a non-volatile memory, and stores computer programs such as system programs.

[0015] The model data determination program may be stored in the ROM 140 instead of the storage device 110. The model data determination program may be provided in a form stored in the storage medium 190, and may be read out via the communication I / F 170 or the like to be installed in the storage device 110 or the ROM 140. Furthermore, if the communication I / F 170 is connected to a communication network, the model data determination program may be distributed from a server connected to the communication network and installed in the storage device 110 or the ROM 140. The server in question may include a cloud server.

[0016] The display device 150 includes, for example, a liquid crystal display. The display device 150 displays a screen for performing various settings or registrations. The display device 150 can also display model data of the player of the musical instrument 200 and model data of the musical instrument itself. The operation unit 160 includes a keyboard or a pointing device. The pointing device includes a mouse, a joystick, or the like. The operation unit 160 is operated by the user of the model data determination device 100 when giving a predetermined designation or instruction to the CPU 120, etc. Note that the user of the model data determination device 100 and the player of the musical instrument 200 may be the same person or different people. The display device 150 and the operation unit 160 may also be integrated into a touch panel device.

[0017] The communication I / F 170 is connected to the musical instrument 200. Specifically, a proximity sensor is provided at a predetermined location on the musical instrument 200. The communication I / F 170 is connected to the proximity sensor of the musical instrument 200. The connection between the communication I / F 170 and the musical instrument 200 may be a wired connection or a wireless connection. The communication I / F 170 is provided with the detection result by the proximity sensor. The detection result by the proximity sensor is used in the model data determination process.

[0018] 2. Musical Instruments FIG. 2 is a diagram showing the configuration of a musical instrument 200. As shown in FIG. 2, the musical instrument 200 includes a musical instrument body 210 and one or more proximity sensors 220. In this example, the musical instrument 200 is a guitar. Therefore, the musical instrument body 210 includes a body 211 and a neck 212. The left side of FIG. 2 shows the front side of the guitar, and the right side of FIG. 2 shows the back side of the guitar. The proximity sensor 220 is realized by, for example, multiple sensor elements arranged in an array or matrix. The sensor elements are, for example, capacitance sensor elements. The proximity sensor 220 detects the position or movement of a part of the player's body based on the distribution of detection results from the multiple sensor elements.

[0019] The detection result by the proximity sensor 220 is transmitted to the communication I / F 170. As a result, the detection result by the proximity sensor 220 is provided to the CPU 120. If multiple proximity sensors 220 are provided on the musical instrument body 210, the detection results by the multiple proximity sensors 220 may be combined and transmitted to the communication I / F 170. The proximity sensor 220 may also be realized by a capacitive touch panel in which multiple electrodes are arranged in a matrix. In this configuration, the position of a nearby object is detected at an intersection of any of the electrodes arranged in a matrix.

[0020] Some of the proximity sensors 220 are provided, for example, within the range of the instrument body 210 where the player's hands can play. Other proximity sensors 220 are provided, for example, in portions of the instrument body 210 that come into contact with parts of the instrument other than the player's hands. In this example, as shown on the left side of Fig. 2, proximity sensors 220 are provided on the front surface of the body 211 and on the fingerboard of the neck 212. Furthermore, as shown on the right side of Fig. 2, proximity sensors 220 are provided on the back surface of the body 211 and on the back surface of the neck 212.

[0021] The identification information assigned to the musical instrument 200 may be associated with the position of the proximity sensor 220. Furthermore, each proximity sensor 220 may be attached to or embedded in a corresponding portion of the musical instrument body 210. A dedicated electrode may be provided as the electrode of the sensor element of each proximity sensor 220, or a conductive part (e.g., a truss rod) that is already disposed as a structural, mechanical, or decorative part of the musical instrument 200 may be used. In this case, the proximity sensor 220 can be provided more compactly.

[0022] In the following description, when distinguishing between the multiple proximity sensors 220 provided in multiple parts of the musical instrument body 210, the proximity sensors 220 provided on the back surface of the neck 212, the front surface of the body 211, the fingerboard of the neck 212, and the back surface of the body 211 will be referred to as proximity sensors 221, 222, 223, and 224, respectively. Also, assuming that the player is right-handed, the hand (arm) that presses the strings will be referred to as the left hand (left arm), and the hand (arm) that plucks the strings will be referred to as the right hand (right arm). Therefore, if the player is left-handed, the left hand (left arm) can be read as the right hand (right arm), and the right hand (right arm) can be read as the left hand (left arm).

[0023] The proximity sensor 221 detects the position of the performer's left hand on the neck 212. Therefore, it is possible to detect not only the performer's performance action of gripping the neck 212 and pressing the strings, but also the preparatory performance action of releasing the fingers from the strings and moving the hand along the neck 212. The proximity sensor 222 detects the positions of the performer's right hand and right arm. Therefore, it is possible to detect the performer's preparatory performance actions, such as swinging down the right arm or swaying the right hand in time with the rhythm.

[0024] The proximity sensor 223 detects the position of the fingers of the player's left hand when pressing down on a string. It can also detect preparatory movements for playing, such as the degree of opening of the left hand when releasing a string. The proximity sensor 224 detects the relative position between the instrument body 210 and the player's torso. Specifically, the proximity sensor 224 detects the part of the player's torso that is in contact with or close to the back of the instrument body 210. Here, the relative position between the instrument body 210 and the torso can be associated with the way the neck 212 is held. Therefore, it can detect how the player holds the neck 212, such as whether the player is pushing the neck 212 back, pulling the neck 212 toward them, or holding the neck 212 horizontally.

[0025] 3. Configuration of the model data determination device Fig. 3 is a block diagram for explaining the configuration of the functional units of the model data determination device 100. As shown in Fig. 3, the model data determination device 100 includes, as functional unit 10, a detection result acquisition unit 11, a registration unit 12, an identification information acquisition unit 13, a position specification unit 14, a relative position determination unit 15, and an attitude determination unit 16. The functional unit 10 is realized by the CPU 120 in Fig. 1 executing a model data determination program stored in a storage device 110 or the like. Part or all of the functional unit 10 may be realized by hardware such as an electronic circuit.

[0026] The detection result acquisition unit 11 acquires the detection results of each proximity sensor 220 provided on the musical instrument 200. The registration unit 12 acquires the detection results of each proximity sensor 220 at a predetermined timing from the detection result acquisition unit 11 during the preparatory operation before the model data determination process is executed, and thereby registers relationship information indicating the relationship between the relative position of at least a part of the player's body with respect to the musical instrument 200 and the detection results of the proximity sensors 220.

[0027] In this example, when registering the relationship information, a predetermined sentence is displayed on the display device 150 of FIG. 1 , for example, to prompt the performer to sequentially perform predetermined performance actions and preparatory actions using the musical instrument 200. When any performance action or preparatory action is performed, the musical instrument 200 and a part of the performer's body are in a specific relative position. At this time, the relationship between the detection results obtained by each proximity sensor 220 and the relative positions is registered by the registration unit 12 as relationship information. The relative positions also include information such as the way the performer holds the musical instrument body 210 or the shape of the fingers. The registered relationship information may be stored in the storage device 110 or the like. Alternatively, the registered relationship information may be associated with the identification information of the musical instrument 200.

[0028] The identification information acquisition unit 13 acquires the identification information assigned to the musical instrument 200. The user may input the identification information of the musical instrument 200 to be acquired into the identification information acquisition unit 13 by operating the operation unit 160. Alternatively, if the musical instrument 200 is an electronic musical instrument and includes a microcomputer or the like in which the identification information of the musical instrument 200 is stored, the identification information acquisition unit 13 may acquire the identification information from the microcomputer or the like of the musical instrument 200.

[0029] The position identifying unit 14 identifies the position of the proximity sensor 220 provided on the musical instrument 200 based on the identification information acquired by the identification information acquiring unit 13. Note that in this example, the identification information includes the position of the proximity sensor 220, but the identification information does not necessarily include the position of the proximity sensor 220 directly as long as the identification information is associated with the position of the proximity sensor 220. For example, correspondence information in which the identification information is associated with the position of the proximity sensor 220 may be stored in the storage device 110 or the like. In this case, the position identifying unit 14 identifies the position of the proximity sensor 220 provided on the musical instrument 200 based on the identification information acquired by the identification information acquiring unit 13 and the correspondence information stored in the storage device 110 or the like.

[0030] During the model data determination process, the relative position determination unit 15 acquires the detection results of each proximity sensor 220 from the detection result acquisition unit 11. Furthermore, the relative position determination unit 15 determines the relative position between the musical instrument 200 and at least a part of the player's body based on the detection results of the proximity sensors 220, the relationship information stored in the storage device 110 or the like by the registration unit 12, and the positions of the proximity sensors 220 identified by the position identification unit 14.

[0031] The posture determination unit 16 determines the posture of the model data of the performer in the virtual space based on the relative position determined by the relative position determination unit 15. The posture of the performer model data to be determined includes the relative posture of the performer model data with respect to the model data of the musical instrument in the virtual space (hereinafter referred to as the relative posture), the posture of the left arm of the performer model data, and the posture of the right arm of the performer model data. In this example, the posture determination unit 16 determines the posture of the left arm and the posture of the right arm of the performer model data by performing calculations based on inverse kinematics on the relative positions determined by the relative position determination unit 15.

[0032] The following describes the operation of the model data determination device 100 for determining the relative posture, the posture of the left arm of the player's model data, and the posture of the right arm of the player's model data. By determining this information, it becomes possible to move the player's model data and the instrument's model data in the virtual space so as to reflect the movements of the player and the instrument 200 in the real space. Here, the posture of the left arm includes not only the posture of the left arm, but also the posture of the left shoulder, elbow, wrist, hand, and fingers. Similarly, the posture of the right arm includes not only the posture of the right arm, but also the posture of the right shoulder, elbow, wrist, hand, and fingers. Furthermore, in this example, it is also possible to move the instrument's model data without moving the player's model data.

[0033] 4. Determining the relative attitude Fig. 4 is a diagram illustrating a procedure for determining a relative posture. In Fig. 4, the left column illustrates the performer and the musical instrument 200 when the musical instrument body 210 and the performer's torso are in a first state. The center column illustrates the performer and the musical instrument 200 when the musical instrument body 210 and the performer's torso are in a second state. The right column illustrates the performer and the musical instrument 200 when the musical instrument body 210 and the performer's torso are in a third state.

[0034] 4, the performer and instrument 200 are shown in a plan view in the upper row. The performer and instrument 200 are shown in a front view in the middle row. Of the multiple sensor elements of the proximity sensor 224 provided on the back surface of the instrument body 210, the sensor element that detects the position or movement of a part of the performer's body (the torso in this example) is shown in the lower row as a black circle. When the proximity sensor 224 is configured using a capacitance sensor in which multiple electrodes are arranged in a matrix, the intersections of the electrodes correspond to the black circles. The sensor element that detects the position or movement of the performer's torso is also shown in the middle diagram.

[0035] 4, in a first state, the performer holds the instrument 200 so that the neck 212 is pushed back. In a second state, the performer holds the instrument 200 so that the neck 212 is pulled forward. In a third state, the performer holds the instrument 200 so that the neck 212 is horizontal. The relative position determination unit 15 determines the relative positions of the performer's torso and the instrument 200 in these states based on the detection results of the proximity sensor 224.

[0036] The posture determination unit 16 determines the position and angle of the instrument model data so that the relative position between the performer model data and the instrument model data in virtual space is the same as the relative position determined by the relative position determination unit 15. The position and angle can be determined using various methods. For example, the distribution of sensor elements that detect the position or movement of the performer's torso may be approximated to a straight line using the least squares method, and the position of the instrument model data may be determined so that the instrument model data is tangent to the performer's torso on this straight line. Alternatively, the angle of the instrument 200 may be determined as the angle of the instrument model data.

[0037] Here, the angles of the instrument model data include a first angle and a second angle. The first angle is the angle that the neck of the instrument model data forms with respect to a line connecting the shoulders of the performer model data in a planar view. The second angle is the angle that the neck of the instrument model data forms with respect to the horizontal plane. To make the angles easier to understand, the real-space angles θ1 and θ2 corresponding to the first and second angles, respectively, are shown in the left column of Figure 4.

[0038] 5. Determine the posture of the left arm The relative position determination unit 15 determines the position of the player's left wrist on the neck 212 based on the detection results of a proximity sensor 221 provided on the back surface of the neck 212. Furthermore, the relative position determination unit 15 estimates how the player's left hand grips the neck 212 based on the detection results of the proximity sensor 221, and determines the angle of the player's left wrist that corresponds to the estimated gripping method.

[0039] The relative position determination unit 15 determines the shape of the fingers of the player's left hand based on the detection results of the proximity sensor 223 provided on the fingerboard of the neck 212. The relative position determination unit 15 also presets the position where the player's left elbow should be when playing so that the player does not assume an unnatural playing posture. The relative position determination unit 15 determines the set position as the position of the player's left elbow.

[0040] The posture determination unit 16 performs calculations based on inverse kinematics using as input the left wrist position, left wrist angle, and left elbow position of the player determined by the relative position determination unit 15. As a result, the left wrist angle, left elbow angle, and left shoulder angle of the player model data are output. Note that the left wrist angle, left elbow angle, and left shoulder angle refer to the angle of the left wrist joint, the angle of the left elbow joint, and the angle of the left shoulder joint, respectively.

[0041] 6. Determine the posture of the right arm 5 and 6 are diagrams for explaining the procedure for determining the posture of the right arm. As shown in Fig. 5, the relative position determination unit 15 determines the position of the performer's right forearm based on the detection results of sensor elements in a predetermined right area of ​​the body 211 (the left area on the paper, i.e., the area indicated by the dotted line) of the body 211, out of the multiple sensor elements of the proximity sensor 222 provided on the surface of the body 211. This identifies a straight line L1 that runs along the performer's right forearm.

[0042] 6, the relative position determination unit 15 determines the position of the performer's right wrist based on the detection result from the proximity sensor 222. Specifically, the relative position determination unit 15 estimates that the position P1 of the performer's body part (the right arm in this example) detected by the proximity sensor 222 that is closest to the neck 212 is the position of the tip of the performer's right hand. Next, the relative position determination unit 15 determines that the position P2, which is a predetermined length L corresponding to the size of the right hand to the right of the estimated position of the tip of the right hand, is the position of the performer's right wrist.

[0043] Furthermore, the relative position determination unit 15 determines the angle of the performer's right wrist based on the position of the right forearm, the position of the tip of the right hand, and the position of the right wrist. Specifically, the angle formed by a line L2 passing through positions P1 and P2 and a line L1 is determined as the angle of the performer's right wrist. The relative position determination unit 15 also determines the position of the performer's right elbow as a position that is a predetermined length, corresponding to the length of the right forearm, in the direction of the right elbow from the determined position of the right wrist. The direction of the right elbow is estimated based on the determined angle of the right wrist.

[0044] The posture determination unit 16 performs calculations based on inverse kinematics using as input the position of the player's right wrist, the angle of the right wrist, and the position of the right elbow determined by the relative position determination unit 15. As a result, the angle of the right wrist, the angle of the right elbow, and the angle of the right shoulder of the player's model data are output. Note that the angle of the right wrist, the angle of the right elbow, and the angle of the right shoulder mean the angle of the right forearm joint, the angle of the right wrist joint, the angle of the right elbow joint, and the angle of the right shoulder joint, respectively.

[0045] 7.Model data determination process Fig. 7 is a flowchart showing an example of model data determination processing by the model data determination device 100. The model data determination processing of Fig. 7 will be described below using the model data determination device 100 of Fig. 3. During the preparatory operation before the model data determination processing of Fig. 7 is executed, relationship information indicating the relationship between the relative position of at least a part of the player's body with respect to the musical instrument 200 and the detection result of the proximity sensor 220 is registered by the registration unit 12.

[0046] First, the identification information acquisition unit 13 acquires the identification information of the musical instrument 200 (step S1). Next, the position identification unit 14 identifies the position of the proximity sensor 220 provided on the musical instrument 200 based on the identification information acquired in step S1 (step S2).

[0047] Next, the relative posture is determined (step S3). Furthermore, the input value of the performer's left arm is determined (step S4). Furthermore, the input value of the performer's right arm is determined (step S5). Details of steps S3 to S5 will be described later. Steps S3 to S5 may be performed one after the other, or may be performed simultaneously.

[0048] Thereafter, the posture determination unit 16 determines the posture of the left arm and the posture of the right arm of the model data of the player based on the input values ​​determined in steps S4 and S5 (step S6). Specifically, the posture of the left arm of the model data of the player is determined by performing a calculation based on inverse kinematics on the input value of the left arm of the player determined in step S4. Similarly, the posture of the right arm of the player is determined by performing a calculation based on inverse kinematics on the input value of the right arm of the player determined in step S5. Note that the shape of the fingers of the left hand of the player determined in the procedure for determining the input value of the left arm, which will be described later, is not used as an input value for the calculation based on inverse kinematics, but is used directly as the shape of the fingers of the left hand of the model data of the player.

[0049] Next, the posture determination unit 16 outputs the relative posture determined in steps S3 and S6, the posture of the left arm of the model data of the performer, and the posture of the right arm of the model data of the performer to an information processing device or the like that provides the virtual space (step S7). Thereafter, the process returns to step S3, and steps S3 to S7 are repeated. This allows the model data of the performer and the model data of the musical instrument in the virtual space to move so as to reflect in real time various movements of the performer in the real space, including preparatory movements for performance.

[0050] The speed at which steps S3 to S7 are repeated may be, for example, 60 times per second. In this case, the frame rate is 60 FPS. This makes it possible to achieve smooth movement of the model data. Alternatively, steps S3 to S7 may be repeated 30 times per second, and the frame rate may be set to 60 FPS by interpolating the movements before and after the repetition (double speed interpolation).

[0051] Fig. 8 is a flowchart showing an example of a procedure for determining a relative attitude in the model data determination process of Fig. 7. First, the detection result acquisition unit 11 acquires a detection result from the proximity sensor 224 (step S11). Next, the relative position determination unit 15 determines the relative position between the musical instrument body 210 and the trunk of the player based on the pre-registered relationship information, the position of the proximity sensor 224 identified in step S2, and the detection result of the proximity sensor 224 acquired in step S11 (step S12).

[0052] Next, the posture determination unit 16 determines the relative posture based on the relative position determined in step S12 (step S13). In this example, the position and angle of the instrument model data are determined so that the relative position between the performer model data and the instrument model data becomes the same as the relative position determined in step S13. This completes the procedure for determining the relative posture.

[0053] 9 is a flowchart showing an example of a procedure for determining an input value for the left arm in the model data determination process of FIG. 7. The detection result acquisition unit 11 acquires detection results from the proximity sensors 221, 223 (step S21). Next, the relative position determination unit 15 determines the position of the player's left wrist on the neck 212 based on the pre-registered relationship information, the position of the proximity sensor 221 identified in step S2, and the detection result of the proximity sensor 221 acquired in step S21 (step S22). Similarly, the relative position determination unit 15 determines the angle of the player's left wrist based on the relationship information, the position of the proximity sensor 221, and the detection result of the proximity sensor 221 (step S23).

[0054] In this example, the relationship information associates the detection result of the proximity sensor 221 with the way the player grips the neck 212 with their left hand. Therefore, the way the player grips the neck 212 with their left hand is estimated based on the detection result of the proximity sensor 221, based on the relationship information. The relationship information also associates the way the player grips the neck 212 with their left hand with the angle of their left wrist. Therefore, the angle of the player's left wrist is determined from the way the player grips the neck 212 with their left hand. However, if the relationship information associates the detection result of the proximity sensor 221 with the angle of the player's left wrist, the angle of the player's left wrist may be determined directly from the detection result of the proximity sensor 221.

[0055] Furthermore, the relative position determination unit 15 determines the shape of the fingers of the performer's left hand based on the relationship information, the position of the proximity sensor 223 identified in step S2, and the detection result of the proximity sensor 223 acquired in step S21 (step S24). Furthermore, the relative position determination unit 15 determines the position of the performer's left elbow based on a preset position (step S25). Steps S22 to S25 may be executed either first or simultaneously. By executing steps S22 to S25, the procedure for determining the input value of the left arm is completed.

[0056] Fig. 10 is a flowchart showing an example of a procedure for determining an input value for the right arm in the model data determination process of Fig. 7. The detection result acquisition unit 11 acquires a detection result from the proximity sensor 222 (step S31). Next, the relative position determination unit 15 determines the position of the player's right forearm based on the position of the proximity sensor 222 identified in step S3 and the detection result of the proximity sensor 222 acquired in step S31 (step S32).

[0057] Furthermore, the relative position determination unit 15 estimates the position of the tip of the performer's right hand based on the position of the proximity sensor 222 and the detection result of the proximity sensor 222 (step S33). Subsequently, the relative position determination unit 15 determines the position of the performer's right wrist based on the position of the tip of the performer's right hand estimated in step S33 (step S34). Step S32 and steps S33 and S34 may be executed first, or may be executed simultaneously.

[0058] Thereafter, the relative position determination unit 15 determines the angle of the performer's right wrist based on the position of the right forearm determined in step S32, the position of the tip of the right hand estimated in step S33, and the position of the right wrist determined in step S34 (step S35). Next, the relative position determination unit 15 determines the position of the performer's right elbow based on the position of the right wrist determined in step S34 and the angle of the right wrist determined in step S35 (step S36). This completes the procedure for determining the input value for the right arm.

[0059] 8. Effects of the embodiment As described above, the model data determination device 100 according to this embodiment includes a detection result acquisition unit 11 that acquires the detection results of the proximity sensor 220 provided on the musical instrument 200, a relative position determination unit 15 that determines the relative position between the musical instrument 200 and at least a part of the player's body based on the detection results of the proximity sensor 220 acquired by the detection result acquisition unit 11, and a posture determination unit 16 that determines the posture of the model data of the player in the virtual space based on the relative position determined by the relative position determination unit 15.

[0060] In this model data determination device 100, a posture including preparatory movements for performance of the model data of the performer in the virtual space is determined in response to the performance movements of the performer in the real space using a proximity sensor 220 provided on the musical instrument 200. This allows the movements of the performer in the real space to be appropriately reflected in the model data in the virtual space.

[0061] The posture of the model data of the performer determined by the posture determination unit 16 may include a relative posture of the model data of the performer with respect to the model data of the musical instrument in the virtual space. In this case, the motion of the musical instrument 200, such as a performance operation, can be appropriately reflected in the model data of the performer. Alternatively, the movement of the musical instrument 200 relative to the performer can be appropriately reflected in the model data of the musical instrument.

[0062] The model data determination device 100 may further include a registration unit 12 that registers the relationship between the detection result of the proximity sensor 220 and the relative position, and the relative position determination unit 15 may determine the relative position between the musical instrument 200 and at least a part of the player's body based on the detection result of the proximity sensor 220 acquired by the detection result acquisition unit 11 and the relationship registered in the registration unit 12. In this case, the relative position between the musical instrument 200 and at least a part of the player's body can be easily determined based on the registered relationship.

[0063] The posture determination unit 16 may determine the posture of the model data of the player by performing calculations based on inverse kinematics on the relative position determined by the relative position determination unit 15. In this case, the posture of the model data of the player can be easily determined based on inverse kinematics from the relative position between the musical instrument 200 and at least a part of the player's body.

[0064] The model data determination device 100 further includes an identification information acquisition unit 13 that acquires unique identification information assigned to the musical instrument 200, the identification information being associated with the position of a proximity sensor 220 provided on the musical instrument 200, and a position identification unit 14 that identifies the position of the proximity sensor 220 provided on the musical instrument 200 based on the identification information acquired by the identification information acquisition unit 13, and the relative position determination unit 15 may determine the relative position of the musical instrument 200 and at least a part of the player's body further based on the position of the proximity sensor 220 identified by the position identification unit 14. In this case, it is possible to easily determine the posture of model data of a player that differs depending on the musical instrument 200.

[0065] The proximity sensor 220 may be provided within the range of the performer's hands on the musical instrument 200. In this case, the posture of the performer's model data corresponding to the performance operation or other actions on the musical instrument 200 can be easily determined.

[0066] The proximity sensor 220 may be provided at a portion of the musical instrument 200 that comes into contact with a part of the instrument other than the player's hand. In this case, it is possible to easily determine the posture of the model data of the player that corresponds to the preparatory movements for performance before and after the performance or during an interlude.

[0067] 9. Variations If the model data determination device 100 is configured to be able to acquire sounds played by a musical instrument, the posture determination unit 16 may determine the posture of the model data of the player further based on the volume, timbre, or pitch. That is, the model data determination device 100 may further include a performance sound acquisition unit that acquires sounds played by the musical instrument 200, and the posture determination unit 16 may determine the posture of the model data of the player further based on the volume, timbre, or pitch of the performance sounds acquired by the performance sound acquisition unit. With this configuration, it is possible to more accurately determine the posture of the model data of the player, which varies depending on the playing behavior, such as playing the musical instrument 200 vigorously when the volume of the performance sounds is high, or playing a high position on the guitar when the pitch of the performance sounds is high.

[0068] Furthermore, the posture determination unit 16 may determine the posture of the model data of the performer from among a plurality of postures (including the movement of the performer or the musical instrument 200) prepared in advance, based on the calculation results based on inverse kinematics. That is, the posture determination unit 16 may perform a calculation based on inverse kinematics on the relative position determined by the relative position determination unit 15, and determine the posture of the model data of the performer from among a plurality of postures prepared in advance, based on the calculation results. In this case, the posture of the model data of the performer can be more reliably determined from the relative position between the musical instrument 200 and at least a part of the performer's body, based on inverse kinematics.

[0069] Alternatively, the posture determination unit 16 may determine the posture of the model data of the player from among a plurality of postures prepared in advance, based on the relative position determined by the relative position determination unit 15. In this case, the posture determination unit 16 does not need to perform calculations based on inverse kinematics for the relative position determined by the relative position determination unit 15. This makes it possible to determine the posture of the model data of the player through simpler processing.

[0070] 10. Other Embodiments (1) In the above embodiment, the musical instrument 200 is provided with proximity sensors 221-224, but the embodiment is not limited to this. Some of the proximity sensors 220 among the proximity sensors 221-224 may not be provided on the musical instrument 200. Alternatively, instead of some of the proximity sensors 220, an existing sensor or an IMU (inertial measurement unit) provided on the musical instrument 200 may be used to determine the relative position between the musical instrument 200 and at least a part of the player's body.

[0071] 8, in step S11, the detection result may be acquired from an IMU instead of the proximity sensor 224. In this case, in step S12, the relative position between the musical instrument body 210 and the torso of the player is determined based on the detection result from the IMU. In this case, the musical instrument 200 does not need to be provided with the proximity sensor 224.

[0072] Furthermore, in this example, the relationship information associates proximity sensor 223 with the shape of the fingers of the performer's left hand, but the embodiment is not limited to this. The relationship information may also associate the detection result of proximity sensor 221 with the shape of the fingers of the performer's left hand. In this case, in step S24 of the procedure for determining the input value of the left arm in FIG. 9, the shape of the fingers of the performer's left hand is determined based on proximity sensor 221 rather than proximity sensor 223.

[0073] Alternatively, if the model data determination device 100 has a performance sound acquisition unit that acquires performance sounds from a musical instrument, a performance operation (such as a chord) may be identified from the performance sound of the musical instrument 200, and the shape of the fingers of the player's left hand may be determined from the performance operation. In the procedure of identifying the performance operation from the performance sound, a trained model that indicates the input / output relationship between the performance sound and the performance operation may be used. Furthermore, the shape of the fingers of the player's left hand may be determined in advance or may be specified by the user. In these cases, the musical instrument 200 does not need to be provided with a proximity sensor 223.

[0074] Furthermore, in the model data determination process, the relative posture, the left arm input value, and the right arm input value are all determined, but the embodiment is not limited to this. In the model data determination process, it is sufficient that at least one of the relative posture, the left arm input value, and the right arm input value is determined. For example, if the relative posture is not determined, the musical instrument 200 does not need to be provided with the proximity sensor 224. In this configuration, step S3 of the model data determination process is omitted. If the relative posture is not determined, a predetermined position or a position specified by the user in the virtual space may be treated as the position of the musical instrument model.

[0075] Similarly, if the input value of the left arm is not determined, the proximity sensors 221, 223 may not be provided in the musical instrument 200. In this configuration, step S4 of the model data determination process is omitted. If the input value of the right arm is not determined, the proximity sensor 222 may not be provided in the musical instrument 200. In this configuration, step S5 of the model data determination process is omitted.

[0076] (2) In the above embodiment, the relative position determination unit 15 determines the relative position and the input value of the left arm from the detection result of the proximity sensor 220 using the relationship information registered by the registration unit 12, but the embodiment is not limited to this. As with the procedure for determining the input value of the right arm, the relative position determination unit 15 may determine the relative position between the musical instrument 200 and at least a part of the player's body by performing a calculation on the detection result of the proximity sensor 220 without using the relationship information.

[0077] For example, it is possible to estimate the proximity between the instrument 200 and the player's body from the maximum detection value of the proximity sensor 220. Therefore, it is possible to detect an action such as the player removing the instrument 200 from their torso and picking it up again, and to reflect this action in the model data of the player and the model data of the instrument.

[0078] When the relative position determination unit 15 determines the relative position, the input value of the left arm, and the input value of the right arm without using the relationship information, the model data determination device 100 does not need to include the registration unit 12. On the other hand, when the relationship information associates the detection result of the proximity sensor 222 with the position of the right forearm, the position of the tip of the right hand, the position of the right wrist, the angle of the right wrist, or the position of the right elbow of the player, the relative position determination unit 15 may determine the input value of the right arm using the relationship information.

[0079] Alternatively, the relative position determination unit 15 may determine the relative position between the musical instrument 200 and at least a part of the player's body from the proximity sensor 220 by another method. Fig. 11 is a block diagram for explaining the configuration of the functional units of a model data determination device 100 according to another embodiment. Below, the differences between the model data determination device 100 according to this embodiment and the model data determination device 100 of Fig. 3 will be explained.

[0080] As shown in FIG. 11 , the functional unit 10 of the model data determination device 100 according to this embodiment includes a model acquisition unit 17 instead of the registration unit 12. The model acquisition unit 17 acquires a trained model indicating an input / output relationship between the detection results of the proximity sensor 220 and the relative position. The relative position determination unit 15 determines the relative position between the musical instrument 200 and at least a part of the player's body based on the detection results of the proximity sensor 220 acquired by the detection result acquisition unit 11 and the trained model acquired by the model acquisition unit 17. With this configuration, the relative position between the musical instrument 200 and at least a part of the player's body can be easily determined based on the trained model.

[0081] (3) In the above embodiment, the model data determination device 100 includes the identification information acquisition unit 13 and the position identification unit 14, but the embodiment is not limited to this. In the model data determination system 300, if the musical instrument 200 to be used is predetermined, the model data determination device 100 does not need to include the identification information acquisition unit 13. In this case, step S1 of the model data determination process is omitted. Also, if the position of the proximity sensor 220 provided on the musical instrument 200 is predetermined, the model data determination device 100 does not need to include the position identification unit 14. In this case, step S2 of the model data determination process is omitted.

[0082] Furthermore, the instrument 200 does not have to be a guitar. Figures 12, 13, and 14 are diagrams showing the configuration of other instruments 200. In the example of Figure 12, the instrument 200 is a shoulder keyboard (keyta). The left side of Figure 12 shows a performer playing the instrument 200, the center of Figure 12 shows the front side of the instrument 200, and the right side of Figure 12 shows the back side of the instrument 200. As shown on the left side of Figure 12, the instrument main body 210 includes a body 211 and a handle 213. The body 211 comes into contact with the torso of the performer. The handle 213 protrudes upward from the top of the body 211 and is held in the performer's left hand.

[0083] As shown in the center of Fig. 12, a first proximity sensor 220 is provided on the part of body 211 in front of the keyboard as seen by the player (the part corresponding to the plectrum of a piano). A second proximity sensor 220 is provided on the side of handle 213. As shown on the right of Fig. 12, a third proximity sensor 220 is provided on the back of handle 213. A fourth proximity sensor 220 is provided on the back of body 211.

[0084] The first proximity sensor 220 detects the position of the performer's right wrist. The second proximity sensor 220 and the third proximity sensor 220 detect the part of the instrument that the performer's left hand is in contact with. The handle portion 213 is also provided with controls such as buttons and a pitch bend knob. Therefore, the second proximity sensor 220 and the third proximity sensor 220 can also detect the operation of the controls. The fourth proximity sensor 220 detects the performer's torso.

[0085] In the example of Fig. 13, the musical instrument 200 is a double bass (contrabass). The leftmost part of Fig. 13 shows a performer playing a position other than the high position of the musical instrument 200. The second from the left in Fig. 13 shows a performer playing a position high on the musical instrument 200. The second from the right in Fig. 13 shows the front of the musical instrument 200, and the rightmost part of Fig. 13 shows the back of the musical instrument 200. As shown in the leftmost and second from the left in Fig. 13, the musical instrument main body 210 includes a body 211 and a neck 212.

[0086] As shown second from the right in Fig. 13, a first proximity sensor 220 is provided from the right side of the front surface of body 211 to the side. A second proximity sensor 220 is provided from the left side of the front surface of body 211 to the side. As shown on the far right in Fig. 13, a third proximity sensor 220 is provided on the back surface of neck 212. A fourth proximity sensor 220 is provided on the back surface of body 211.

[0087] The first proximity sensor 220 detects the performer's right arm. The second proximity sensor 220 detects the performer's left arm when playing high positions. The third proximity sensor 220 detects the performer's left arm when not playing high positions. The fourth proximity sensor 220 detects the performer's torso.

[0088] In the example of FIG. 14 , the musical instrument 200 is a cajon. The left side of FIG. 14 shows a player playing the musical instrument 200, and the right side of FIG. 14 shows a perspective view of the musical instrument 200. As shown in the left side of FIG. 14 , the musical instrument body 210 has a rectangular parallelepiped shape. The player plays the musical instrument 200 by striking the front of the instrument 200 with both hands while sitting on the seat, which is the upper surface of the musical instrument 200. As shown in the right side of FIG. 14 , a first proximity sensor 220 is provided on the front of the musical instrument body 210. A second proximity sensor 220 is provided on the seat of the musical instrument body 210.

[0089] The first proximity sensor 220 detects the left and right hands of the player. Specifically, the first proximity sensor 220 detects the movement of both hands striking the front of the musical instrument 200 from a position relatively close (e.g., 20 cm) from the front of the musical instrument body 210. The second proximity sensor 220 detects the player's buttocks and thighs. The second proximity sensor 220 can also detect movements such as the player swaying their body back and forth or side to side based on changes in the distribution of detected areas.

[0090] Furthermore, when the performer makes rhythmic movements such as raising and lowering their heels, a gap occurs between the right or left thigh and the seat of the musical instrument body 210. The second proximity sensor 220 can detect this gap based on changes in detection intensity or distribution of detected parts. Therefore, the second proximity sensor 220 can also detect the performer's movements of keeping rhythm using their legs.

[0091] (4) In the above embodiment, the model data determination device 100 is provided outside the musical instrument 200, but the embodiment is not limited to this. The model data determination device 100 may be provided inside the musical instrument 200 as a part of the musical instrument 200.

[0092] 11. Summary of the embodiment (1) The method for determining model data according to paragraph 1 is as follows: Obtaining the detection result of a proximity sensor provided on the musical instrument; determining a relative position between the musical instrument and at least a part of the player's body based on the obtained detection result of the proximity sensor; determining a posture of the model data of the player in a virtual space based on the determined relative position; It is executed by a computer.

[0093] According to this model data determination method, a proximity sensor attached to the musical instrument is used to determine the posture, including the preparatory movements, of the model data of the performer in the virtual space in response to the performance movements of the performer in the real space, thereby allowing the movements of the performer in the real space to be appropriately reflected in the model data in the virtual space.

[0094] (Item 2) In the model data determination method described in item 1, The determined posture of the model data of the player may include a relative posture of the model data of the player with respect to the model data of the musical instrument in the virtual space.

[0095] In this case, the player's actions, such as playing operations on the instrument, can be appropriately reflected in the model data of the player, or the movement of the instrument relative to the player can be appropriately reflected in the model data of the instrument.

[0096] (Item 3) The model data determination method according to item 1 or 2 includes: further registering the relationship between the detection result of the proximity sensor and the relative position; Determining the relative position may include determining the relative position between the musical instrument and at least a part of the player's body based on the obtained detection results of the proximity sensor and the registered relationship.

[0097] In this case, the relative position between the musical instrument and at least a part of the player's body can be easily determined based on the registered relationship.

[0098] (4) The model data determination method according to any one of paragraphs 1 to 3, Further, a trained model indicating an input / output relationship between the detection result of the proximity sensor and the relative position is acquired; The determining of the relative position may include determining the relative position between the musical instrument and at least a part of the player's body based on the obtained detection results of the proximity sensor and the obtained trained model.

[0099] In this case, the relative position between the musical instrument and at least a part of the player's body can be easily determined based on the trained model.

[0100] (Item 5) In the model data determination method according to any one of items 1 to 4, Determining the posture may include determining the posture of the model data of the player by performing a calculation based on inverse kinematics on the determined relative position.

[0101] In this case, the posture of the model data of the player can be easily determined based on the relative position between the musical instrument and at least a part of the player's body, based on inverse kinematics.

[0102] (Item 6) In the model data determination method according to any one of items 1 to 5, Determining the posture may include determining the posture of the model data of the player from among a plurality of postures prepared in advance, based on the determined relative position.

[0103] In this case, the posture of the model data of the player can be determined through simpler processing.

[0104] (Item 7) In the model data determination method according to any one of items 1 to 6, Determining the posture may include performing a calculation based on inverse kinematics on the determined relative position, and determining the posture of the model data of the player from a plurality of postures prepared in advance based on the calculation results.

[0105] In this case, the posture of the model data of the player can be determined more reliably based on the relative position between the musical instrument and at least a part of the player's body, based on inverse kinematics.

[0106] (Item 8) The model data determination method according to any one of items 1 to 7, further acquiring unique identification information assigned to the musical instrument, the identification information being associated with the position of the proximity sensor provided on the musical instrument; Further determining the location of the proximity sensor provided on the musical instrument based on the acquired identification information; Determining the relative position may include determining the relative position of the musical instrument and at least a portion of the player's body further based on the identified position of the proximity sensor.

[0107] In this case, it is possible to easily determine the posture of the model data of the player, which differs depending on the instrument.

[0108] (Item 9) In the model data determination method according to any one of items 1 to 8, The proximity sensor may be provided in a range of the instrument that the player can manipulate with his or her hands.

[0109] In this case, the posture of the model data of the performer corresponding to the action of playing the instrument can be easily determined.

[0110] (Item 10) In the model data determination method according to any one of items 1 to 9, The proximity sensor may be provided at a portion of the musical instrument that comes into contact with a part of the instrument other than the player's hand.

[0111] In this case, the posture of the model data of the performer corresponding to the preparatory movements for performance before and after the performance or during the interlude can be easily determined.

[0112] (Item 11) The model data determination device according to item 11 comprises: a detection result acquisition unit that acquires a detection result of a proximity sensor provided on the musical instrument; a relative position determination unit that determines a relative position between the musical instrument and at least a part of a player's body based on the detection result of the proximity sensor acquired by the detection result acquisition unit; and a posture determination unit that determines a posture of the model data of the player in the virtual space based on the relative position determined by the relative position determination unit.

[0113] In this model data determination device, a proximity sensor provided on the musical instrument is used to determine the posture, including the preparatory movements, of the model data of the performer in the virtual space in response to the performance movements of the performer in the real space, thereby allowing the movements of the performer in the real space to be appropriately reflected in the model data in the virtual space.

[0114] (12) The model data determination program according to the 12th paragraph is A process of acquiring a detection result of a proximity sensor provided on the musical instrument; determining a relative position between the musical instrument and at least a part of the player's body based on the obtained detection result of the proximity sensor; and determining a posture of the model data of the player in a virtual space based on the determined relative position. Have the computer run it.

[0115] According to this model data determination program, a proximity sensor provided on the musical instrument is used to determine the posture, including the preparatory movements, of the model data of the performer in the virtual space in response to the performance movements of the performer in the real space, thereby allowing the movements of the performer in the real space to be appropriately reflected in the model data in the virtual space. [Explanation of symbols]

[0116] 10...functional unit, 11...detection result acquisition unit, 12...registration unit, 13...identification information acquisition unit, 14...position identification unit, 15...relative position determination unit, 16...posture determination unit, 17...model acquisition unit, 100...model data determination device, 110...storage device, 120...CPU, 130...RAM, 140...ROM, 150...display device, 160...operation unit, 170...communication I / F, 180...bus, 190...storage medium, 200...musical instrument, 210...musical instrument main body, 211...body, 212...neck, 213...handle portion, 220-224...proximity sensor, 300...model data determination system, L1, L2...straight line, P1, P2...position

Claims

1. Obtaining the detection result of a proximity sensor provided on the musical instrument; determining a relative position between the musical instrument and at least a part of the player's body based on the obtained detection result of the proximity sensor; determining a posture of the model data of the player in a virtual space based on the determined relative position; A computer-implemented method for determining model data.

2. 2. The model data determination method according to claim 1, wherein the determined posture of the model data of the performer includes a relative posture of the model data of the performer with respect to the model data of the musical instrument in the virtual space.

3. further registering the relationship between the detection result of the proximity sensor and the relative position; 3. The model data determination method according to claim 1, wherein determining the relative position includes determining the relative position between the musical instrument and at least a part of the player's body based on the obtained detection result of the proximity sensor and a registered relationship.

4. Further, a trained model indicating an input / output relationship between the detection result of the proximity sensor and the relative position is acquired; 3. The model data determination method according to claim 1, wherein determining the relative position includes determining the relative position between the musical instrument and at least a part of the player's body based on the obtained detection results of the proximity sensor and the obtained trained model.

5. 3. The model data determination method according to claim 1, wherein determining the posture includes determining the posture of the model data of the player by performing a calculation based on inverse kinematics on the determined relative position.

6. 3. The model data determination method according to claim 1, wherein determining the posture includes determining a posture of the model data of the player from a plurality of postures prepared in advance based on the determined relative position.

7. 3. The model data determination method according to claim 1, wherein determining the posture includes performing a calculation based on inverse kinematics on the determined relative position, and determining a posture of the model data of the player from a plurality of postures prepared in advance based on the calculation result.

8. further acquiring unique identification information assigned to the musical instrument, the identification information being associated with the position of the proximity sensor provided on the musical instrument; Further determining the location of the proximity sensor provided on the musical instrument based on the acquired identification information; 3. The model data determination method according to claim 1, wherein determining the relative position includes determining the relative position between the musical instrument and at least a part of the player's body further based on the identified position of the proximity sensor.

9. 3. The model data determination method according to claim 1, wherein the proximity sensor is provided within a range of the instrument that can be manipulated by the player's hands.

10. 3. The model data determination method according to claim 1, wherein the proximity sensor is provided at a portion of the musical instrument that comes into contact with a part of the instrument other than the player's hand.

11. a detection result acquisition unit that acquires a detection result of a proximity sensor provided on the musical instrument; a relative position determination unit that determines a relative position between the musical instrument and at least a part of a player's body based on the detection result of the proximity sensor acquired by the detection result acquisition unit; a posture determination unit that determines a posture of the model data of the player in a virtual space based on the relative position determined by the relative position determination unit.

12. A process of acquiring a detection result of a proximity sensor provided on the musical instrument; determining a relative position between the musical instrument and at least a part of the player's body based on the obtained detection result of the proximity sensor; and determining a posture of the model data of the player in a virtual space based on the determined relative position. A model data determination program executed by a computer.

Images