Object posture control program and object posture control system

The object posture control program simplifies the process of controlling an object's posture in a virtual space by allowing users to interactively set and move target parts, improving user interaction in virtual environments.

JP2024164219AActive Publication Date: 2024-11-26MIXI INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2024145825
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-26
Estimated Expiration
2039-03-25

AI Technical Summary

Technical Problem

Controlling the posture of an object displayed in a virtual space requires complicated input processing when specifying a target region.

Method used

An information processing device displays an object with interlocking parts in a virtual space and uses an object posture control program to control the posture of the object by setting and moving target parts based on user input, such as touch operations, to align and adjust the object's position and orientation.

Benefits of technology

Enables easy and intuitive control of the object's posture by allowing users to manipulate interlocking parts through simple interactions, enhancing user experience in virtual environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164219000001_ABST
    Figure 2024164219000001_ABST
Patent Text Reader

Abstract

To allow for easily controlling posture of an object.SOLUTION: An object posture control program makes an information processing device perform steps of displaying an object with multiple interlocking parts placed in a virtual space on a display unit, and controlling posture of the object to move a target part of the multiple interlocking parts according to information on movement of the information processing device. With the object posture control program, the information processing device may include the display unit.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an object attitude control program and an object attitude control system. [Background technology]

[0002] In recent years, the development of electronic devices has made it possible to display high-definition still images and videos on terminals such as smartphones and personal computers. Furthermore, research and development of information processing technology has progressed, and display technology that combines virtual space and real space using augmented reality (AR) is becoming practical. For example, Patent Document 1 discloses a technology for displaying objects arranged in a virtual space in the display area of ​​an information terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-41126 A Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, controlling the posture of an object displayed in a display area may require complicated input processing in order to specify a target part.

[0005] In view of the above problems, one object of the present invention is to easily control the attitude of an object placed in a virtual space. [Means for solving the problem]

[0006] According to one embodiment of the present invention, an object posture control program is provided for causing an information processing device to display on a display unit an object having multiple interlocking parts arranged in a virtual space, and to control the posture of the object so as to move a target part of the multiple interlocking parts based on information relating to the movement of the information processing device.

[0007] In the above object posture control program, the information processing device may have a setting unit, and the setting unit may set a target part of the multiple interlocking parts based on information input when a part of the object displayed on the display unit is touch-operated.

[0008] In the above object posture control program, the setting unit may set a first target part of the object which is a part of the plurality of interlocking parts and is capable of moving the object, and a second target part of the plurality of interlocking parts may be set by the setting unit based on the set first target part, and the first target part may move based on the second target part.

[0009] In the above object posture control program, when the first target part reaches the limit of a movable range, a third target part is set on the opposite side of the first target part relative to the second target part, connecting to the second target part, and the second target part of the object may be moved based on the third target part.

[0010] In the above object posture control program, when a portion where multiple parts overlap is specified when setting the first target part or the second target part in the object displayed on the display unit, the direction in which the object is displayed may be changed.

[0011] In the above-mentioned object posture control program, when a portion where multiple parts overlap is specified in the object displayed on the display unit when setting the first target part or the second target part, a part that satisfies a predetermined condition may be set as the first target part or the second target part.

[0012] In the above-mentioned object posture control program, when a portion where multiple parts overlap is specified in the object displayed on the display unit when setting the first target portion or the second target portion, the first target portion or the second target portion may be set based on a predetermined operation on the overlapping portion.

[0013] In the above object posture control program, a movable fourth target part of the object different from the first target part and the second target part is set, a fifth target part connected to the fourth target part is set, and based on information regarding the movement of the information processing device, the first target part of the object moves based on the second target part and the fourth target part moves based on the fifth target part.

[0014] In the above object posture control program, when the first target part of the object moves and comes into contact with a movable part different from the first target part and the second target part, it may be set as a fourth target part, a fifth target part connected to the fourth target part may be set, and the fourth target part may move based on the movement of the first target part and with the fifth target part as a reference.

[0015] In the above object posture control program, when the information processing device is moved while the first target part or the second target part is touched, the first target part may move based on the second target part of the object in the virtual space while the displayed object is stationary.

[0016] According to one embodiment of the present invention, there is provided an object posture control system for controlling the posture of an object placed in a virtual space, which displays an object having a plurality of interlocking parts on a display unit, and controls the posture of the object so as to move some of the plurality of interlocking parts based on information relating to the movement of an information processing device.

[0017] According to one embodiment of the present invention, there is provided an information processing device that controls the attitude of an object placed in a virtual space, which displays the object having a plurality of interlocking parts on a display unit, and controls the attitude of the object so as to move a target part of the plurality of interlocking parts based on information regarding the movement of the information processing device. Effect of the Invention

[0018] Using an embodiment of the present invention, the pose of an object can be easily controlled. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram showing a hardware configuration of an object attitude control system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a functional block diagram of an object posture control unit according to the first embodiment of the present invention. [Diagram 3] FIG. 4 is a flowchart of an object attitude control process according to the first embodiment of the present invention. [Figure 4] FIG. 1 is a flowchart of a first object attitude control process according to the first embodiment of the present invention. [Diagram 5] 13 is an example of alignment between coordinates in a virtual space and coordinates displayed on an information processing device in the first object attitude control processing according to the first embodiment of the present invention. [Figure 6] 4 is an example of coordinates of each part of an object according to the first embodiment of the present invention. [Figure 7]13 is an example of a user interface displayed on a display unit of the information processing device in the first object attitude control process according to the first embodiment of the present invention. [Figure 8] FIG. 11 is a flowchart of a second object attitude control process according to the first embodiment of the present invention. [Figure 9] 13 is an example of a user interface displayed on a display unit of the information processing device in the second object attitude control process according to the first embodiment of the present invention. [Figure 10] 11 is an example of a user interface displayed on a display unit of the information processing device in the second object attitude control process according to the first embodiment of the present invention. [Figure 11] FIG. 11 is a flowchart of a third object attitude control process according to the first embodiment of the present invention. [Figure 12] 13 is an example of a case where the position of the information processing device is changed in the third object attitude control process according to the first embodiment of the present invention. [Figure 13] 13 shows position coordinates of a terminal before and after movement and position coordinates of an object in a third object attitude control process according to the first embodiment of the present invention. [Figure 14] 11 is an example of a user interface displayed on a display unit of the information processing device in the third object attitude control process according to the first embodiment of the present invention. [Figure 15] 13 is an example of a user interface displayed on a display unit of an information processing device according to a second embodiment of the present invention. [Figure 16] 13 is an example of a user interface displayed on a display unit of an information processing device according to a second embodiment of the present invention. [Figure 17] FIG. 13 is a functional block diagram of an object posture control unit according to a third embodiment of the present invention. [Figure 18] FIG. 13 is a flowchart of a third object attitude control process according to the third embodiment of the present invention. [Figure 19] This is a data structure that indicates the movable area of ​​each body part. [Figure 20]13 is an example of a user interface displayed on a display unit of an information processing device in the third object attitude control process according to the third embodiment of the present invention. [Figure 21] 13 is an example of a user interface displayed on a display unit of an information processing device in the third object attitude control process according to the third embodiment of the present invention. [Figure 22] FIG. 13 is a functional block diagram of an object posture control unit according to a fourth embodiment of the present invention. [Figure 23] FIG. 13 is a flowchart of a second object attitude control process according to a fourth embodiment of the present invention. [Figure 24] 13 is an example of a modified example of a user interface displayed on a display unit of an information processing device in the second object attitude control process according to the fourth embodiment of the present invention. [Diagram 25] FIG. 13 is a flowchart of a third object attitude control process according to the fourth embodiment of the present invention. [Figure 26] 23 is an example of a case where the position of the information processing device is changed in the third object attitude control process according to the fourth embodiment of the present invention. [Figure 27] 13 is an example of a user interface displayed on a display unit of an information processing device according to a fourth embodiment of the present invention. [Figure 28] 13 is an example of a user interface displayed on a display unit of an information processing device according to a fifth embodiment of the present invention. [Figure 29] 13 is an example of a user interface displayed on a display unit of an information processing device according to a fifth embodiment of the present invention. [Diagram 30] 13 is an example of a user interface displayed on a display unit of an information processing device according to a fifth embodiment of the present invention. [Diagram 31] 13 is an example of a user interface displayed on a display unit of an information processing device according to a sixth embodiment of the present invention. [Diagram 32] 13 is an example of a user interface displayed on a display unit of an information processing device according to a sixth embodiment of the present invention. [Diagram 33]13 shows a data structure of coordinates of a first target portion of an object before and after movement of a terminal according to the seventh embodiment of the present invention. [Diagram 34] 13 is an example of a user interface displayed on a display unit of an information processing device according to a seventh embodiment of the present invention. [Diagram 35] 1 shows a modification of the hardware configuration in the object attitude control system according to the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, the embodiment of the present invention will be described with reference to the drawings. However, the present invention can be implemented in many different aspects, and is not to be interpreted as being limited to the description of the embodiment exemplified below. The drawings may be shown in a schematic manner to make the explanation clearer, but they are merely examples and do not limit the interpretation of the present invention. Furthermore, the letters "first" and "second" attached to each element are convenient labels used to distinguish each element, and have no further meaning unless otherwise explained. In addition, in the drawings referred to in this embodiment, the same parts or parts having similar functions are given the same or similar symbols (symbols simply obtained by adding A and B to the number xxx), and repeated explanations thereof may be omitted. In addition, some of the configurations may be omitted from the drawings. In addition, if a person having ordinary knowledge in the field to which the present invention belongs can recognize the above, no special explanation will be given.

[0021] The "target part" described in one embodiment of the present invention refers to a movable part of an object, and a linked part is formed by combining multiple target parts. The first target part corresponds to the part that moves among the linked parts. The second target part corresponds to the part that serves as the fulcrum for the movement of the first target part based on the set first target part among the linked parts.

[0022] First Embodiment An object attitude control system according to a first embodiment of the present invention will be described in detail with reference to the drawings.

[0023] (1-1. Hardware configuration of object attitude control system) Fig. 1 shows a hardware configuration and a functional block diagram of an object posture control system 1. As shown in Fig. 1, the object posture control system 1 includes a terminal 10 and a server 20. The terminal 10 and the server 20 may be collectively referred to as an information processing device.

[0024] The terminal 10 is a type of computer, and includes a display unit 11, a control unit 12, a storage unit 13, an operation unit 14, a communication unit 15, a sensor unit 16, a camera unit 17, and a microphone unit 18. In this example, a smartphone is used as the terminal 10. Note that the terminal 10 is not limited to a smartphone, and may be a mobile phone (feature phone), a tablet terminal, a notebook PC (Personal Computer), an IoT device (a device equipped with a power supply mechanism, a communication function, and an information storage mechanism), or the like, and is applicable as long as it is capable of communicating with the server 20 via a network.

[0025] The display unit 11 is a display device such as a liquid crystal display or an organic EL display, and the display content is controlled by a signal input from the control unit 12.

[0026] The control unit 12 includes a CPU (Central Processing Unit), a RAM This is realized by providing a random access memory (Random Access Memory), an application specific integrated circuit (ASIC), or a flexible programmable gate array (FPGA). The control unit 12 executes an application including an object attitude control program stored in the storage unit 13 based on the operations of the display unit 11 and the operation unit 14.

[0027] The storage unit 13 functions as a database for storing an object attitude control program and spatial information used in the object attitude control program. The storage unit 13 may be a memory, an SSD, or a memory-capable element.

[0028] The operation unit 14 includes a controller, a button, or a switch. When the operation unit 14 is moved up, down, left, right, pressed, rotated, or the like, information based on the operation is transmitted to the control unit 12. In this embodiment, the terminal 10 is a display device (touch panel) having a touch sensor, so that the display unit 11 and the operation unit 14 may be disposed in the same place. This makes it possible to specify a target part of an object while displaying the object, thereby facilitating attitude control of the object.

[0029] The communication unit 15 has a function of transmitting and receiving with the server 20. A LAN transceiver (e.g., a Wi-Fi transceiver) is used for the communication unit 15. The transceiver is not limited to a LAN transceiver. If the terminal is a portable terminal, a transceiver for portable terminal communication (e.g., LTE communication) may be provided, or a transceiver for short-range wireless communication may be provided. The terminal 10 is connected to the server 20 via a network 50.

[0030] The sensor unit 16 has a function of detecting the location information of the terminal. (Global Positioning System), acceleration sensor, and gyro sensor (angular velocity sensor).

[0031] The camera unit 17 has a function of capturing an image of the environment, and captures an image of an object on which a marker unit is provided for displaying the object. The camera unit 17 is, for example, a CMOS image sensor.

[0032] The microphone unit 18 has a function of capturing oral information uttered by the user and inputting it to the control unit 12 or the storage unit 13 .

[0033] The server 20 has a communication unit 21, a storage unit 22, a control unit 23, and a display unit 24. The server 20 functions as a database and an application server. Note that if all information related to the object attitude control program is pre-installed in the terminal 10, the server 20 does not necessarily have to be used.

[0034] The communication unit 21 has a transceiver, and communicates control information of objects with the terminal 10 via the network 50. An intranet LAN transceiver is used for the communication unit 21. Note that the transceiver is not limited to an intranet LAN transceiver, and any device capable of communicating in the same manner as the terminal 10 is used.

[0035] The storage unit 22, using a hard disk and an SSD, functions as a database of information used in the object attitude control program.

[0036] The control unit 23 controls the processing of the object attitude control program using a CPU, a RAM, an ASIC, or an FPGA. In addition, a user interface for executing the object attitude control program may be provided to the display unit 24 by an instruction from the control unit 23.

[0037] (1-2. Configuration of Object Attitude Control Unit 100) 2 shows a functional block diagram of an object posture control unit 100 which is configured by each component of the terminal 10 and each configuration of the server 20 in the object posture control system 1 and controls a program (object posture control program) that realizes the object posture control function. In this example, an example in which the object posture control program is provided in the terminal will be described.

[0038] The object posture control unit 100 includes a captured image acquisition unit 110, a marker detection unit 120, a space definition unit 130, an object display unit 140, a first target part setting unit 150, a second target part setting unit 160, a linked part determination unit 170, a movement information acquisition unit 180, and an object coordinate change unit 190.

[0039] The captured image acquisition unit 110 has a function of acquiring an environmental image of the real space captured by the camera unit 17 .

[0040] The marker detection unit 120 has a function of detecting markers from the image acquired by the photographed image acquisition unit 110. In this example, information on the markers is stored in advance in the storage unit 13. Note that the information on the markers does not necessarily have to be stored in the storage unit 13, and feature points that can be markers may be extracted from the photographed image.

[0041] The space definition unit 130 has a function of aligning the virtual space with the coordinates of the display unit 11. In this embodiment, the space definition unit 130 uses a marker-based alignment method to align the virtual space with the coordinates of the display unit 11 based on the detected marker.

[0042] The object display unit 140 has a function of displaying on the display unit 11 an object that is placed in a virtual space and is the target of posture control.

[0043] The first target part setting section 150 has a function of setting a movable first target part among a plurality of interlocking parts. The second target part setting section 160 has a function of setting a second target part based on the set first target part. In this embodiment, the first target part setting section 150 and the second target part setting section 160 can be collectively referred to as a setting section, and the setting section is a general term for a functional section that sets a plurality of interlocking parts of an object. When it is not necessary to separately describe the first target part setting section 150 and the second target part setting section 160, they will be described as a setting section.

[0044] The interlocking part determination unit 170 has a function of determining whether or not the multiple parts set in the setting unit are to be interlocked. The determination of whether or not to be interlocked may be made using information previously incorporated in the object posture control program, or may be made based on a signal input by the user.

[0045] The movement information acquisition unit 180 has a function of acquiring movement information based on the coordinates of the terminal 10 before and after movement input from the sensor unit 16 .

[0046] In order to display the moved object on display unit 11, object coordinate change unit 190 has a function of changing and recording the coordinates of some target parts of the interlocking parts, and transmitting the position information to object display unit 140. Movement information acquisition unit 180, object coordinate change unit 190, and object display unit 140 acquire movement information of target parts of the object 60 times per second, and can display the movement (posture change) of the object as a still image or a video by changing the spatial coordinates and the coordinates on the display screen of the target parts.

[0047] (1-3. Object Attitude Control Processing) Next, an object posture control process based on an instruction by an object posture control program in the object posture control unit 100 will be described. Fig. 3 is a flow diagram of the object posture control process. As shown in Fig. 3, the object posture control process includes a first object posture control process S100, a second object posture control process S200, and a third object posture control process S300.

[0048] The first object posture control process S100 includes a captured image acquisition and display process, a marker information detection process, a space definition process, and a captured image / object superimposition display process. The second object posture control process S200 includes a movement mode determination process, a first target part information input process, a first target part setting process, a second target part information input process, a second target part setting process, and a process for determining whether or not the part is linked. The third object posture control process S300 includes a terminal movement amount acquisition process, an object coordinate change process, a process for displaying the object after movement, and a process for determining whether or not the terminal movement has ended. Each object posture control process will be described separately.

[0049] (1-3-1. First object attitude control process S100) The first object posture control processing S100 is shown in Fig. 4. The first object posture control processing S100 is started when an application including an object posture control program is started.

[0050] In the first object posture process S100, the captured image acquisition unit 110 captures an environmental image of the real space, acquires the captured image (S110), and displays the captured image on the display unit 11 of the terminal 10.

[0051] Fig. 5 is an example of capturing an image of a real space with the terminal 10. In Fig. 5, the real space has the terminal 10, a table 60, and a seat 70 arranged therein. The seat 70 includes four markers 75. In this example, the terminal 10 captures an image of a part of the table 60 and the seat 70 so as to include the four markers 75.

[0052] Returning to FIG. 4, the explanation will be given. Next, the marker detection unit 120 performs a marker detection process (S120). In this example, information on four markers is registered in advance in the object posture control program. The marker information relates to the form and arrangement of the marker. When the four markers are not detected (S120; No), the process returns to the capture and display process of the captured image.

[0053] Next, when four markers are detected (S120; Yes), the space definition unit 130 performs alignment (space definition) of the coordinates between the real space and the virtual space using a marker-based alignment method (S130). At this time, the space definition unit 130 compares the marker 75 in the real space displayed on the display unit 11 with the marker in the virtual space stored in the object attitude control program, and performs adjustment so that the coordinates of the virtual space can be displayed on the display unit 11 as the coordinates of the real space. For example, the position coordinates of the marker 75-1 are defined as the position coordinates in the real space (Xr75-1, Yr75-1, Zr75-1), the position coordinates on the display unit 11 (Xd75-1, Yd75-1), and the position coordinates in the virtual space (Xv75-1, Yv75-1, Zv75-1).

[0054] Next, the space definition unit 130 performs a process of acquiring object coordinates in the virtual space (S140). FIG. 6 shows the coordinates of each part of the object in the virtual space. As shown in FIG. 6, the coordinate information of each part of the object is set with respect to an origin that is set based on the coordinates of the four markers. Through the above process, the information on the display unit 11 of the marker 75 is acquired, and the coordinates of each part of the object are acquired.

[0055] Next, the object display unit 140 displays the image of the object superimposed on the captured image (S150). FIG. 7 shows a user interface of the object displayed on the display unit 11 of the terminal 10. In FIG. 7, the object 80 is arranged and displayed on the sheet 70. In this example, the object 80 has a humanoid body shape and has a plurality of parts 81 and a plurality of bones 82. The part 81 is a movable part and corresponds to a human joint. The part 81 (part 81a) is connected to the adjacent part 81 (part 81b) via the bone 82. In other words, it can be said that the parts 81a and 81b are linked parts. It is to be noted that the sheet 70 does not need to be displayed when the object 80 is displayed. With the above, the first object attitude control process S100 ends.

[0056] (1-3-2. Second object attitude control process S200) A flow diagram of the second object posture control process S200 is shown in Fig. 8. The second object posture control process S200 is executed upon completion of the first object posture control process S100.

[0057] First, it is determined whether the mode is the moving mode (S210). The moving mode may be determined based on whether the sensor unit 16 detects the motion of the terminal 10, or may be determined based on an input from the user. If the mode is the moving mode (S210; Yes), the second object attitude control process ends.

[0058] When the mode is not the moving mode (S210; No), information on the first target part is input to the first target part setting unit 150 as a part of the interlocking parts (S220). FIG. 9 is a user interface showing an example when inputting information on the first target part. As shown in FIG. 9, when inputting information on the first target part, a message "Please touch the first target part" may be displayed on the display screen. In this example, when inputting information on the first target part, the user touches a part of the object displayed on the display unit 11 that corresponds to the part 81a. The part 81a corresponds to the wrist joint of the object 80. As described above, the terminal 10 has a touch panel, and the position information of the part 81a is input to the first target part setting unit 150 by touching the part 81a.

[0059] When inputting information about the portion 81a, the portion corresponding to the portion 81a may be traced so as to surround it, or multiple fingers may be used to surround the portion corresponding to the portion 81a.

[0060] Returning to Fig. 8, the first target part setting section 150 then sets the touch-operated part 81a as the first target part based on the input information (S230).

[0061] Next, information on the second target part is input to the second target part setting unit 160 (S220). FIG. 10 is a user interface showing an example when inputting information on the second target part. As shown in FIG. 10, when inputting information on the second target part, as in the case of the first target part, "Please touch the second target part" is displayed, and the user touches a part of the object displayed on the display unit 11 that corresponds to the part 81b that is the second target part. The part 81b corresponds to the elbow joint of the object 80. At this time, as in the case of the part 81a, the part 81b is touched, and the position information of the part 81b is input to the second target part setting unit 160.

[0062] Returning to Fig. 8, the explanation will now be made. Next, based on the input information, the second target part setting section 160 sets the touch-operated part 81b as the second target part (S250).

[0063] Next, the interlocking part determination unit 170 determines whether the parts 81a and 81b move in an interlocking manner. If it is determined that the parts 81a and 81b are not interlocking parts (S260; No), the information on the first target part is input again (S220). In this example, the parts 81a and 81b are connected via the bone 82a, and therefore are determined to be interlocking parts (S260; Yes). With this, the second object posture control process S200 ends.

[0064] (1-3-3. Third object attitude control process S300) 11 is a flow diagram of the third object posture control process S300. The third object posture control process S300 is started upon completion of the second object posture control process S200.

[0065] In the third object posture control process S300, first, the movement information acquisition unit 180 acquires the amount of movement (also referred to as information related to the movement) of the terminal 10 (S310). In this example, tracking of the object posture accompanying changes in the position information of the terminal 10 is performed by the V-SLAM (Visual Simultaneous Localization and Mapping) method. FIG. 12 is an example of the case where the terminal 10 moves in accordance with the movement of a user 90. When moving the terminal 10, the user does not need to be in contact with the display unit 11 of the terminal 10. Measurements are made using the sensor unit 16 and acquired by the movement information acquisition unit 180. A GPS, an acceleration sensor, or an angular velocity sensor is used as the sensor unit 16. FIG. 13 (A) is a data structure indicating the initial value (before movement), the position coordinates of the terminal 10 after time t1 (after movement), and the amount of movement of the terminal 10 calculated from the coordinates before and after the movement.

[0066] Next, based on the acquired amount of movement, the object coordinate change unit 190 changes and records the coordinates of the first target part (part 81a) of the object 80 which is the part to be moved (S320). FIG. 13(B) shows a data structure of the coordinates in the virtual space of the first target part (part 81a) of the object after time t1 and the coordinates on the display unit 11. As shown in FIG. 13(B), information on the first target part (part 81a) of the object 80 in the virtual space and information on the first target part of the object 80 on the display unit 11 are stored based on the movement information of the terminal 10. The coordinate information of (part 81a) is changed, and the changed coordinate information is stored (the coordinate information is rewritten).

[0067] Returning to FIG. 11, the description will be given. Next, the object display unit 140 displays the object 80 on the display unit 11 using information on the first target part after the movement (after time t1) (S330). FIG. 14 is a user interface displaying the object 80 with the position of the first target part changed on the display unit 11. As shown in FIG. 14, the first target part (part 81a) moves from the position displayed in FIG. 7 with the second target part (part 81b) as a reference. That is, it is possible to move some of the target parts among the multiple interlocking parts based on information on the movement of the information processing device. The above process is repeated as long as the terminal 10 moves (S340; Yes). In this example, the amount of movement of the terminal is acquired 60 times per second, so that the change in the attitude of the object is displayed smoothly. When the movement of the terminal 10 ends (S340; No), the third object attitude control process S300 ends.

[0068] As described above, by using the object posture control program of this embodiment, the posture of an object can be easily controlled by setting the linked parts and moving the terminal, without performing complex control.

[0069] <Second embodiment> In this embodiment, an object posture control unit and an object posture control method different from those in the first embodiment will be described. Specifically, an example in which the fulcrum when moving the first target part is different will be described. Note that the same configuration and method as those in the first embodiment may be omitted as appropriate.

[0070] Fig. 15 is a user interface showing an example of inputting information on the second target part. As shown in Fig. 15, the part corresponding to part 81c may be touched to input information on the second target part and set as the second target part. In this case, part 81a and part 81c are separated from each other, but by setting in advance in the program that bone 82a, part 81b, and bone 82b are regarded as one bone, they can be determined as linked parts.

[0071] As a result, by using this embodiment, when the terminal 10 is moved, as shown in FIG. 16, the part 81a is moved with the part 81c as a reference, thereby making it possible to control the attitude of the object.

[0072] In addition, when the bone 82a, the part 81b, and the bone 82b are not regarded as one fixed bone, the part 81b may be moved in conjunction with the movement of the part 81a. In this case, the part 81b is not controlled, so it may move simultaneously with the movement of the part 81a, or may move with a delay.

[0073] <Third embodiment> In this embodiment, an object posture control unit and an object posture control method different from those in the first embodiment will be described. Specifically, a process of setting a third target part when the movement area of ​​the first target part reaches a limit point in performing the third object posture control process S300 will be described. Note that configurations and methods similar to those in the first and second embodiments may be omitted as appropriate.

[0074] (3-1. Configuration of Object Attitude Control Unit 100A) Fig. 17 shows a functional block diagram of an object posture control unit 100A in the object posture control system 1A. As shown in Fig. 17, the object posture control unit 100A includes a photographed image acquisition unit 110, a marker detection unit 120, a space definition unit 130, an object display unit 140, a first target part setting unit 150, a second target part setting unit 160, an interlocking part determination unit 170, a movement information acquisition unit 180, and an object coordinate change unit 190, as well as a limit point detection unit 195 and a third target part setting unit 200.

[0075] Limit point detection section 195 has a function of detecting movement of the first target part, and inputting information to third target part setting section 200 when the first target part reaches the limit point of the movable range. Third target part setting section 200 has a function of setting a third target part based on the signal input from limit point detection section 195. When the third target part is set, the second target part switches from a fulcrum to a moving part, and the second target part moves based on the third target part.

[0076] (3-2. Object Attitude Control Processing) Fig. 18 shows a flow diagram (S300A) of the second object posture control process based on commands by the object posture control program in the object posture control unit 100A. As shown in Fig. 18, the terminal 10A is moved, the amount of movement of the terminal 10A is acquired (S310), and the coordinates of the first target part (part 81a) in the virtual space and on the display unit 11 are changed (S320).

[0077] FIG. 19 shows a data structure indicating the movable area of ​​each part. As shown in FIG. 19, the relative position coordinates with respect to the second target part are set as the movable area of ​​the first target part. At this time, when the limit point detection unit 195 determines that the limit point of the movable area of ​​the first target part (in this example, the part 81a) is not exceeded (S325; No), the limit point detection unit 195 displays the object in the same manner as in the first embodiment (S330). On the other hand, as shown in FIG. 20, when the limit point detection unit 195 determines that the limit point of the movable area of ​​the first target part is exceeded (S325; Yes), the limit point detection unit 195 inputs the information to the third target part setting unit 200. Based on the input information, the third target part setting unit 200 connects to the second target part and sets the part provided on the opposite side of the first target part with respect to the second target part as the third target part (S350). In this example, a part 81c (corresponding to a shoulder joint) provided on the opposite side of a part 81a (corresponding to a wrist joint) corresponding to a first target part is set as a third target part with respect to a part 81b (corresponding to an elbow joint) corresponding to a second target part. Part 81c is connected to part 81b via a bone 82b. In this embodiment, part 81c is set automatically. If part 81c is not set automatically, the third target part may be set after inputting information about the third target part by a user operation, as in the case of the first target part. In this case, it may be determined whether the third target part is an interlocking part.

[0078] Next, after setting the third target part, the amount of movement of the terminal 10 is acquired (S360), the coordinates of the second target part (part 81b) are changed (S370), and based on the changed coordinates of the second target part, an object in which the second target part (part 81b) has moved with reference to the third target part (part 81c) is displayed (S380) as shown in Fig. 21. As described above, by using this embodiment, the attitude of the object can be easily controlled even when the moving part reaches the limit of the movable area.

[0079] The same process as above may be repeated when the moving part, which moves with respect to the fixed part, detects the limit of the movable area. In other words, when the (n-1)th target part moves with respect to the nth (n is a natural number of 2 or more) target part, which is a fixed point, and the (n-1)th target part reaches the limit of the movable area, the (n+1)th target part may be connected to the nth target part and set on the opposite side of the (n-1)th target part. This allows the posture of the object to be controlled more smoothly.

[0080] <Fourth embodiment> In this embodiment, an object attitude control process different from that in the first embodiment will be described. Specifically, an example in which multiple parts move based on the movement of one terminal will be described. Note that configurations and methods similar to those in the first to third embodiments may be omitted as appropriate.

[0081] (4-1. Configuration of Server and Object Attitude Control Unit 100B) Fig. 22 shows a functional block diagram of an object posture control unit 100B in an object posture control system 1B. As shown in Fig. 22, the object posture control unit 100B includes a photographed image acquisition unit 110, a marker detection unit 120, a space definition unit 130, an object display unit 140, a first target part setting unit 150, a second target part setting unit 160, an interlocking part determination unit 170, a movement information acquisition unit 180, and an object coordinate change unit 190, as well as an interlocking part addition unit 210, a fourth target part setting unit 220, and a fifth target part setting unit 230.

[0082] The interlocking part adding section 210 has a function of setting an additional interlocking part (sometimes referred to as the second interlocking part) different from the interlocking parts (sometimes referred to as the first interlocking part) including the first target part and the second target part. The fourth target part setting section 220 has a function of setting a fourth target part. The fourth target part corresponds to the part of the second interlocking part that moves. The fifth target part setting section 230 has a function of setting a fifth target part. The fifth target part corresponds to the part of the second interlocking part that serves as the fulcrum for the movement of the fourth target part.

[0083] (4-2. Object Attitude Control Processing) FIG. 23 shows a flow diagram of the second object posture control process S200B based on a command by the object posture control program in the object posture control unit 100B. As shown in FIG. 23, after the first target part and the second target part are set and it is determined that they are interlocking parts (S250; Yes), the interlocking part adding unit 210 determines whether to add an interlocking part (S255). The above determination may be made based on an input from the user. At this time, the display unit 11 may display "Do you want to add an interlocking part?" along with "Yes" and "No" buttons. When the user presses the "Yes" button, a signal for adding an interlocking part is input, and the mode is switched to adding an interlocking part (S255; Yes).

[0084] 24, the part 81d is touched, and position information of the part 81d is input to the fourth target part setting section 220 (S260). Based on the input information, the fourth target part setting section 220 sets the touched part 81d as the fourth target part (S265).

[0085] Next, the part 81e shown in FIG. 24 is touched, and the position information of the part 81e is input to the fifth target part setting section 230 (S270). Based on the input information, the fifth target part setting section 230 sets the touched part 81e as the fifth target part. (S275).

[0086] Next, the interlocking part determination unit 170 determines whether the fourth target part and the fifth target part are interlocking parts, similar to the first target part and the second target part (S280). If it is determined that the set fourth target part and the fifth target part are interlocking parts (S280; Yes), the second object posture control process S200B ends.

[0087] Fig. 25 is a flow diagram of the third object control process S300B. The movement of the object accompanying the movement of the terminal 10B is realized in the same manner as in the first embodiment, but in the case of this embodiment, as shown in Fig. 26, when the position of the terminal 10B is changed from a lying state to a standing state, the coordinate change of the first target part and the fourth target part is changed based on this movement amount (S322), and at the same time, as shown in Fig. 27, the first target part (part 81a) moves with the second target part (part 81b) as a reference, and at the same time, an object in which the fourth target part (part 81d) moves with the fifth target part (part 81e) as a reference is displayed (S330). In other words, when one terminal moves, a plurality of interlocking parts can be moved to control the attitude of the object.

[0088] <Fifth embodiment> In this embodiment, an object posture control process different from that in the first embodiment will be described. Specifically, an example of setting a target part when multiple parts are displayed in a superimposed manner will be described. Note that configurations and methods similar to those in the first to fourth embodiments may be omitted as appropriate.

[0089] FIG. 28 is a user interface showing a case where a plurality of parts are displayed in an overlapping manner. As shown in FIG. 28, a part 81a (corresponding to a wrist joint) of a user is displayed in an overlapping manner with a part 81g (corresponding to a waist joint). At this time, when the overlapping part is specified by touch operation, the display direction of the object 80 on the display unit 11 may be changed. For example, when the overlapping part is touch operated, as shown in FIG. 29, a message "Do you want to change the display direction of the object?" may be displayed on the display unit 11, and "Yes" and "No" buttons may be displayed. When the "Yes" button is pressed, as shown in FIG. 30, the object 80 may be displayed rotated 90 degrees clockwise with respect to the sheet 70. As a result, the overlapping parts 81a and 81g are displayed separately, and a specific part can be set.

[0090] The method of setting a specific part in the overlapping part is not limited to the above method. For example, a part that satisfies a predetermined condition may be set as the part to be specified (first target part or second target part). Specifically, when the overlapping part is specified by performing a touch operation, the part displayed in the foreground may be set as the part to be specified.

[0091] As another method, the part to be specified may be set based on a predetermined operation on the overlapping part. Specifically, when the overlapping part is touched for a short time (e.g., 1 sec), the front part may be set as the part to be specified, and when the overlapping part is touched for a long time (e.g., 5 sec), the rear part may be set as the part to be specified.

[0092] Sixth Embodiment In this embodiment, an object posture control process different from the first to fifth embodiments will be described. Specifically, an example will be described in which a new connecting part is set when the first target part moves and comes into contact with a movable part different from the first target part and the second target part. Note that configurations and methods similar to those of the first to fifth embodiments may be omitted as appropriate.

[0093] Fig. 31 is a user interface showing an example in which the first target part is in contact with a different part. As shown in Fig. 31, the user's part 81a (corresponding to the left wrist joint) is displayed in contact with part 81e (corresponding to the right elbow joint). At this time, if part 81e has been input in advance as a movable part, it may be set as the fourth target part. At this time, part 81e and part 81f are parts that are linked via bone 82e, so part 81f may be set as the fifth target part.

[0094] In the above, when the first target portion, portion 81a, moves based on the second target portion, portion 81b, as a reference, the fourth target portion, portion 81e, may move based on the fifth target portion, portion 81f, as shown in FIG. 32, based on the movement of portion 81a.

[0095] By using this embodiment, the movement of one interlocking part can be interfered with to control the posture of another interlocking part.

[0096] The above control is not limited to the posture of one interlocking part. For example, the posture of another interlocking part may be controlled by an object other than the object 80. In other words, it is possible to set a passive interlocking part without a user operation.

[0097] Seventh embodiment In this embodiment, an object attitude control process different from the first to fifth embodiments will be described. Specifically, an example of controlling the attitude of an object using an angle θ in addition to the X coordinate, Y coordinate, and Z coordinate will be described. Note that configurations and methods similar to those of the first to sixth embodiments may be omitted as appropriate.

[0098] FIG. 33 shows the data structure of the coordinates in the virtual space of the first target part (part 81a) of the object before and after the movement of the terminal and the coordinates on the display unit 11. As shown in FIG. 33, in addition to the X coordinate, Y coordinate, and Z coordinate, the angle θ may be used in the movement information of the terminal 10. In this case, any of the X coordinate, the Y coordinate, and the Z coordinate does not have to be used. The information of the first target part (part 81a) of the object 80 in the virtual space and the information of the coordinates of the first target part (part 81a) of the object 80 on the display unit 11 are changed in accordance with the movement of the terminal 10, and the information of the changed coordinates is stored (the coordinate information is rewritten).

[0099] Fig. 34 is an example of a user interface of an object after the posture displayed on the display unit 11 has been changed. As shown in Fig. 34, by using this embodiment, it is possible to control the posture of the object, particularly the twist posture.

[0100] It should be noted that within the scope of the concept of the present invention, a person skilled in the art may come up with various modifications and alterations, and it is understood that these modifications and alterations also fall within the scope of the present invention. For example, those in which a person skilled in the art appropriately adds, deletes, or modifies the design of the above-mentioned embodiments, or adds, omits, or modifies the conditions of steps, are also included in the scope of the present invention as long as they include the gist of the present invention.

[0101] (Modification) In the first embodiment of the present invention, an example is shown in which information on a first target part is input by a touch operation to set the first target part, and information on a second target part is input by a touch operation to set the second target part, but this is not limiting. For example, the second target part may be automatically set when the first target part is set. In this case, the second target part may be automatically set to a part that is higher than the first target part.

[0102] In the first embodiment of the present invention, an example in which the information on the first target part and the second target part is input by the user's touch operation is shown, but the present invention is not limited to this. For example, the information on the target parts may be input orally using the microphone unit 18 of the terminal 10.

[0103] Furthermore, in the first embodiment of the present invention, when the terminal 10 is moved, the terminal may be moved in a state where the first target part or the second target part of the object displayed on the display unit 11 is touched. At this time, only the coordinates of the first target part in the virtual space may be changed in a state where the object 80 displayed on the display unit 11 is stationary. In the virtual space, the first target part moves with the second target part as a reference. When the movement of the terminal 10 is completed, the object with the position of the first target part changed may be displayed on the display unit 11.

[0104] In the first embodiment of the present invention, an example is shown in which a captured image and an object in a virtual space are aligned by a marker-based alignment method and displayed in a superimposed manner, but the present invention is not limited to this. As a method for aligning a virtual space including an object with a captured image, a natural feature-based alignment method that does not use a specific marker, a vision-based alignment method such as a model-based alignment method, or a sensor-based alignment method using a sensor may be used as appropriate, in addition to the marker-based alignment method. In addition, it is not necessary to align the captured image with the object in the virtual space, and any appropriate method may be used as long as the coordinates of the object in the virtual space can be set.

[0105] In the first embodiment of the present invention, an example of displaying an object using augmented reality is shown, but the present invention is not limited to this. As long as the coordinates of an object can be specified, the embodiment of the present invention can be applied to virtual reality (VR), mixed reality (MR), and the like in addition to augmented reality.

[0106] In the first embodiment of the present invention, an example is shown in which one terminal is used to display an object and control the object's attitude, but the present invention is not limited to this. A plurality of terminals may be linked via a network 50 to control the attitude of the object. For example, as shown in FIG. 35, one terminal 10 (terminal 10-1) may display an object, and the position of another terminal 10 (terminal 10-2) may be changed to control the attitude of the object. This allows the object to be fixed and continuously displayed on one terminal 10, so that the attitude of the object can be precisely controlled. Furthermore, the terminals 10-1 and 10-2 do not have to be the same type of terminal. For example, a goggle-type terminal may be used as a terminal for display, and a watch-type terminal may be used as a terminal for attitude control. [Explanation of symbols]

[0107] 1 ···Object attitude control system, 10 ···Terminal, 11 ···Display unit, 12 ···Control unit, 13 ···Memory unit, 14 ···Operation unit, 15 ···Communication unit, 16 ···Sensor unit, 17 ···Camera unit, 18 ···Microphone unit, 20 ···Server, 21 ···Communication unit, 22 ···Memory unit, 23 ···Control unit, 24 ···Display unit, 50 ···Network, 60 ···Table, 70 ···Sheet, 75 ···Marker, 80 ···Object, 81 ···Part, 82 ···Bone, 90 ···User, 100...Object posture control unit, 110...Photographed image acquisition unit, 120...Marker detection unit, 130...Space definition unit, 140...Object display unit, 150...First target part setting unit, 160...Second target part setting unit, 170...Interlocking part determination unit, 180...Movement information acquisition unit, 190...Object coordinate change unit, 195...Limit point detection unit, 200...Third target part setting unit, 210...Interlocking part addition unit, 220...Fourth target part setting unit, 230...Fifth target part setting unit

Claims

1. In the information processing device, Displaying an object having a plurality of interlocking parts arranged in a virtual space on a display unit; An object posture control program for controlling the posture of the object so as to move a portion of target parts among a plurality of interlocking parts based on information regarding the movement of the information processing device.

2. The information processing device has a setting unit, the setting unit sets a part of target parts among the plurality of interlocking parts based on information input when a part of the object displayed on the display unit is touch-operated.

2. The object attitude control program according to claim 1.

3. the setting unit sets a first target part, which is the part of the plurality of interlocking parts, to which the object can move; A second target part among the plurality of interlocking parts is set by the setting unit based on the set first target part, The first target portion moves relative to the second target portion.

3. The object attitude control program according to claim 2.

4. When the first target portion reaches a limit point of a movable range, the first target portion is connected to the second target portion, and a third target portion is set on an opposite side of the first target portion with respect to the second target portion; The object attitude control program according to claim 3 , further comprising moving the second target portion of the object with respect to the third target portion.

5. changing a display direction of the object when a portion where a plurality of parts are overlapped is designated in setting the first target part or the second target part in the object displayed on the display unit; 5. The object attitude control program according to claim 3.

6. When a portion where a plurality of parts are overlapped is specified in the object displayed on the display unit when setting the first target part or the second target part, a part that satisfies a predetermined condition is set as the first target part or the second target part.

5. The object attitude control program according to claim 3.

7. when a portion on which a plurality of portions are overlapped is designated in setting the first target portion or the second target portion in the object displayed on the display unit, the first target portion or the second target portion is set based on a predetermined operation on the overlapping portion.

6. The object attitude control program according to claim 4 or 5.

8. setting a fourth target portion, different from the first target portion and the second target portion, to which the object can be moved; setting a fifth target portion connected to the fourth target portion; the first target part of the object moves with the second target part as a reference, and the fourth target part moves with the fifth target part as a reference, based on information regarding a movement of the information processing device.

8. The object attitude control program according to claim 3.

9. When the first target part of the object moves and comes into contact with a movable part different from the first target part and the second target part, the first target part is set as a fourth target part; setting a fifth target portion connected to the fourth target portion; The fourth target portion moves based on the movement of the first target portion, with the fifth target portion as a reference.

9. The object attitude control program according to claim 3.

10. 10. The object posture control program according to claim 3, wherein when the information processing device is moved while the first target part or the second target part is touched, the first target part moves based on the second target part of the object in the virtual space while the displayed object is stationary.

11. An object attitude control system for controlling an attitude of an object placed in a virtual space, comprising: Displaying the object having a plurality of interlocking parts on a display unit; controlling a posture of the object so as to move a part of the plurality of interlocking parts based on information relating to a movement of the information processing device; Object Attitude Control System

Citation Information

Patent Citations

  • Image processing program, image processor, image processing system and image processing method

    JP2012069126A

  • Vehicle seat slide device

    JP2012183923A

  • Information processing method, device, and program for causing computer to execute the method

    JP2019020836A

  • Information processing apparatus and program

    JP2019040340A

  • Information processing device and information processing method

    JP2015041126A