Information processing apparatus, information processing method, and program

Abstract

To allow a user to operate a 3D object as intended while using a system with a simple configuration.SOLUTION: An information processing apparatus includes: an operation region management unit which arranges an operation region in which a three-dimensional object can be operated, in a three-dimensional digital space; and a three-dimensional object management unit which arranges the three-dimensional object in the operation region. The three-dimensional object management unit changes the attitude of the three-dimensional object in the three-dimensional digital space, specifies one or more movable directions of the three-dimensional object in the three-dimensional digital space, for each operation region, and changes the position of the three-dimensional object in a moving direction, which is one of the movable directions.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, with the spread of remote conferencing systems in which multiple users make voice calls while sharing images via a network, the quality of communication between users located at remote locations has been greatly improved. In such a remote conferencing system, each of a plurality of users can add static information (for example, characters, figures, lines, etc.) or dynamic information (for example, movement of a cursor, etc.) to an image. As a result, the variations in expressions in remote communication have increased.

[0003] Furthermore, the information added to an image is not limited to two-dimensional information, but also extends to three-dimensional information. That is, by enabling a three-dimensional (hereinafter, the dimension is also simply denoted as "D") object to be placed in a 3D digital space or a 3D object to be moved in a 3D digital space, three-dimensional expressions in remote communication have become possible.

[0004] As technologies for operating a 3D object, various technologies are known. For example, Patent Document 1 discloses a technology for detecting a user's viewpoint and gesture, and moving a 3D object hit by the user's viewpoint according to the user's gesture. A light source and a camera are used to detect the user's viewpoint, and an optical sensor is used to detect the user's gesture.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in such a technique, the system configuration for detecting the user's line of sight and gestures becomes complicated. As a result, not only does it cost to install a system having a complicated configuration, but also space is required to install a system having a complicated configuration in the real space.

[0007] Therefore, the present invention solves the above problems, and a main object thereof is to provide a technique that enables a user to operate a 3D object as intended while using a system having a simple configuration.

Means for Solving the Problems

[0008] In order to solve the above problems, according to an aspect of the present invention, there is provided an information processing apparatus including: an operation area management unit that arranges an operation area, which is an area where a three-dimensional object can be operated, in a three-dimensional digital space; and a three-dimensional object management unit that arranges the three-dimensional object within the operation area, wherein the three-dimensional object management unit changes the posture of the three-dimensional object in the three-dimensional digital space, specifies one or a plurality of movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changes the position of the three-dimensional object in a movement direction that is any one of the movement allowable directions.

[0009] The operation area management unit may change the posture of the operation area in the three-dimensional digital space based on a posture control operation on the operation area, and the three-dimensional object management unit may specify the movement allowable direction based on the posture of the operation area after the change.

[0010] The three-dimensional object management unit may change the position of the three-dimensional object in the three-dimensional digital space in the moving direction by maintaining the relative position of the three-dimensional object with respect to the posture of the operation area.

[0011] The operation area management unit may specify the moving direction based on a position control operation on the operation area and the allowable moving direction, and change the position of the operation area in the three-dimensional digital space in the moving direction.

[0012] The three-dimensional object management unit may change the position of the three-dimensional object in the three-dimensional digital space in the moving direction by changing the relative position of the three-dimensional object with respect to the position of the operation area while restricting the position of the three-dimensional object within the operation area.

[0013] The three-dimensional object management unit may specify the moving direction based on a position control operation on the three-dimensional object and the allowable moving direction, and change the position of the three-dimensional object in the three-dimensional digital space in the moving direction.

[0014] Even if the posture of the operation area in the three-dimensional digital space changes, the three-dimensional object management unit may change the posture of the three-dimensional object in the three-dimensional digital space by maintaining the relative posture of the three-dimensional object with respect to the posture of the operation area.

[0015] The three-dimensional object management unit may change the posture of the three-dimensional object in the three-dimensional digital space by changing the relative posture of the three-dimensional object with respect to the posture of the operation area.

[0016] The three-dimensional object management unit may change the posture of the three-dimensional object in the three-dimensional digital space based on a posture control operation on the three-dimensional object.

[0017] The operation area management unit may change a first scale factor of the operation area based on a scale factor designation operation for the operation area, and calculate the size of the operation area based on the multiplication of the changed first scale factor and the initial size of the operation area.

[0018] The three-dimensional object management unit may calculate the size of the three-dimensional object based on the multiplication of the changed first scale factor, a second scale factor of the three-dimensional object, and the initial size of the three-dimensional object.

[0019] The operation area management unit may arrange a plurality of the operation areas in a three-dimensional digital space.

[0020] The information processing apparatus may include an input unit that receives an operation indicated by two-dimensional coordinates.

[0021] Further, according to another aspect of the present invention to solve the above problems, arranging an operation area, which is an area where a three-dimensional object can be operated, in a three-dimensional digital space, arranging the three-dimensional object within the operation area, changing the posture of the three-dimensional object in the three-dimensional digital space, specifying one or more movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changing the position of the three-dimensional object in a movement direction that is any one of the movement allowable directions. An information processing method executed by a computer is provided.

[0022] Also, according to another aspect of the present invention for solving the above problems, a computer is provided with an operation area management unit that arranges an operation area, which is an area where a three-dimensional object can be operated, in a three-dimensional digital space, and a three-dimensional object management unit that arranges the three-dimensional object within the operation area. The three-dimensional object management unit changes the posture of the three-dimensional object in the three-dimensional digital space, specifies one or a plurality of movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changes the position of the three-dimensional object in a movement direction that is one of the movement allowable directions. A program is provided that causes the computer to function as an information processing device.

Advantages of the Invention

[0023] As described above, according to the present invention, a technique is provided that enables a user to operate a 3D object as intended while using a system having a simple configuration.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Embodiments for Carrying Out the Invention

[0025] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0026] (0. Overview) First, the overview of the embodiments of the present invention will be described.

[0027] FIG. 1 is a diagram for explaining the overview of the embodiments of the present invention. Referring to FIG. 1, a 3D digital space 100 is constructed. Also, referring to FIG. 1, a world coordinate system XYZ is shown. For example, a position in the 3D digital space 100 can be represented by an X coordinate, a Y coordinate, and a Z coordinate in the world coordinate system XYZ. The 3D digital space 100 can be constructed by a system. For example, the system is the 3D object operation processing system 20 shown in FIG. 2.

[0028] In the 3D digital space 100, 2D objects 101 are arranged. For example, the 2D objects 101 may be images shared by a plurality of users via a network, and the plurality of users may be able to conduct a voice call while viewing such images. That is, the system according to the embodiment of the present invention can be applied to a remote conferencing system. In the example shown in FIG. 1, the 2D object 101 is an image of a distribution board, but the 2D object 101 may be any image.

[0029] The GUI (Graphical User Interface) layer 104 is arranged in the 3D digital space 100 and is a layer for generating visual information presented to the user.

[0030] In the embodiment of the present invention, the 3D object existence area 102 is an area sandwiched between the 2D object 101 and the GUI layer 104, and is an area where the 3D object 103 is allowed to exist. The 2D object 101 arranged in the 3D digital space 100 and the 3D object 103 arranged in the 3D object existence area 102 are projected onto the GUI layer 104 by a perspective projection toward the viewpoint 105, and an image presented to the user is generated.

[0031] The user attempts to operate the 3D object 103 while viewing the image generated by being projected onto the GUI layer 104. However, when the system has a simple configuration (for example, when the system accepts operations by two-dimensional coordinates), the user needs to specify the moving direction of the 3D object 103 two-dimensionally, so it may be difficult to operate the 3D object 103 as intended.

[0032] Therefore, in this specification, a technique is mainly proposed that enables the user to operate the 3D object 103 as intended while using a system having a simple configuration.

[0033] The above is an overview of the embodiments of the present invention.

[0034] (1. Details of the Embodiment) Next, the details of the embodiments of the present invention will be described.

[0035] (1-1. Configuration of the 3D Object Manipulation Processing System) First, a configuration example of the 3D object manipulation processing system 20 according to the embodiment of the present invention will be described. The 3D object manipulation processing system 20 can be realized by one or more computers. Since the 3D object manipulation processing system 20 can be realized by a device that processes various types of information, it can also be referred to as an information processing device.

[0036] FIG. 2 is a diagram showing a functional configuration example of the 3D object manipulation processing system 20 according to the embodiment of the present invention. The 3D object manipulation processing system 20 includes an input unit 210, a control unit (not shown), a storage unit (not shown), and a display unit 250.

[0037] The control unit (not shown) includes an arithmetic device such as a CPU (Central Processing Unit), and its function can be realized by a program stored in a ROM (Read Only Memory) being expanded into a RAM by the arithmetic device and executed. At this time, a computer-readable recording medium storing the program may also be provided.

[0038] Alternatively, the control unit (not shown) may be configured by dedicated hardware or may be configured by a combination of a plurality of hardware components. Data required for arithmetic operations by the arithmetic device is appropriately stored by a storage unit (not shown). The control unit (not shown) includes an input processing unit 211, a 2D object management unit 220, an operation area management unit 230, a 3D object management unit 240, and a display control unit 251.

[0039] The memory unit (not shown) is a memory device capable of storing a program for operating a control unit (not shown) and various information. For example, the memory unit (not shown) may be constituted by a non-volatile memory. For example, the memory unit (not shown) can also temporarily store data required in the process of operating the control unit (not shown).

[0040] The memory unit (not shown) includes a 2D object storage unit 222, a 2D object related database 224, an operation area related database 234, a 3D object storage unit 242, and a 3D object related database 244.

[0041] (Input unit 210) The input unit 210 receives an operation input by the user. In the embodiment of the present invention, it is mainly assumed that the input unit 210 is an input device that receives an operation indicated by two-dimensional coordinates. Such an input device may be realized by a mouse, a keyboard, a touch panel, or the like.

[0042] (Display unit 250) The display unit 250 is constituted by a display and has a function of performing display according to the control by the input processing unit 211. Here, the form of the display is not particularly limited. For example, the display unit 250 may be a liquid crystal display (LCD) device or an organic light emitting diode (OLED) device.

[0043] (Input processing unit 211) The input processing unit 211 discriminates the type of operation based on the operation received from the user by the input unit 210. For example, when a click operation or a tap operation on a button is performed, the input processing unit 211 may discriminate that a selection operation of the button has been performed. Similarly, when a click operation or a tap operation on an object is performed, the input processing unit 211 may discriminate that a selection operation of the object has been performed.

[0044] Then, according to the type of operation determined, the input processing unit 211 outputs that an operation has been input by the user to at least one of the 2D object management unit 220, the operation area management unit 230, and the 3D object management unit 240. The types of operations that can be input by the user will be described in detail later.

[0045] (2D Object Management Unit 220) The 2D object management unit 220 arranges 2D objects in the 3D digital space. The 2D object may be drawn by computer graphics. For example, the 2D object may have a single planar shape of a rectangle surrounded by vertices, sides, and faces, but the shape of the 2D object may not be limited.

[0046] The 2D object may be composed of a texture that is image data pasted on a single rectangular plane, the color of the surface of the 2D object, and the texture of the surface. In the embodiments of the present invention, it is mainly assumed that the 2D object is fixed at a specific position and posture in the 3D digital space. However, the position and posture of the 2D object may change based on an operation by the user or the like.

[0047] (2D Object Storage Unit 222) The 2D object storage unit 222 stores a texture that is image data pasted on a single rectangular plane. For example, the format of the texture may be JPEG (Joint Photographic Experts Group) or PNG (Portable Network Graphics), etc. However, the format of the texture may not be limited to a specific format.

[0048] (2D Object Related Database 224) The 2D object-related database 224 is a database that accumulates information about 2D objects. For example, the 2D object-related database 224 may be configured by associating the name, position, orientation, size, model data of the 2D object, which is information about the shape of the 2D object, and material data that constitutes the 2D object. A configuration example of the 2D object-related database 224 will be described in detail later.

[0049] (Operation area management unit 230) The operation area management unit 230 arranges an operation area, which is an area where a 3D object can be operated, in the 3D digital space. That is, in the embodiment of the present invention, the position of the 3D object is restricted to the inside of the operation area, and the operation can be performed while the 3D object exists inside the operation area. For example, when the input processing unit 211 determines that the input unit 210 has received an operation of selecting an operation area generation button (not shown), the operation area management unit 230 arranges the operation area in the 3D digital space.

[0050] The operation area may be drawn by computer graphics. For example, the operation area may have a rectangular parallelepiped shape surrounded by vertices, edges, and faces, but the shape of the operation area may not be limited. Operations on the 3D object may include operations that change the position, orientation, or scale factor (e.g., magnification or reduction rate, etc.) of the 3D object.

[0051] (Operation area-related database 234) The operation area-related database 234 is a database that accumulates information about the operation area. For example, the operation area-related database 234 may be configured by associating the ID, name, position, orientation, scale factor, size, and material data that constitutes the operation area of the operation area. A configuration example of the operation area-related database 234 will be described in detail later.

[0052] (3D object management unit 240) The 3D object management unit 240 arranges 3D objects within the operation area arranged by the operation area management unit 230. For example, when the input processing unit 211 determines that an operation of selecting a 3D object generation button (not shown) has been received by the input unit 210 while the operation area is selected, the 3D object management unit 240 arranges the 3D object within the operation area.

[0053] The 3D object may be drawn by computer graphics. For example, the 3D object may have a predetermined shape surrounded by vertices, edges, and faces. For example, the 3D object management unit 240 can change the posture of the 3D object in the 3D digital space.

[0054] In addition, the 3D object management unit 240 specifies the allowable movement direction of the 3D object in the 3D digital space for each operation area. The allowable movement direction may be one direction or a plurality of directions for each operation area. Then, the 3D object management unit 240 changes the position of the 3D object in the movement direction, which is any one of the specified allowable movement directions.

[0055] (3D object storage unit 242) The 3D object storage unit 242 stores the generation data of the 3D object. The generation data of the 3D object may include the model data of the 3D object, which is information regarding the shape, and the material data of the 3D object.

[0056] (3D object related database 244) The 3D object-related database 244 is a database that stores information about 3D objects. For example, the 3D object-related database 244 may be configured by associating the ID of a 3D object, its name, the operation area enclosing the 3D object, the position, orientation, scale factor of the 3D object, the reference information of the model data, and the reference information of the material data constituting the 3D object. A configuration example of the 3D object-related database 244 will be described in detail later.

[0057] (Display control unit 251) The display control unit 251 generates visual information to be displayed by the display unit 250. Further, the display control unit 251 controls the display unit 250 so that the generated visual information is displayed by the display unit 250. For example, the display control unit 251 generates visual information based on various objects arranged in the 3D digital space. The objects may include 2D objects, operation areas, and 3D objects.

[0058] More specifically, the display control unit 251 projects various objects by perspective projection to generate visual information. Note that the visual information generated by the display control unit 251 may also include various buttons. The visual information displayed by the display unit 250 can be visually recognized by the user.

[0059] The configuration example of the 3D object operation processing system 20 according to the embodiment of the present invention has been described above.

[0060] (1-2. Detailed functions of the 3D object operation processing system 20) With reference to FIGS. 3 to 19 (and also referring to FIGS. 1 and 2 as appropriate), the detailed functions of the 3D object operation processing system 20 according to the embodiment of the present invention will be described.

[0061] The 2D object management unit 220 generates 2D objects and arranges the generated 2D objects in the 3D digital space. Then, the 2D object management unit 220 registers information about the generated 2D objects in the 2D object-related database 224. With reference to FIG. 3, a configuration example of the 2D object-related database 224 will be described.

[0062] FIG. 3 is a diagram showing a configuration example of the 2D object-related database 224. As shown in FIG. 3, the 2D object-related database 224 is configured by associating the name, position, orientation, scale factor, size, model data, and material data constituting the 2D object of the 2D object.

[0063] The name of the 2D object is a name for identifying the 2D object. For example, the name of the 2D object may be given by the user, or a name that has not been used as the name of other 2D objects may be automatically given by the 2D object management unit 220.

[0064] The position of the 2D object is the three-dimensional position of the 2D object in the 3D digital space. That is, the position of the 2D object can be expressed by the three-dimensional coordinates (x, y, z) in the world coordinate system XYZ that defines the 3D digital space. For example, the position of the 2D object may be the representative position of the 2D object. For example, the representative position of the 2D object may be the center of gravity position of the 2D object. For example, the position of the 2D object may be specified by the user or may be determined in advance.

[0065] The posture of the 2D object is the posture of the 2D object in the 3D digital space. That is, the posture of the 2D object can be represented by three-dimensional coordinates (θx, θy, θz) in the world coordinate system XYZ that represents the 3D digital space. For example, the posture of the 2D object may be specified by the user or may be predetermined. More specifically, the predetermined posture of the 2D object may be a posture that aligns the coordinate system (hereinafter, also referred to as the "operation area coordinate system") based on the posture of the operation area 30 in the 3D digital space with the world coordinate system XYZ.

[0066] The size of the 2D object is the size of the 2D object in the 3D digital space. For example, when the 2D object has a rectangular shape, the size of the 2D object may be represented by a combination (w, h) of the width of the rectangle and the height of the rectangle. For example, the size of the 2D object may be specified by the user or may be predetermined.

[0067] The model data of the 2D object is information regarding the shape of the 2D object in the 3D digital space. For example, the model data of the 2D object may be data indicating the three-dimensional positions of the vertices, sides, and faces that constitute the 2D object. For example, the model data of the 2D object may be specified by the user or may be predetermined.

[0068] The material data of the 2D object is data including reference information of a texture that is image data pasted on the model of the 2D object, the color of the surface of the 2D object, and the texture of the surface. For example, the material data of the 2D object may be specified by the user or may be predetermined.

[0069] With reference to FIG. 3, a configuration example of the 2D object-related database 224 has been described.

[0070] When the input processing unit 211 determines that the input unit 210 has received an operation of selecting an operation area generation button (not shown), the operation area management unit 230 generates an operation area and arranges the generated operation area in the 3D digital space. Then, the operation area management unit 230 registers information regarding the generated operation area in the operation area related database 234. With reference to FIG. 4, a configuration example of the operation area related database 234 will be described.

[0071] FIG. 4 is a diagram showing a configuration example of the operation area related database 234. As shown in FIG. 4, the operation area related database 234 is configured by associating an ID of the operation area, a name, a position, an orientation, a scale factor, a size, and material data constituting the operation area.

[0072] The ID of the operation area is identification information for identifying the operation area. For example, the ID of the operation area may be assigned by the user, or an ID not used as the ID of another operation area may be automatically assigned by the operation area management unit 230.

[0073] The name of the operation area is a name for identifying the operation area. For example, the name of the operation area may be assigned by the user, or a name not used as the name of another operation area may be automatically assigned by the operation area management unit 230.

[0074] The position of the operation area is the three-dimensional position of the operation area in the 3D digital space. That is, the position of the operation area can be expressed by three-dimensional coordinates (x, y, z) in the world coordinate system XYZ that defines the 3D digital space. For example, the position of the operation area may be the representative position of the operation area. For example, the representative position of the operation area may be the centroid position of the operation area. For example, the position of the operation area may be specified by the user, or may be predetermined at the origin position of the world coordinate system XYZ or the like.

[0075] The posture of the operation area is the posture of the operation area in the 3D digital space. That is, the posture of the operation area can be represented by three-dimensional coordinates (θx, θy, θz) in the world coordinate system XYZ that defines the 3D digital space. For example, the posture of the operation area may be specified by the user or may be predetermined.

[0076] The scale factor (first factor) of the operation area is the factor (enlargement factor or reduction factor) with respect to the initial size of the operation area in the 3D digital space. For example, when the operation area has a rectangular parallelepiped shape, it may be expressed by a combination (sx, sy, sz) of the factor (sx) with respect to the width (w) of the rectangular parallelepiped, which is the initial size of the operation area, the factor (sy) with respect to the height (h) of the rectangular parallelepiped, and the factor (sz) with respect to the depth (d) of the rectangular parallelepiped. For example, the scale factor of the operation area may be specified by the user or may be predetermined.

[0077] The size of the operation area is the size of the operation area in the 3D digital space. For example, when the operation area has a rectangular parallelepiped shape, the size of the operation area may be expressed by a combination (w, h, d) of the width (w) of the rectangular parallelepiped, the height (h) of the rectangular parallelepiped, and the depth (d) of the rectangular parallelepiped. For example, the size of the operation area may be specified by the user or may be predetermined.

[0078] The material data of the operation area may be information for visualizing the operation area. For example, when visualization is performed by coloring the operation area, the material data of the operation area may include the color of the surface of the operation area. Further, when visualization is performed by imparting a grid-like representation, the material data of the operation area may include the number of grids indicating how many grid lines are used to impart the grid-like representation.

[0079] With reference to FIG. 4, a configuration example of the 2D object-related database 224 has been described. Subsequently, with reference to FIG. 5, an example of the operation area 30 will be described.

[0080] FIG. 5 is a diagram showing an example of the operation area 30. Referring to FIG. 5, an example of the operation area 30 arranged in the 3D digital space is shown. In the example shown in FIG. 5, the operation area 30 has a rectangular parallelepiped shape, and a grid-like representation is given to the operation area 30. Note that the xyz coordinate system shown in FIG. 5 is the operation area coordinate system. Subsequently, with reference to FIG. 6, the operations on the operation area 30 will be described.

[0081] FIG. 6 is a diagram for explaining the operations on the operation area 30. Referring to FIG. 6, the operation area 30 is shown. In FIG. 6, a 3D object 31 is shown, but here, the situation before the 3D object 31 is included in the operation area 30 will be described.

[0082] When the input processing unit 211 determines that the input unit 210 has received an operation for selecting the operation area 30, the operation area management unit 230 transitions the state of the operation area 30 to the selected state, and based on the position, orientation, and size of the operation area 30, as shown in the display example 1002, various operation objects for the operation area 30 are arranged in the 3D digital space.

[0083] Display examples 1002a to 1002d are diagrams showing various operation objects for the operation area 30 decomposed. In display example 1002a, arrows x, y, and z that coincide with the axes of the operation area coordinate system xyz and start from the surface of the operation area 30 are shown as position control objects. In display example 1002b, arcs a to c around the axes of the operation area coordinate system xyz are shown as attitude control objects.

[0084] Also, in display example 1002c, arrows x, y, and z that coincide with the axes of the operation area coordinate system xyz and start from the center of gravity position of the operation area 30 are shown as scale magnification specification objects. In display example 1002d, round objects arranged at each vertex and the midpoint of each side of the operation area 30 are shown as size change objects. Operations on the operation area 30 can be performed by a drag operation on these operation objects.

[0085] For example, when the input unit 210 receives a drag operation on the arrow x in the display example 1002a, the input processing unit 211 determines that a position control operation on the operation area 30 in the x direction has been performed. At this time, the operation area management unit 230 may change the position of the operation area 30 by an amount corresponding to the drag operation amount in the x direction.

[0086] In addition, when the input unit 210 receives a drag operation on the arrow y in the display example 1002a, similar processing corresponding to the arrow y may be executed. Also, when the input unit 210 receives a drag operation on the arrow z in the display example 1002a, similar processing corresponding to the arrow z may be executed.

[0087]

[0088]

[0089] ​​Also, for example, when the input unit 210 receives a drag operation on the arrow x in the display example 1002c, the input processing unit 211 determines that a scale magnification specifying operation on the operation area 30 in the x direction has been performed. At this time, the operation area management unit 230 may change the scale magnification of the operation area 30 in the x direction by an amount corresponding to the drag operation amount. The operation area management unit 230 calculates a new size of the operation area 30 in the x direction based on the multiplication of the changed scale magnification of the operation area 30 in the x direction and the initial size of the operation area 30 in the x direction, and changes the size of the operation area 30 in the x direction to the calculated new size.

[0090] In addition, when the input unit 210 receives a drag operation on the arrow y in the display example 1002c, similar processing corresponding to the arrow y may be executed. Also, when the input unit 210 receives a drag operation on the arrow z in the display example 1002c, similar processing corresponding to the arrow z may be executed.

[0091] Also, for example, when the input unit 210 receives a drag operation on the circular object in the display example 1002d, the input processing unit 211 determines that a size change operation on the operation area 30 has been performed. At this time, the operation area management unit 230 may move the circular object on which the drag operation has been performed according to the drag operation, and change the size of the operation area 30 according to the position of the moved circular object.

[0092] For example, when the input unit 210 receives a drag operation on a vertex, the operation area management unit 230 may move the vertex on which the drag operation has been performed according to the drag operation, and regenerate a rectangular parallelepiped shape with the moved vertex as one of the new vertices. Then, the operation area management unit 230 may use the size of the regenerated rectangular parallelepiped shape as the size of the changed operation area 30.

[0093] Alternatively, when the input unit 210 receives a drag operation on the midpoint of a side, the operation area management unit 230 may move the midpoint of the side on which the drag operation is performed according to the drag operation, and regenerate a rectangular parallelepiped shape with the moved midpoint of the side as one of the new midpoints of the sides. Then, the operation area management unit 230 may use the size of the regenerated rectangular parallelepiped shape as the size of the changed operation area 30.

[0094] Note that when the input processing unit 211 determines that the input unit 210 has received a selection operation on a position where neither the operation object for the operation area 30 nor the operation area 30 exists, the operation area management unit 230 may cancel the selection state of the operation area 30.

[0095] When the input processing unit 211 determines that the input unit 210 has received an operation of selecting a 3D object generation button (not shown) in the selected state of the operation area 30, the 3D object management unit 240 generates a 3D object and arranges the generated 3D object inside the selected operation area 30. Then, the 3D object management unit 240 registers information about the generated 3D object in the 3D object related database 244.

[0096] FIG. 7 is a diagram showing an example of the relationship between the operation area and the 3D object. As shown in FIG. 7, one or more 3D objects including the 3D object 31-1a may be included in the operation area 30-1. Similarly, one or more 3D objects including the 3D object 31-2a may be included in the operation area 30-2. Similarly, one or more 3D objects including the 3D object 31-na (n is an integer of 1 or more) may be included in the operation area 30-n.

[0097] In this way, one or more operation areas 30 may be arranged in the 3D digital space, and one or more 3D objects may be included in each of the one or more operation areas 30. Subsequently, the generation of the 3D object will be described with reference to FIGS. 8 and 9.

[0098] FIG. 8 is a diagram showing an example of data stored by the 3D object storage unit 242. Referring to FIG. 8, the data stored by the 3D object storage unit 242 is configured by associating the ID of the generation data, the name, the model data of the 3D object, and the material data constituting the 3D object. The model data of the 3D object and the material data constituting the 3D object may form the generation data.

[0099] The ID of the generation data is identification information for identifying the generation data.

[0100] The name of the generation data is a name for identifying the generation data.

[0101] The model data of the 3D object may be data indicating the three-dimensional positions of the vertices, edges, and faces constituting the 3D object.

[0102] The material data constituting the 3D model is data including a texture that is image data pasted on the model of the 3D object, the color of the surface of the 3D object, and the texture of the surface.

[0103] FIG. 9 is a diagram showing an example of the appearance of a 3D object generated by the generation data. Referring to FIG. 9, examples of the appearance of 3D objects generated by the generation data (that is, the model data and the material data) corresponding to the names "3dm0", "3dm1", "3dm2", and "3dm3" of the generation data are shown.

[0104] For example, the display control unit 251 may control the display unit 250 to display a generation data selection screen including the data stored by the 3D object storage unit 242 (FIG. 8) and the appearance example of the 3D object generated from the generation data (FIG. 9). Then, when the input processing unit 211 determines that an operation for selecting generation data has been received by the input unit 210, the 3D object management unit 240 may generate a 3D object based on the selected generation data. With reference to FIG. 10, a configuration example of the 3D object related database 244 will be described.

[0105] FIG. 10 is a diagram showing a configuration example of the 3D object related database 244. As shown in FIG. 10, the 3D object related database 244 is configured by associating the ID of the 3D object, the name, the ID of the operation area enclosing the 3D object, the position of the 3D object, the orientation, the scale factor, the reference destination information of the model data of the 3D object, and the reference destination information of the material data constituting the 3D object.

[0106] The ID of the 3D object is identification information for identifying the 3D object. For example, the ID of the 3D object may be assigned by the user, or an ID that has not been used as the ID of other 3D objects may be automatically assigned by the 3D object management unit 240.

[0107] The name of the 3D object is a name for identifying the 3D object. For example, the name of the 3D object may be assigned by the user, or a name that has not been used as the name of other 3D objects may be automatically assigned by the 3D object management unit 240.

[0108] The ID of the operation area enclosing the 3D object is identification information for identifying the operation area enclosing the 3D object. For example, the ID of the operation area enclosing the 3D object may be the ID of the selected operation area.

[0109] The position of the 3D object is the three-dimensional position of the 3D object in the operation area. That is, the position of the 3D object can be represented by three-dimensional coordinates (x’, y’, z’) in the operation area coordinate system xyz. For example, the position of the 3D object may be the representative position of the 3D object. For example, the representative position of the 3D object may be the center of gravity position of the 3D object. For example, the position of the 3D object may be specified by the user or may be determined in advance, such as the origin position of the operation area coordinate system xyz.

[0110] The orientation of the 3D object is the orientation of the 3D object in the operation area. That is, the orientation of the 3D object can be represented by three-dimensional coordinates (θx’, θy’, θz’) in the operation area coordinate system xyz. For example, the orientation of the 3D object may be specified by the user or may be determined in advance. More specifically, the predetermined orientation of the 3D object may be an orientation that aligns the coordinate system (hereinafter also referred to as the “3D object coordinate system”) based on the orientation of the 3D object in the 3D digital space with the operation area coordinate system xyz.

[0111] The scale factor (second factor) of the 3D object is the factor (enlargement factor or reduction factor) with respect to the initial size of the 3D object in the 3D digital space. For example, when the 3D object has an ellipsoidal shape, it may be represented by a combination (sx’, sy’, sz’) of the factor (sx’) with respect to the diameter in the x’ direction of the ellipsoid, which is the initial size of the 3D object, the factor (sy’) with respect to the diameter in the y’ direction of the ellipsoid, and the factor (sz’) with respect to the diameter in the z’ direction of the ellipsoid. For example, the scale factor of the 3D object may be specified by the user or may be determined in advance.

[0112] The reference information of the model data of the 3D object is information indicating the reference destination of the model data of the 3D object. For example, the reference information of the model data of the 3D object may be the name of the generation data corresponding to the model data of the 3D object.

[0113] The reference information of the material data of the 3D object is information indicating the reference destination of the material data of the 3D object. For example, the reference information of the material data of the 3D object may be the name of the generation data corresponding to the material data of the 3D object.

[0114] With reference to FIG. 10, a configuration example of the 3D object-related database 244 was described. Subsequently, with reference to FIGS. 11 and 12, an arrangement example of the 3D object will be described.

[0115] FIG. 11 is a diagram showing an example of the 3D digital space after the operation area is arranged and before the 3D object is arranged. Referring to FIG. 11, the 3D digital space 100 is shown. In FIG. 11, the world coordinate system XYZ defining the 3D digital space 100 is shown. Also, a 2D object 101 is arranged in the 3D digital space 100.

[0116] Furthermore, referring to FIG. 11, an operation area 30 is arranged in the 3D digital space 100. Also, an operation area coordinate system xyz based on the posture of the operation area 30 in the 3D digital space 100 is shown.

[0117] FIG. 12 is a diagram showing an example of the 3D digital space after the 3D object is arranged. Referring to FIG. 12, at the origin position of the operation area coordinate system xyz, the 3D object 31 is arranged in a posture such that the 3D object coordinate system x’y’z’ based on the posture of the 3D object 31 in the 3D digital space 100 coincides with the operation area coordinate system xyz. Subsequently, with reference to FIG. 13, operations on the 3D object 31 will be described.

[0118] FIG. 13 is a diagram for explaining operations on the 3D object 31. Referring to FIG. 13, an operation area 30 containing the 3D object 31 is shown.

[0119] When the input processing unit 211 determines that the input unit 210 has received an operation for selecting the 3D object 31, the 3D object management unit 240 changes the state of the 3D object 31 to the selected state, and based on the position, orientation, and size of the 3D object 31, as shown in display example 1202, arranges various operation objects for the 3D object 31 in the 3D digital space.

[0120] Display examples 1202a to 1202c are diagrams showing various operation objects for the 3D object 31 disassembled. In display example 1202a, arrows x, y, and z that coincide with the axes of the operation area coordinate system xyz and start from the surface of the operation area 30 are shown as position control objects.

[0121] For each operation area 30, the 3D object management unit 240 specifies the directions of the arrows x, y, and z respectively as the allowable movement directions of the 3D object 31. Note that the allowable movement directions of the 3D object 31 for each operation area 30 do not have to be limited to three. That is, the allowable movement directions of the 3D object 31 for each operation area 30 may be one or more.

[0122] In display example 1202b, arcs a' to c' around the axes of the 3D object coordinate system x'y'z' are shown as attitude control objects.

[0123] Also, in display example 1202c, arrows x, y, and z that coincide with the axes of the operation area coordinate system xyz and start from the center of gravity position of the operation area 30 are shown as scale factor specification objects.

[0124] For example, when a drag operation on arrow x, arrow y, or arrow z in display example 1202a is received by the input unit 210, it is determined by the input processing unit 211 that a position control operation on the 3D object 31 has been performed. At this time, the 3D object management unit 240 specifies one of the x direction, y direction, and z direction, which is the specified allowable movement direction, as the movement direction, and changes the position of the 3D object 31 in the specified movement direction.

[0125] For example, the 3D object management unit 240 may change the position of the 3D object 31 by changing the relative position of the 3D object 31 with respect to the position of the operation area 30 while restricting the position of the 3D object 31 within the operation area 30. That is, until the 3D object 31 reaches the edge of the operation area 30, the amount of movement of the 3D object 31 may be an amount corresponding to the amount of the drag operation. On the other hand, after the 3D object 31 reaches the edge of the operation area 30, the 3D object 31 may be fixed to the corresponding edge of the operation area 30.

[0126] As an example, the 3D object management unit 240 may specify the movement direction based on the position control operation on the 3D object 31 and the allowable movement directions (x direction, y direction, and z direction). Then, the 3D object management unit 240 may change the position of the 3D object 31 in the specified movement direction.

[0127] For example, when a drag operation on arrow x in display example 1202a is received by the input unit 210, the 3D object management unit 240 may specify the x direction as the movement direction and change the position of the 3D object 31 in the x direction.

[0128] In addition, when a drag operation on arrow y in display example 1202a is received by the input unit 210, similar processing corresponding to arrow y may also be executed. Further, when a drag operation on arrow z in display example 1202a is received by the input unit 210, similar processing corresponding to arrow z may also be executed.

[0129] Also, for example, when a drag operation on the arc a' in the display example 1202b is received by the input unit 210, the input processing unit 211 determines that a posture control operation on the 3D object 31 in the direction along the arc a' has been performed. At this time, the 3D object management unit 240 may change the relative posture of the 3D object 31 based on the posture of the operation area 30 by an amount corresponding to the drag operation amount in the direction along the arc a'.

[0130] In addition, when a drag operation on the arc b' in the display example 1202b is received by the input unit 210, similar processing corresponding to the arc b' may be executed. Also, when a drag operation on the arc c' in the display example 1202b is received by the input unit 210, similar processing corresponding to the arc c' may be executed.

[0131] Also, for example, when a drag operation on the arrow x in the display example 1202c is received by the input unit 210, the input processing unit 211 determines that a scale magnification specifying operation on the 3D object 31 in the x direction has been performed. At this time, the operation area management unit 230 may change the scale magnification of the 3D object 31 in the x direction by an amount corresponding to the drag operation amount. The operation area management unit 230 calculates a new size of the 3D object 31 in the x direction based on the multiplication of the changed scale magnification of the 3D object 31 in the x direction and the initial size of the 3D object 31 in the x direction, and changes the size of the 3D object 31 in the x direction to the calculated new size.

[0132] In addition, when a drag operation on the arrow y in the display example 1202c is received by the input unit 210, similar processing corresponding to the arrow y may be executed. Also, when a drag operation on the arrow z in the display example 1202c is received by the input unit 210, similar processing corresponding to the arrow z may be executed.

[0133] Next, returning to FIG. 6, a case will be described in which the input processing unit 211 determines that an operation to select the operation area 30 has been received by the input unit 210 after the 3D object 31 is included in the operation area 30. At this time, the operation area management unit 230 transitions the state of the operation area 30 to the selected state, and based on the position, orientation, and size of the operation area 30, as shown in the display example 1002, various operation objects for the operation area 30 are arranged in the 3D digital space.

[0134] For example, when a drag operation on the arrow x in the display example 1002a is received by the input unit 210, the input processing unit 211 determines that a position control operation in the x direction for the operation area 30 has been performed. At this time, the operation area management unit 230 may specify the movement direction as the x direction based on the position control operation in the x direction for the operation area 30 and the movement allowable directions (x direction, y direction, and z direction). Then, the operation area management unit 230 may change the position of the operation area 30 in the specified movement direction, which is the x direction.

[0135] The operation area management unit 230 may change the position of the operation area 30 by an amount corresponding to the drag operation amount in the specified movement direction, which is the x direction. At this time, the 3D object management unit 240 may change the position of the 3D object 31 in the 3D digital space by an amount corresponding to the drag operation amount in the movement direction, which is the x direction, by maintaining the relative position of the 3D object 31 with respect to the orientation of the operation area 30.

[0136] Note that when a drag operation on the arrow y in the display example 1002a is received by the input unit 210, similar processing corresponding to the arrow y may be executed. Also, when a drag operation on the arrow z in the display example 1002a is received by the input unit 210, similar processing corresponding to the arrow z may be executed.

[0137] Further, for example, when a drag operation on the arc a in the display example 1002b is received by the input unit 210, it is determined by the input processing unit 211 that a posture control operation on the operation area 30 in the direction along the arc a has been performed. The operation area management unit 230 may change the posture of the operation area 30 by an amount corresponding to the drag operation amount in the direction along the arc a based on the posture control operation.

[0138] At this time, even if the posture of the operation area 30 in the 3D digital space changes in the direction along the arc a, the 3D object management unit 240 may change the posture of the 3D object 31 in the 3D digital space in the direction along the arc a by maintaining the relative posture of the 3D object 31 with respect to the posture of the operation area 30. Further, the 3D object management unit 240 may specify each axis of the changed operation area coordinate system xyz as a movement allowable direction based on the posture of the changed operation area 30.

[0139] In addition, when a drag operation on the arc b in the display example 1002b is received by the input unit 210, similar processing corresponding to the arc b may be executed. Also, when a drag operation on the arc c in the display example 1002b is received by the input unit 210, similar processing corresponding to the arc c may be executed.

[0140] Further, for example, when a drag operation on the arrow x in the display example 1002c is received by the input unit 210, it is determined by the input processing unit 211 that a scale magnification specifying operation on the operation area 30 in the x direction has been performed. At this time, the operation area management unit 230 may change the scale magnification of the operation area 30 in the x direction by an amount corresponding to the drag operation amount. Further, the operation area management unit 230 may calculate a new size of the operation area 30 in the x direction based on the multiplication of the changed scale magnification of the operation area 30 in the x direction and the initial size of the operation area 30 in the x direction, and change the size of the operation area 30 in the x direction to the calculated new size.

[0141] The 3D object management unit 240 calculates the new size of the 3D object 31 in the x direction based on the multiplication of the scale factor after the change of the operation area 30 in the x direction, the scale factor of the 3D object 31 in the x direction, and the initial size of the 3D object 31 in the x direction, and the size of the 3D object 31 in the x direction may be changed to the calculated new size.

[0142] In addition, when a drag operation on the arrow y in the display example 1002c is received by the input unit 210, similar processing corresponding to the arrow y may be executed. Also, when a drag operation on the arrow z in the display example 1002c is received by the input unit 210, similar processing corresponding to the arrow z may be executed.

[0143] In addition, the processing executed based on the drag operation on the circular object in the display example 1002d is the same regardless of whether the 3D object 31 is included in the operation area 30. Therefore, the description of the processing executed based on the drag operation on the circular object in the display example 1002d when the 3D object 31 is included in the operation area 30 is omitted.

[0144] When the input processing unit 211 determines that a selection operation on a position where neither the operation object for the 3D object 31 nor the 3D object 31 exists is received by the input unit 210, the operation area management unit 230 may cancel the selection state of the 3D object 31.

[0145] As described above, in the embodiment of the present invention, the posture of the 3D object 31 can be changed by an operation on the 3D object 31. And the allowable movement direction of the 3D object 31 can be changed by an operation on an operation area 30 different from the 3D object 31. That is, the posture of the 3D object 31 and the allowable movement direction of the 3D object 31 can be changed by different operations.

[0146] Therefore, while using a system with a simple configuration, the user can arbitrarily set the combination of the posture and the moving direction of the 3D object 31. For example, while using a system with a simple configuration, the user can move the 3D object 31 while arbitrarily maintaining the relative posture of the 3D object 31 based on the posture and position of the 2D object arranged in the 3D digital space.

[0147] Referring to FIGS. 14 to 17, the state in which the combination of the posture and the moving direction of the 3D object 31 is arbitrarily set will be described. FIG. 14 is a diagram showing the state immediately after the 3D object 31 is arranged inside the operation area 30. Referring to FIG. 14, the operation area 30 is arranged in the 3D digital space 100, and the 3D object 31 is arranged inside the operation area 30. Immediately after the 3D object 31 is arranged inside the operation area 30, the operation area coordinate system xyz and the 3D object coordinate system x’y’z’ coincide with each other.

[0148] FIG. 15 is a diagram showing the state in which the posture of the 3D object 31 is changing due to the operation on the 3D object 31 from the state shown in FIG. 14. Further, along with the change in the posture of the 3D object 31, the 3D object coordinate system x’y’z’ is also changing. However, the posture of the operation area 30 is not changing. That is, the user can maintain the moving direction of the 3D object 31 specified according to the posture of the operation area 30.

[0149] FIG. 16 is a diagram showing the state in which the postures of the operation area 30 and the 3D object 31 are changing due to the operation on the operation area 30 from the state shown in FIG. 14. Further, along with the change in the postures of the operation area 30 and the 3D object 31, the operation area coordinate system xyz and the 3D object coordinate system x’y’z’ are also changing. In this way, the user can also change the posture of the 3D object 31 and the moving direction of the 3D object 31 in the same direction.

[0150] FIG. 17 is a diagram showing a state in which the posture of the 3D object 31 has changed by an operation on the object 31 from the state shown in FIG. 16. Further, as the posture of the 3D object 31 changes, the 3D object coordinate system x’y’z’ also changes. However, the posture of the operation area 30 has not changed. That is, the user can maintain the moving direction of the 3D object 31 specified according to the posture of the operation area 30.

[0151] FIG. 18 is a diagram showing a state in which various objects are arranged in the 3D digital space 100. Referring to FIG. 18, in the 3D digital space 100, there are a 2D object 101, an operation area 30-1, 3D objects 31-1a and 3D objects 31-1b existing inside the operation area 30-1, an operation area 30-2, and 3D objects 31-2a and 3D objects 31-2b existing inside the operation area 30-2.

[0152] FIG. 19 is a diagram showing an example of visual information generated based on various objects arranged in the 3D digital space 100 shown in FIG. 18. Referring to FIG. 19, visual information 802 is shown. The visual information 802 can be generated by projecting the 2D object 101, the operation area 30-1, the 3D objects 31-1a and 3D objects 31-1b existing inside the operation area 30-1, the operation area 30-2, and the 3D objects 31-2a and 3D objects 31-2b existing inside the operation area 30-2 by perspective projection.

[0153] The functional details of the 3D object operation processing system 20 according to the embodiment of the present invention have been described above.

[0154] (1-3. Operation Example of 3D Object Operation Processing System 20) Next, with reference to FIGS. 20 to 23, an operation example of the 3D object operation processing system 20 according to an embodiment of the present invention will be briefly organized. Note that the processes shown by the flowcharts shown in FIGS. 20 to 23 can be executed in parallel. FIG. 20 is a flowchart showing an example of a visual information update process executed by the 3D object operation processing system 20 according to an embodiment of the present invention.

[0155] As shown in FIG. 20, the display control unit 251 generates visual information to be displayed by the display unit 250 (S101). For example, the display control unit 251 generates visual information based on various objects arranged in the 3D digital space. The objects may include 2D objects, operation areas, and 3D objects. Further, the display control unit 251 updates the visual information already displayed by the display unit 250 with the generated visual information (S102).

[0156] If the display control unit 251 does not end the visual information update process (``NO'' in S102), the display control unit 251 ends the visual information update process. On the other hand, if the display control unit 251 ends the visual information update process (``YES'' in S102), the display control unit 251 ends the visual information update process.

[0157] FIG. 21 is a flowchart showing an example of an operation area generation process executed by the 3D object operation processing system 20 according to an embodiment of the present invention.

[0158] As shown in FIG. 21, the input processing unit 211 determines whether a user operation has been input to the input unit 210 (S201). If the input processing unit 211 determines that no user operation has been input to the input unit 210 ( "NO" in S201), the operation proceeds to S201. On the other hand, if the input processing unit 211 determines that a user operation has been input to the input unit 210 ( "YES" in S201), it determines whether the user operation is an operation instructing the generation of an operation area (S202). Note that the operation instructing the generation of the operation area may be an operation of selecting an operation area generation button (not shown).

[0159] If it is determined that the user operation is not an operation instructing the generation of the operation area ( "NO" in S202), the operation proceeds to S201. On the other hand, when the operation area management unit 230 determines that the user operation is an operation instructing the generation of the operation area ( "YES" in S202), it generates an operation area (S203) and arranges the generated operation area in the 3D digital space. Then, the operation area management unit 230 registers information regarding the generated operation area in the operation area related database 234.

[0160] If the input processing unit 211 does not end the operation area generation process ( "NO" in S204), the operation proceeds to S201. On the other hand, when the input processing unit 211 ends the operation area generation process ( "YES" in S204), it ends the operation area generation process.

[0161] FIG. 22 is a flowchart showing an example of a 3D object generation process executed by the 3D object operation processing system 20 according to an embodiment of the present invention.

[0162] As shown in FIG. 22, the input processing unit 211 determines whether a user operation has been input to the input unit 210 (S301). If the input processing unit 211 determines that no user operation has been input to the input unit 210 (”NO” in S301), the operation proceeds to S301. On the other hand, if the input processing unit 211 determines that a user operation has been input to the input unit 210 (”YES” in S301), it determines whether the operation area is being selected (S302).

[0163] If the input processing unit 211 determines that the operation area is not being selected (”NO” in S302), the operation proceeds to S301. If the input processing unit 211 determines that the operation area is being selected (”YES” in S302), it determines whether the user operation is an operation instructing the generation of a 3D object (S303). Note that the operation instructing the generation of a 3D object may be an operation of selecting a 3D object generation button (not shown).

[0164] If it is determined that the user operation is not an operation instructing the generation of a 3D object (”NO” in S303), the operation proceeds to S301. On the other hand, if the 3D object management unit 240 determines that the user operation is an operation instructing the generation of a 3D object (”YES” in S303), it generates a 3D object (S304) and arranges the generated 3D object inside the operation area 30 in the selected state. Then, the 3D object management unit 240 registers information regarding the generated 3D object in the 3D object-related database 244.

[0165] If the input processing unit 211 does not end the 3D object generation process (”NO” in S305), the operation proceeds to S301. On the other hand, if the input processing unit 211 ends the 3D object generation process (”YES” in S305), it ends the 3D object generation process.

[0166] FIG. 23 is a flowchart showing an example of an operation area and operation processing for a 3D object executed by a 3D object operation processing system 20 according to an embodiment of the present invention.

[0167] As shown in FIG. 23, the input processing unit 211 determines whether a user operation has been input to the input unit 210 (S401). If the input processing unit 211 determines that no user operation has been input to the input unit 210 ( "NO" in S401), the operation proceeds to S401. On the other hand, if the input processing unit 211 determines that a user operation has been input to the input unit 210 ( "YES" in S401), it determines whether the operation area is being selected (S402).

[0168] If the input processing unit 211 determines that the operation area is not being selected ( "NO" in S402), the operation proceeds to S405. On the other hand, if the input processing unit 211 determines that the operation area is being selected ( "YES" in S402), it determines whether the user operation is an operation on the operation area (S403). If it is determined that the user operation is not an operation on the operation area ( "NO" in S403), the operation proceeds to S401.

[0169] On the other hand, when it is determined that the user operation is an operation on the operation area ( "YES" in S403), the operation area management unit 230 updates information related to the operation area, such as the position, orientation, scale factor, or size of the operation area, based on the operation on the operation area (S404). In particular, by updating the orientation of the operation area, the allowable movement direction of the 3D object changes. The updated information related to the operation area is reflected in the operation area related database 234.

[0170] When the operation proceeds to S405, the input processing unit 211 determines whether the 3D object is being selected (S405).

[0171] If the input processing unit 211 determines that the 3D object is not selected (i.e., "NO" in S405), it proceeds to S401. On the other hand, if the input processing unit 211 determines that the 3D object is selected (i.e., "YES" in S405), it determines whether the user's operation is an operation on the 3D object (S406). If it is determined that the user's operation is not an operation on the 3D object (i.e., "NO" in S406), the operation proceeds to S401.

[0172] On the other hand, if the operation area management unit 230 determines that the user's operation is an operation on the 3D object (i.e., "YES" in S406), based on the operation on the 3D object, it updates information regarding the 3D object, such as the position, orientation, or scale factor of the 3D object (S407). In particular, by updating the orientation of the 3D object, the allowable movement direction of the 3D object does not change. The updated information regarding the 3D object is reflected in the 3D object-related database 244.

[0173] The operation example of the 3D object operation processing system 20 according to the embodiment of the present invention has been described above.

[0174] (1-4. Effects) According to the embodiment of the present invention, the orientation of the 3D object 31 can be changed by an operation on the 3D object 31. And the allowable movement direction of the 3D object 31 can be changed by an operation on the operation area 30 different from the 3D object 31. That is, the orientation of the 3D object 31 and the allowable movement direction of the 3D object 31 can be changed by different operations.

[0175] Therefore, while using a system with a simple configuration, the user can arbitrarily set the combination of the posture and the moving direction of the 3D object 31. For example, while using a system with a simple configuration, the user can move the 3D object 31 while arbitrarily maintaining the relative posture of the 3D object 31 with reference to the posture and position of the 2D object arranged in the 3D digital space.

[0176] The effects achieved by the embodiments of the present invention have been described above.

[0177] (2. Hardware configuration example) Next, a hardware configuration example of the 3D object operation processing system 20 according to an embodiment of the present invention will be described.

[0178] Hereinafter, as a hardware configuration example of the 3D object operation processing system 20 according to an embodiment of the present invention, a hardware configuration example of the information processing apparatus 900 will be described. Note that the hardware configuration example of the information processing apparatus 900 described below is merely an example of the hardware configuration of the 3D object operation processing system 20. Therefore, the hardware configuration of the 3D object operation processing system 20 may have unnecessary configurations deleted from the hardware configuration of the information processing apparatus 900 described below, or new configurations may be added.

[0179] FIG. 24 is a diagram showing the hardware configuration of the information processing apparatus 900 as an example of the 3D object operation processing system 20 according to an embodiment of the present invention. The information processing apparatus 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0180] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operations within the information processing device 900 according to various programs. Also, the CPU 901 may be a microprocessor. The ROM 902 stores programs, arithmetic parameters, etc. used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change as appropriate during the execution. These are interconnected by a host bus 904 composed of a CPU bus, etc.

[0181] The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905. Note that it is not necessarily required to separately configure the host bus 904, the bridge 905, and the external bus 906, and these functions may be implemented on a single bus.

[0182] The input device 908 is composed of input means such as a mouse, keyboard, touch panel, button, microphone, switch, and lever for the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. The user who operates the information processing device 900 can input various data to the information processing device 900 or instruct processing operations by operating this input device 908.

[0183] The output device 909 includes, for example, display devices such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a lamp, and a voice output device such as a speaker.

[0184] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage device 910 is constituted by, for example, an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data.

[0185] The communication device 911 is a communication interface constituted by, for example, a communication device for connecting to a network. Further, the communication device 911 may support either wireless communication or wired communication.

[0186] The hardware configuration example of the 3D object operation processing system 20 according to the embodiment of the present invention has been described above.

[0187] (3. Various Modification Examples) Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and these are naturally understood to belong to the technical scope of the present invention.

[0188] For example, in the above, the case where the visualization of the operation area is always performed based on the information for visualizing the operation area has been described. However, the visualization of the operation area may not always be performed, and whether to perform the visualization of the operation area may be switchable by a predetermined switching operation. The predetermined switching operation may be a GUI operation for selecting whether to perform the visualization of the operation area, or may be a key operation assigned to whether to perform the visualization of the operation area.

[0189] In addition, an action on a 3D object existing inside the operation area may be added. For example, a physical calculation of a force acting on the 3D object may be prepared for each operation area where the 3D object exists. For example, the force acting on the 3D object may be gravity applied to the 3D object, frictional force applied by the 3D object from the end of the operation area, repulsive force, or the like.

[0190] Further, the position and orientation of the 3D object may be affected by determining that the operation area and the 3D object existing inside the operation area are in contact or interference. Also, the positions and orientations of the plurality of 3D objects existing inside the operation area may be affected by determining that the plurality of 3D objects are in contact or interference with each other.

[0191] In the above, it is mainly assumed that operations on the operation area and the 3D object (for example, position control operation, orientation control operation, scale magnification specification operation, etc.) are performed by selecting buttons. However, the operations on the operation area and the 3D object may be performed by a predetermined key operation. Alternatively, some or all of the operations on the operation area and the 3D object may be performed by a predetermined operation using a mouse (for example, a drag operation or a mouse wheel operation).

Description of Reference Numerals

[0192] 20 3D object operation processing system 210 Input unit 211 Input processing unit 220 2D object management unit 222 2D object storage unit 224 2D object related database 230 Operation area management unit 234 Operation area related database 240 3D object management unit 242 3D object storage unit 244 3D object-related database 250 Display unit 251 Display control unit

Claims

1. An operation area management unit that arranges an operation area, which is an area where operations on a three-dimensional object are possible, in a three-dimensional digital space, A three-dimensional object management unit that arranges the three-dimensional object within the operation area, Comprising, The three-dimensional object management unit, Changes the posture of the three-dimensional object in the three-dimensional digital space, Identifies one or more movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changes the position of the three-dimensional object in a movement direction that is one of the movement allowable directions, An information processing apparatus.

2. The operation area management unit changes the posture of the operation area in the three-dimensional digital space based on a posture control operation on the operation area, The three-dimensional object management unit identifies the movement allowable direction based on the posture of the operation area after the change, The information processing apparatus according to Claim 1.

3. The three-dimensional object management unit changes the position of the three-dimensional object in the three-dimensional digital space in the movement direction by maintaining the relative position of the three-dimensional object with respect to the posture of the operation area, The information processing apparatus according to Claim 1.

4. The operation area management unit identifies the movement direction based on a position control operation on the operation area and the movement allowable direction, and changes the position of the operation area in the three-dimensional digital space in the movement direction, The information processing apparatus according to Claim 3.

5. The three-dimensional object management unit changes the position of the three-dimensional object in the three-dimensional digital space in the movement direction by changing the relative position of the three-dimensional object with respect to the position of the operation area while restricting the position of the three-dimensional object within the operation area, The information processing apparatus according to Claim 1.

6. The three-dimensional object management unit identifies the movement direction based on a position control operation on the three-dimensional object and the movement allowable direction, and changes the position of the three-dimensional object in the three-dimensional digital space in the movement direction, The information processing apparatus according to Claim 5.

7. The three-dimensional object management unit, Even if the posture of the operation area in the three-dimensional digital space changes, the posture of the three-dimensional object in the three-dimensional digital space is changed by maintaining the relative posture of the three-dimensional object with respect to the posture of the operation area. The information processing apparatus according to claim 1.

8. The three-dimensional object management unit changes the posture of the three-dimensional object in the three-dimensional digital space by changing the relative posture of the three-dimensional object with respect to the posture of the operation area. The information processing apparatus according to claim 1.

9. The three-dimensional object management unit changes the posture of the three-dimensional object in the three-dimensional digital space based on a posture control operation on the three-dimensional object. The information processing apparatus according to claim 8.

10. The operation area management unit changes the first scale factor of the operation area based on a scale factor designation operation on the operation area, and calculates the size of the operation area based on the multiplication of the changed first scale factor and the initial size of the operation area. The information processing apparatus according to claim 1.

11. The three-dimensional object management unit calculates the size of the three-dimensional object based on the multiplication of the changed first scale factor, the second scale factor of the three-dimensional object, and the initial size of the three-dimensional object. The information processing apparatus according to claim 10.

12. The operation area management unit arranges a plurality of the operation areas in the three-dimensional digital space. The information processing apparatus according to claim 1.

13. The information processing apparatus includes an input unit that receives an operation indicated by two-dimensional coordinates. The information processing apparatus according to any one of claims 1 to 12.

14. Arranging an operation area, which is an area where a three-dimensional object can be operated, in a three-dimensional digital space, placing the three-dimensional object within the operation area, changing the posture of the three-dimensional object in the three-dimensional digital space, identifying one or more movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changing the position of the three-dimensional object in a movement direction that is any one of the movement allowable directions, An information processing method executed by a computer, including.

15. A computer An operation area management unit that arranges an operation area, which is an area where operations on a three-dimensional object are possible, in a three-dimensional digital space; A three-dimensional object management unit that arranges the three-dimensional object within the operation area; Comprising; The three-dimensional object management unit: Changes the posture of the three-dimensional object in the three-dimensional digital space; Specifies one or more movement allowable directions of the three-dimensional object in the three-dimensional digital space for each operation area, and changes the position of the three-dimensional object in a movement direction that is any one of the movement allowable directions; A program that functions as an information processing apparatus.

Images