Computer program, information processing terminal and control method thereof

Abstract

To provide a novel mechanism for operating a three-dimensional model, for example, a figure displayed on a screen.SOLUTION: An information processing terminal receives a user operation via a screen displayed on a display part of the information processing terminal, displays a three-dimensional model of a figure on the display part, and displays movable points of the three-dimensional model in such a manner as to be operable by a user. When receiving a changeover instruction to change the number of movable points to be displayed on the display part, the information processing terminal changes and displays the number of movable points to be displayed on the display part in accordance with the changeover instruction.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing terminal, and a control method thereof. [Background technology]

[0002] A movable figure having many joints is known. In addition, a technology is known in which an object such as a figure is imaged from multiple directions, a three-dimensional model is generated from the multiple captured images, and the model is displayed in a virtual space. Patent Document 1 proposes a technology in which a texture is mapped onto the three-dimensional model generated from the multiple captured images, the model is arranged in a virtual space, and the texture is moved in accordance with the rotational movement and translational movement of the primitive. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-107252 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, the entire three-dimensional model generated from the figure included in the captured image can be rotated or translated on the screen. On the other hand, although the above-mentioned conventional technology can move the entire figure, it does not mention moving the joints, etc. like a real movable figure. However, if the joints, etc. of the three-dimensional model displayed on the screen can be moved like a real movable figure, a more realistic feeling can be given. Furthermore, if the joints, etc. can be moved in response to a user operation, the interest can be further increased.

[0005] The present invention provides a novel mechanism for manipulating three-dimensional models, such as figures, displayed on a screen. [Means for solving the problem]

[0006] The present invention is characterized in that, for example, a computer of an information processing terminal functions as a reception means for receiving user operations via a screen displayed on a display unit of the information processing terminal, and a display control means for displaying a three-dimensional model of a model on the display unit and displaying movable points of the three-dimensional model in an operable manner, and when the display control means receives a switching instruction from the reception means to switch the number of movable points to be displayed on the display unit, it switches and displays the number of movable points to be displayed on the display unit in accordance with the switching instruction.

[0007] The present invention also provides, for example, an information processing terminal comprising a reception means for receiving user operations via a screen displayed on a display unit of the information processing terminal, and a display control means for displaying a three-dimensional model of a model on the display unit and displaying movable points of the three-dimensional model in an operable manner, wherein when the display control means receives a switching instruction to switch the number of movable points to be displayed on the display unit via the reception means, the display control means switches and displays the number of movable points to be displayed on the display unit in accordance with the switching instruction.

[0008] The present invention is also, for example, a control method for an information processing terminal, comprising a reception step of receiving user operation via a screen displayed on a display unit of the information processing terminal, and a display control step of displaying a three-dimensional model of a model on the display unit and displaying movable points of the three-dimensional model in an operable manner, wherein, when a switching instruction for switching the number of movable points to be displayed on the display unit is received in the display control step, the number of movable points to be displayed on the display unit is switched and displayed in accordance with the switching instruction. Effect of the Invention

[0009] According to the present invention, for example, a novel mechanism for manipulating a three-dimensional model such as a figure displayed on a screen can be provided. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of each device according to an embodiment. [Diagram 3] 1A to 1C are diagrams showing an example of screen transition of a digital figure provided by a system according to an embodiment. [Figure 4] 11A and 11B are diagrams illustrating a stepwise increase and decrease of a movable point according to an embodiment. [Diagram 5] 11A to 11C are diagrams illustrating a movable operation of a three-dimensional model according to an embodiment. [Figure 6] FIG. 2 is a diagram showing a sequence of a system according to an embodiment. [Figure 7] 10 is a flowchart showing a processing procedure of basic control according to an embodiment. [Figure 8] 11 is a flowchart showing a processing procedure for pose editing according to an embodiment. [Figure 9] 10 is a flowchart showing a processing procedure of movable display control according to an embodiment. [Figure 10] FIG. 13 is a diagram showing a first modified example of the movable operation of a three-dimensional model according to an embodiment. [Figure 11] FIG. 11 is a diagram showing a second modified example of the movable operation of a three-dimensional model according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features among the multiple features described in the embodiments may be arbitrarily combined. In addition, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] <System configuration> First, the system configuration according to the present embodiment will be described with reference to Fig. 1. Note that a simple configuration with the minimum necessary will be described here, but this does not limit the present invention. For example, each device may include multiple devices, or multiple servers may be integrated.

[0013] This system includes information processing terminals 101a, 101b, and 101c, and an application server 102. In the following, when describing the configuration and control applicable to any of the information processing terminals 101a, 101b, and 101c, the alphabet suffixed to the reference numeral will be omitted and the information processing terminal will simply be referred to as information processing terminal 101. The information processing terminal 101 and the application server 102 are connected to each other via a network 103 so that they can communicate with each other.

[0014] The information processing terminal 101 is, for example, an information processing terminal such as a smartphone, a mobile phone, a tablet PC, or a stand-alone PC. Any device may be used as long as the information processing terminal has at least a communication unit capable of communicating with the application server 102 via a network and a display unit for displaying images. The information processing terminal 101 downloads and installs an application for carrying out the present invention from the application server 102 via a network 103. When the application is executed on the information processing terminal 101, a three-dimensional model of a predetermined model (movable figure) described below is displayed on the screen, and the predetermined model (figure) can be moved on the screen by operating a movable part. A figure that is represented on the screen in such a manner that it can be operated is hereinafter referred to as a digital figure.

[0015] The application server 102 provides an application to the information processing terminal 101. When the application is started on the information processing terminal 101, the application server 102 provides the information processing terminal 101 with figures selectable by the user and pose (posture) information of each figure according to the user's authentication information. The user can purchase digital figures on the application. The application server 102 registers the digital figures purchased by the user in association with the user's authentication information. Therefore, the application server 102 can specify the digital figures that can be provided by referring to the user's authentication information and the above-mentioned registered information linked to the authentication information. Note that the three-dimensional model information of the digital figure is information managed by the application server 102, but may be stored in a local storage area of ​​the information processing terminal 101.

[0016] <Configuration of each device> Next, a configuration example of the information processing terminal 101 and the application server 102 according to this embodiment will be described with reference to Fig. 2. Here, only devices important for explaining the present invention will be described. Therefore, the information processing terminal 101 and the application server 102 may be configured to include other devices instead or in addition.

[0017] First, a description will be given of the configuration of the information processing terminal 101. The information processing terminal 101 includes a CPU 201, a storage unit 202, a communication control unit 203, a display unit 204, an operation unit 205, and a camera 206. Each component can transmit and receive data to and from each other via a system bus 210.

[0018] The CPU 201 is a central processor that performs overall control of each component connected via a system bus 210. The CPU 201 executes each process described below by executing a computer program stored in a storage unit 202. The storage unit 202 is used as a work area or temporary area for the CPU 201, and also stores the control program executed by the CPU 201 and various data.

[0019] The communication control unit 203 can perform bidirectional communication with the application server 102 via the network 103 by broadband wireless communication. Note that the communication control unit 203 may have a short-range wireless communication function such as wireless LAN (WiFi), Bluetooth (registered trademark) communication, and infrared communication in addition to or instead of the broadband wireless communication. For example, when the communication control unit 203 does not have a broadband wireless communication function but has a WiFi communication function, it connects to the network 103 via a nearby access point.

[0020] The display unit 204 may be a touch panel type liquid crystal display, which displays various screens as well as a digital figure selected by the user on the application screen. The operation unit 205 functions as a reception means and is an operation input unit that receives user operations, and may be provided integrally with the display unit 204. The operation unit 205 may also include a physically configured push-type or slide-type button, etc.

[0021] The camera 206 is an imaging unit that captures images of the surrounding environment of the information processing terminal 101. It is also possible to place and operate a digital figure in an image captured by the camera 206. Note that the camera 206 is not an essential component for realizing the present invention. For example, the information processing terminal 101 according to this embodiment is also capable of combining a digital figure with a captured image received from the outside and displaying it.

[0022] Next, a description will be given of the configuration of the application server 102. The application server 102 includes a CPU 211, a ROM 212, a RAM 213, a display unit 214, an operation unit 215, a communication control unit 216, and a hard disk drive (HDD) 217. The CPU 211 is a central processor that performs overall control of each component connected via a system bus 220.

[0023] The CPU 211 executes various processes by loading computer programs stored in the ROM 212 and the HDD 217 into the RAM 213 and executing them. The ROM 212 is a non-volatile memory that stores various programs and various data. The RAM 213 is a volatile memory that is used as a work area or temporary area for the CPU 211. The HDD 217 is a large-capacity storage device that stores user authentication information and registration information for each user linked to the authentication information.

[0024] The communication control unit 216 can perform two-way communication with the information processing terminal 101 via the network 103. The communication control unit 216 connects to the network 103 via a nearby access point by wireless LAN (such as WiFi). The display unit 214 displays various screens. The operation unit 215 is an operation input unit that accepts operations by an administrator of the system, etc. Furthermore, the operation unit 215 may be a pointing device such as a mouse.

[0025] <Digital Figure> Next, an example of a digital figure provided by the system according to this embodiment will be described with reference to Fig. 3. Here, it is assumed that an application providing the digital figure has been downloaded from the application server 102 to the information processing terminal 101 and installed. When the application is executed by the information processing terminal 101, it displays a three-dimensional model on the display unit 204 in an operable manner. Figs. 3(a) to 3(d) show screen transitions in the above application.

[0026] A screen 300 shown in FIG. 3(a) is a figure selection screen displayed on the display unit 204 of the information processing terminal 101 when an application that provides digital figures is started. The figure selection screen 300 includes a selection area 301 in which a plurality of digital figures can be selected, and a decision button 302. The user selects the selection area 303 of the figure that the user wishes to operate on the screen, and operates the decision button 302. In response to the user's operation, the screen transitions from the figure selection screen 300 to a pose selection screen 310. The selectable digital figures are linked to user information such as authentication information and are managed in the application server 102. For example, a new digital figure can be purchased via a screen not shown, and the purchased digital figure is registered in the application server 102 together with the user information, and is added to the selection area 301 of the figure selection screen 300.

[0027] FIG. 3B shows a pose selection screen 310. The pose selection screen 310 is a screen for selecting a default posture when starting to operate a digital figure. The pose selection screen 310 is displayed including a selection area 311 in which multiple poses (postures) can be selected, a back button 312, a decision button 313, a menu button 314, an edit data selection button 316, and a preset selection button 317. The user selects the selection area 315 of the pose he / she wants to customize on the screen and operates the decision button 313. In response to the user's operation, the pose selection screen 310 transitions to an edit screen 320. When the back button 312 is operated, the screen returns to the figure selection screen 300. When the menu button 314 is operated, various setting items are displayed. When the edit data selection button 316 is operated, one or more poses that the user has previously customized by operating the digital figure are displayed in a selectable manner in the selection area 315. On the other hand, when the preset selection button 317 is operated, multiple poses that have been preset for the target figure are displayed in a selectable manner in the selection area 315.

[0028] FIG. 3(c) shows an editing screen 320. In the editing screen 320, the displayed digital figure is displayed so that it can be edited. The editing screen 320 includes a digital figure 350, buttons 323 to 325, a cost display 329, and a posture monitor 328. When the button 323 is operated, the screen transitions to a screen in which the movable points of the digital figure are displayed so that they can be operated. This screen will be described later with reference to FIG. 4 and FIG. 5. When the button 324 is operated, a pop-up display of changeable parts is displayed so that they can be selected. For example, the head, arms, belt, etc., which can be selected for the figure, are displayed so that they can be selected. When the button 327 is operated, various setting items are displayed. When the button 326 is operated, editing of the pose on the editing screen 320 is finished. When the button 327 is operated, the pose of the digital figure being edited is saved to the application server 102. As a save destination, the pose may be saved in a local memory area of ​​the information processing terminal 101 in addition to or instead of the application server 102.

[0029] Cost display 329 indicates the processing cost when the digital figure being edited is placed in a diorama or the like in a virtual space. There is an upper limit to the number of objects that can be placed in the diorama or the like at one time depending on the processing performance of information processing terminal 101 or the like, and the processing cost displayed serves as a guide to this. Orientation monitor 328 displays the orientation of digital figure 350 being displayed on up to three axes. On editing screen 320, the digital figure is facing forward, so it is represented on two axes. Note that by operating orientation monitor 328, digital figure 350 is also rotated and displayed in conjunction with the operation. Meanwhile, by rotating digital figure 350, the display of orientation monitor 328 is updated in conjunction with the operation.

[0030] FIG. 3(d) shows an editing screen 330 in which the display has been updated by rotating digital figure 350 to the right on editing screen 320. On editing screen 330, digital figure 351 is displayed, which is the result of rotating digital figure 350, which faces forward, to the right by a rotation operation. In addition to the display of editing screen 320, editing screen 330 also displays buttons 321 and 322. Buttons 321 and 322 are displayed when a pose is edited from digital figure 350, which is the default pose, by a user operation. Button 321 is a button for canceling the previous editing operation and returning to the previous pose. Button 322 is a button for moving from a state where the pose has been returned to a past pose by selecting button 321 to the next pose stored in the history. Also, when the pose is edited, posture monitor 331 is updated accordingly.

[0031] <Movable points> Next, the movable points of a digital figure according to this embodiment will be described with reference to Fig. 4. Fig. 4(a) to Fig. 4(c) show how the number of movable points of a digital figure is gradually increased or decreased in accordance with user operations. Note that movable points can be set at the joint positions of the figure, the connections of individual parts, etc., but are not limited thereto.

[0032] FIG. 4(a) shows an editing screen 400 that is displayed when the button 323 is operated on the editing screen 330 shown in FIG. 3(d). On the editing screen 400, as shown in 403, a plurality of movable points of the figure 351 are displayed superimposed on the positions of the movable parts of the figure 351. The movable points are displayed as operable marks, for example, in double circles. The operation method will be described later with reference to FIG. 5. On the editing screen 400, a mode selection area 401 and a mode display section 402 are further displayed. The mode selection area 401 is a slide button for gradually increasing or decreasing the number of movable points to be displayed. On the editing screen 400, an indication that the first mode is selected as the display mode is displayed in the left area of ​​the mode selection area 401 and in the mode display section 402. On the editing screen 400, the button 323 is selected and the display of the movable points is started, so the default first mode is selected. According to this embodiment, the display mode of the movable points includes a plurality of modes, for example, a first mode ("easy" mode), a second mode ("so-so" mode), and a third mode ("firm" mode), and the number of displayed movable points is increased or decreased in stages. In the third mode, the same number of movable points as the maximum number of movable parts of the real figure are displayed. Here, the movable parts of the real figure refer to the movable parts set when the digital figure displayed on the screen is assumed to be a real figure. Therefore, the real figure here does not necessarily have to exist as an actual figure, and the movable parts can be set at any position, and may be set based on, for example, design data, etc. However, when the movable points of the digital figure are set based on the movable parts of the design data of the actual figure, the reality of the movement is increased, and a more preferable digital figure can be provided. Note that, although the modes are divided into three stages here, there is no intention to limit the present invention, and they may be divided into two modes or even more modes.

[0033] FIG. 4(b) shows an editing screen 410 to which the screen transitions when the second mode is selected in the mode selection area 401. The user can select the second mode by sliding the screen from the left area of ​​the mode selection area 401 to near the center on the editing screen 400. This operation corresponds to a switching instruction. In the editing screen 410, compared to the editing screen 400, a movable point surrounded by a dotted line is newly added and displayed, and the editing screen 410 includes a mode selection area 411 and a mode display section 412. In the editing screen 410, an indication indicating that the second mode has been selected as the display mode is displayed near the center of the mode selection area 411 and in the mode display section 412.

[0034] FIG. 4(c) shows an editing screen 420 to which the transition occurs when the third mode is selected in the mode selection area 411. The user can select the third mode by performing a sliding operation from near the center of the mode selection area 411 to the right area in the editing screen 410. This operation corresponds to a switching instruction. In the editing screen 420, compared to the editing screen 410, a movable point surrounded by a dotted line is newly and additionally displayed, and the editing screen 420 includes a mode selection area 421 and a mode display section 422. In the editing screen 420, an indication indicating that the third mode has been selected as the display mode is displayed in the right area of ​​the mode selection area 421 and in the mode display section 422.

[0035] Among the movable points newly added and displayed on the editing screen 420, there are movable points displayed as simple circle marks instead of double circles. This indicates that the movable types are different from those of the movable points displayed as double circles. The movable types include, for example, movable points (double circles) that can rotate in multiple directions by a spherical connecting part, and movable points (single circles) that can rotate only in a single direction (for example, only up and down). Thus, according to this embodiment, different movable types may be represented depending on the type (shape, color, size, etc.) of the object indicating the movable point. This allows the user to easily recognize the movable type and the movable direction of each movable point.

[0036] In the above example, an example in which the number of movable points to be displayed is increased stepwise has been described. On the other hand, it is also possible to decrease the number stepwise. For example, it is also possible to shift from the third mode to the second mode, or from the second mode to the first mode. Also, as will be described later, it is also possible to shift to another mode after performing a movable operation in a predetermined mode. Also, in the above example, an example in which the number of movable points is increased or decreased has been described, but the number of movable points may be the same, and multiple modes in which the positions of the movable points are different may be switched. Alternatively, multiple modes in which the number of movable points is increased or decreased while the positions of the movable points are different may be used.

[0037] <How to operate digital figures> Next, a method for operating a digital figure according to this embodiment will be described with reference to Fig. 5. Here, as an example, as shown in an editing screen 400, screen transitions according to operations when the first mode is selected as the display mode for movable points will be described.

[0038] FIG. 5(a) shows an editing screen 500 in a state where the user has selected the movable point of the right ankle of the digital figure 351. 501 indicates an object that highlights the selected movable point. The object 501 is displayed superimposed over the selected movable point. The selected movable point is also displayed visible through the object 501. This allows the user to easily recognize which movable point is selected. When any of the movable points is selected, buttons 502 and 503 are additionally displayed. The button 502 is a button for reducing the display scale of the digital figure 351. On the other hand, the button 503 is a button for enlarging the display scale of the digital figure 351. When the display scale is enlarged, the enlarged display is performed with the selected movable point at the center.

[0039] FIG. 5(b) shows an editing screen 510 in which, with a movable point selected as shown in 501, an operation is performed to rotate the right leg of the figure 351 backward, and the display is updated. In the editing screen 510, the display of the figure 351 is updated to show the state in which the right leg has moved backward. As shown in 511, the display of the object indicating the selected movable point is also moved and displayed accordingly. As a user operation, for example, with a predetermined movable point pressed, a slide operation (swipe operation, flick operation, etc.) is performed in a predetermined direction, and the part related to the selected movable point is also moved and displayed in the predetermined direction. Alternatively, it may be realized by performing a touch operation to select a predetermined movable point, releasing the touch operation once, and then performing a slide operation (swipe operation, flick operation, etc.) on the screen in the direction in which the selected movable point is to be rotated as the center. The range in which the related parts of the digital figure 351 are rotated may be changed according to the operation width and operation speed of the slide operation.

[0040] FIG. 5(c) shows an edit screen 520 in which the right leg of the figure 351 is rotated backward from the state of FIG. 5(b) and the display is updated. In the edit screen 520, the display of the figure 351 is updated to show the state in which the right leg has moved further backward. As shown in 521, the display of the object indicating the selected movable point is also moved and displayed accordingly. As shown in FIG. 5(b) and FIG. 5(c), the selected movable point is the position of the right ankle, and the digital figure can be moved so that the movable point moves in the direction of the operation by sliding backward. Furthermore, according to this embodiment, not only the lower leg part that abuts against the movable point moves, but the upper leg part also moves in accordance with the movement of the lower leg part. Thus, according to the method of operating a digital figure according to this embodiment, display control is performed in response to a user operation so that the parts related to the operated movable point and the parts following the parts move in conjunction with each other. This allows the user to perform a movable operation more intuitively, similar to a real figure. It is preferable that the moving display is dynamically updated following the user's slide operation. On the other hand, in order to reduce the processing load, the display is not updated during the operation (when the finger is in contact with the screen), but is updated when the user completes the slide operation, i.e., when the finger is released from the screen, by determining the moving distance based on the release position.

[0041] In FIG. 5, the screen transition when the first mode is selected as the display mode of the movable points is described. However, when the second or third mode is selected as the display mode of the movable points, the figure 351 can be operated in the same manner as in FIG. 5 based on the movable points shown in FIG. 4(b) or FIG. 4(c). When the second mode is selected, for example, the movable points of the toes are increased, and the position (angle) of the toes can be freely moved. In this embodiment, since the settings related to the movable points can be changed, when the user is unfamiliar with the operation of the digital figure, the first mode with a small number of movable points is set, and when the user becomes accustomed to the operation, the second or third mode with a large number of movable points is changed. In this way, the movable points can be set according to the user's proficiency and preferences regarding operation, and the convenience of the system can be improved.

[0042] <Sequence> Next, the sequence between information processing terminal 101 and application server 102 when realizing a digital figure according to this embodiment will be described with reference to Fig. 6. Here, it is assumed that an application providing the digital figure has been downloaded from application server 102 to information processing terminal 101 and installed. Also, the operation of manipulating the digital figure to edit its pose and saving the edited pose in the server will be described.

[0043] When an operation to start an application is accepted, the information processing terminal 101 starts the application in S1. When the application is started, the application notifies the application server 102 of a user's login request together with the user's authentication information in S2. The authentication information may be input information accepted from the user when the application is started for the first time, and may be used the next time the application is started or may be accepted from the user every time the application is started.

[0044] In S3, the application server 102 authenticates the requested user, and if the authentication is successful, returns user-related information related to the user to the information processing terminal 101. The user-related information includes selectable figure information and selectable pose information for each figure, which are linked to the user's authentication information and registered. Upon receiving the user-related information, the application displays a figure selection screen 300 on the display unit 204 based on the user's selectable figure information.

[0045] Next, in S4, the application accepts the selection of a figure via the figure selection screen 300, and displays the pose selection screen 310 on the display unit 204 based on the pose information received in S3. Note that, at the stage where the user selects a figure desired via the figure selection screen 300, the information processing terminal 101 may send a request to the application server 102 to receive pose information related to that figure. This makes it possible to reduce memory usage in the information processing terminal 101, compared to the case where pose information for all figures is received in advance in S3.

[0046] Next, in S5, the application accepts the selection of a pose for the figure via the pose selection screen 310, and requests three-dimensional model data representing the digital figure of the figure from the application. In S7, the application server 102 returns the requested three-dimensional model data of the digital figure to the information processing terminal 101. Note that the three-dimensional model data transmitted here may be data specific to the selected pose, or may be data of a predetermined reference pose. When transmitting a predetermined reference pose, the information processing terminal 101 needs to perform control to change and display the three-dimensional model in accordance with the selected pose.

[0047] When the three-dimensional model data is received, the application displays the editing screen 320 on the display unit 204 based on the received data in S8. Thereafter, when the button 323 is operated, the application displays the editing screens 400, 500 displaying the movable points in an operable manner in S9. Furthermore, when a mode selection or an operation of the movable points is accepted, the application updates the display of the digital figure displayed on the editing screens 400, 500.

[0048] Thereafter, when the button 325 is operated, in S10 the application notifies the application server 102 of a request to save the current pose state of the digital figure. In S11, the application server 102 registers the information in association with the user's authentication information, and sends a response to the information processing terminal 101 indicating that saving has been completed.

[0049] <Basic control> Next, a basic control process procedure in an application that provides a digital figure according to this embodiment will be described with reference to Fig. 7. The process described below is realized, for example, by the CPU 201 reading and executing a control program or the like that is pre-stored in the storage unit 202.

[0050] First, in S101, when an application that provides digital figures according to this embodiment is started, the CPU 201 displays a figure selection screen 300 on the display unit 204. When an arbitrary figure is selected via the figure selection screen 300, in S102 the CPU 201 displays a pose selection screen 310 on the display unit 204. When an arbitrary pose is selected via the pose selection screen 310, in S103 the CPU 201 notifies the application server 102 of the pose information of the selected figure and obtains three-dimensional model data of the figure. The three-dimensional model data also includes information regarding the movable points that can be displayed.

[0051] Next, in S104, the CPU 201 displays an editing screen for the digital figure on the display unit 204 in accordance with the three-dimensional model data acquired in S103. When a user operation is accepted via the displayed editing screen, in S105 the CPU 201 determines whether or not the button 323 for displaying the movable points has been operated. If it is determined that the button has been operated, the process proceeds to S109, otherwise, the process proceeds to S106. In S109, the CPU 201 displays the movable points with the default number of display points and accepts the editing operation. Detailed control of S109 will be described later with reference to FIG. 8.

[0052] On the other hand, in S106, the CPU 201 judges whether the button 325 for saving the pose has been operated. If it is determined that it has been operated, the process proceeds to S107, otherwise, the process proceeds to S108. In S107, the CPU 201 notifies the application server 102 of a save instruction request together with posture information indicating the current pose state, and the process proceeds to S108.

[0053] In S108, the CPU 201 determines whether the application has been terminated. If the application has not been terminated, the process returns to S105, and if the application has been terminated, the process of this flowchart is terminated.

[0054] <Display control of movable points> Next, the processing procedure of the pause editing control (S109) according to this embodiment will be described with reference to Fig. 8. The processing described below is realized, for example, by the CPU 201 reading and executing a control program or the like stored in advance in the storage unit 202.

[0055] First, in S201, the CPU 201 acquires information about the movable points from the three-dimensional model data acquired in S103. Next, in S202, the CPU 201 displays the movable points superimposed on the digital figure in the default display mode for the movable points (here, the first mode) in accordance with the acquired information about the movable points.

[0056] Next, in S203, the CPU 201 determines whether the number of movable points to be displayed has been changed via the mode selection area 401. If it has been changed, the process proceeds to S207, and if not, the process proceeds to S204. In S207, the CPU 201 increases or decreases the number of movable points to be displayed according to the changed display mode of the movable points, and the process proceeds to S204.

[0057] In S204, the CPU 201 judges whether or not the movable point has been operated. If so, the process proceeds to S205, otherwise to S206. In S205, the CPU 201 rotates the digital figure according to the user's operation information to update the display, and the process proceeds to S206. The detailed process will be described later with reference to FIG. 9. In S206, the CPU 201 judges whether or not the button 325 or the button 326 has been operated, and judges whether or not the editing is to be ended. If the editing is to be ended, the process of this flowchart is ended and the process proceeds to S106, otherwise, the process returns to S203. Here, it is judged that the editing is to be ended when the button (save instruction) 325 is operated, but the editing may be continued without being ended. In that case, the same save process as in S107 is executed, and the process returns to S203.

[0058] <Movable display control> Next, the processing procedure of the movable display control (S205) according to this embodiment will be described with reference to Fig. 9. The processing described below is realized, for example, by the CPU 201 reading and executing a control program or the like stored in advance in the storage unit 202.

[0059] First, in S301, the CPU 201 acquires operation information related to the selected movable point. Next, in S302, the CPU 201 identifies related parts of the digital figure according to the selected movable point. For example, when the movable point of the right ankle in Fig. 5 is selected, the parts connected to the movable part corresponding to the movable point, for example, the parts of the lower leg and foot, are identified as related parts.

[0060] Next, in S303, the CPU 201 determines the motion of the identified related part according to the operation information. Here, for example, the motion direction is determined according to the operation direction, and the motion width (movement distance of the part) is determined according to the operation width and operation speed. Next, in S304, the CPU 201 determines the motion of the part that follows the motion of the related part according to the determined motion of the related part. The determination method may be determined using a learned algorithm that has been machine-learned to output the motion of the part that follows according to the motion direction and motion width of the part related to each joint. Alternatively, the determination may be made by referring to a table that defines the motion pattern of the part that follows according to the motion of each part.

[0061] When the movements of the associated parts and the following parts are determined, in S305 the CPU 201 updates the display of the digital figure to display the pose after the movement, and ends the processing of this flowchart. The display may be dynamically updated following the user's slide operation, or the display may not be updated during the operation (when the finger is touching the screen), but may be updated when the user completes the operation, i.e., when the finger is released from the screen.

[0062] As described above, the information processing terminal 101 according to this embodiment accepts user operations via a screen displayed on the display unit of the information processing terminal, displays a three-dimensional model of a model on the display unit, and displays movable points of the three-dimensional model in an operable manner. Furthermore, when the information processing terminal 101 accepts a switching instruction to switch the number of movable points displayed on the display unit, the information processing terminal 101 switches and displays the number of movable points displayed on the display unit in accordance with the switching instruction. As a result, according to this embodiment, the user can display movable points that are easy for the user to handle and perform movable operations, and can easily operate joints, etc. on the screen of a three-dimensional model such as a figure displayed on the display unit.

[0063] <Modification 1 of the operation method> The present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the invention. For example, in the above-mentioned embodiment, an example was described in which the display is controlled so that related parts and following parts are moved by an intuitive operation, such as selecting a movable point arranged near a part (such as a leg) that the user wants to move and moving (pulling) that part. However, the present invention is not limited to this, and for example, only the joint corresponding to the selected movable point may be controlled to rotate in response to the operation.

[0064] Fig. 10 explains a first modified example of a method for operating a digital figure. Here, as an example, as shown in an editing screen 400, a screen transition according to an operation when a first mode is selected as a display mode of a movable point is explained. Fig. 10 also shows an example in which a digital figure is operated in the same manner as in Fig. 5, using an operation method different from the operation method shown in Fig. 5.

[0065] Fig. 10(a) shows an editing screen 1000 in a state where the user has selected a movable point at the base of the right foot of the digital figure 351. 1001 indicates an object that highlights the selected movable point. The object 1001 is displayed superimposed over the selected movable point. The selected movable point is also displayed visible through the object 1001. This allows the user to easily recognize which movable point has been selected. In this modified example, only the movable part corresponding to the selected movable point can be rotated.

[0066] 10(b) shows an edit screen 1010 in which a slide operation is performed to rotate the right leg of the figure 351 backward as shown by the arrow direction with a movable point selected as shown by 1001, and the display is updated. In the edit screen 1010, the display of the figure 351 is updated to show a state in which the right leg has moved backward. Here, by rotating the movable part corresponding to the selected movable point in accordance with the operation shown by the arrow, the display is controlled so that it appears as if the group of parts connected to the movable part (the right leg) is moving.

[0067] Fig. 10(c) shows an editing screen 1020 in which the display is updated when the selection of the movable point is further changed from 1001 to 1021 from the state of Fig. 10(b) and a slide operation is performed in the direction of the arrow. It can be seen that 1021 is the movable point corresponding to the knee of the right leg, and only the parts below the knee are moved by the slide operation. In this way, according to this modified example, the digital figure can be operated so that only the parts connected to the selected movable point are moved. In this case, more detailed movements can be specified by the user's operation compared to the above embodiment.

[0068] <Modification 2 of the operation method> Next, a second modified example of the method for operating a digital figure will be described with reference to Fig. 11. A second modified example of the method for operating a digital figure will be described. Here, as an example, as shown in an editing screen 400, a screen transition in response to an operation when the first mode is selected as the display mode of the movable points will be described.

[0069] FIG. 11(a) shows an editing screen 1100 in a state where a user has selected a movable point to be fixed and further selected a movable point of the right ankle to be rotated. 1101 is an object showing the state where the movable point has been selected by the user to fix the rotation at that movable point. For example, the user can select the movable point as the fixed movable point by performing an operation such as a double tap or a long press on the movable point. The object 1101 is displayed superimposed over the movable point mark to show that the rotation of the movable point has been fixed. Note that the object 1101 is shown as a different display object (for example, a different type of shading) from the object 1102 showing that it has been selected as the point to be rotated.

[0070] Fig. 11(b) shows an edit screen 1110 in which, with the movable point selected as indicated by 1102, a slide operation is performed in the direction of rotating the right leg of the figure 351 backward, similar to the operation described in Fig. 5, and the display is updated. In the edit screen 1110, the display of the figure 351 is updated to show the state in which the right leg has moved backward. However, in this modified example, the movable point at the base of the right leg is fixed, so the upper thigh of the right leg is fixed and only the parts below the knee rotate.

[0071] Fig. 11(c) shows the editing screen 1120 in which the display has been updated after a slide operation has been performed in a direction to rotate the right leg of the figure 351 backward from the state shown in Fig. 11(b). Here again, the movable point at the base of the right leg is fixed, so the upper thigh of the right leg is fixed and only the parts below the knee rotate. In this way, according to this modified example, it is possible to select a movable point whose rotation is to be fixed from among a plurality of movable points that can be operated. In this way, when a movable point to be fixed is selected and a movable point to be rotated is operated, the digital figure can be operated so that only the related parts move, excluding the selected fixed movable point.

[0072] In addition, in the present embodiment, a humanoid model has been described as an example of a target model, but this is not intended to limit the present invention, and the present invention can be applied to models of various shapes, such as humans, animals, robots, insects, dinosaurs, etc. In any case, as described in the above embodiment, the convenience of the system can be improved by changing the settings related to the movable points of the model.

[0073] <Summary of the embodiment> The above-described embodiments disclose at least the following computer program, information processing terminal, and control method thereof.

[0074] (1) A computer that is an information processing terminal, a reception means for receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control means for displaying a three-dimensional model of the model on a display unit and displaying movable points of the three-dimensional model in an operable manner; Functioning as a The display control means is a computer program characterized in that, when a switching instruction to switch the number of movable points displayed on the display unit is received by the receiving means, the display control means switches and displays the number of movable points displayed on the display unit in accordance with the switching instruction.

[0075] (2) The computer program according to (1), wherein the display control means increases or decreases the number of the movable points in a stepwise manner in accordance with the switching instruction.

[0076] (3) A computer program as described in (1) or (2), characterized in that the maximum number of movable points to be displayed is the same as the maximum number of movable parts in a real model corresponding to the three-dimensional model.

[0077] (4) The computer program described in any one of (1) to (3), characterized in that the display control means displays an object indicating the movable point superimposed on the position of the movable part of the three-dimensional model displayed on the display unit.

[0078] (5) when a user operation for operating the movable point is received by the receiving means, the computer of the information processing terminal is caused to function as a determination means for determining a movement of at least one part of the three-dimensional model in response to the user operation; The computer program according to any one of (1) to (4), characterized in that the display control means updates the display of the three-dimensional model in accordance with the movement of the at least one part determined by the determination means.

[0079] (6) The determining means further determines a motion of a part that follows the motion of the at least one part determined in response to the user operation; The computer program described in (5), characterized in that the display control means updates the display of the three-dimensional model according to the movement of the at least one part determined by the determination means as well as the movement of the following part.

[0080] (7) The computer program described in (5) or (6), characterized in that the user operation for manipulating the movable points is an operation of pressing one of the movable points displayed on the display unit and, while holding it down, swiping in a direction that rotates the corresponding part.

[0081] (8) The computer program described in (5) or (6), characterized in that the user operation for manipulating the movable point is an operation of selecting one movable point displayed on the display unit and, after selection, swiping in a direction to rotate the corresponding part.

[0082] (9) The receiving means is capable of receiving a user operation to fix the movable point, The computer program according to any one of (5) to (8), characterized in that the determination means determines the movement of at least one part of the three-dimensional model, excluding parts associated with the movable point that is fixed in response to the user operation.

[0083] (10) The computer program according to any one of (5) to (9), further comprising causing the computer of the information processing terminal to function as a notification means for notifying an application server with which communication is possible via a network of a save request together with posture information of the edited three-dimensional model when a save instruction to save the posture of the edited three-dimensional model is received by the receiving means.

[0084] (11) The computer program according to any one of (5) to (10), wherein the display control means rotates the three-dimensional model displayed on the display unit in multiple directions in response to a user operation accepted by the accepting means.

[0085] (12) An information processing terminal, a reception means for receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control means for displaying a three-dimensional model of the model on a display unit and displaying movable points of the three-dimensional model in an operable manner; Equipped with The information processing terminal is characterized in that, when a switching instruction to switch the number of movable points displayed on the display unit is received by the receiving means, the display control means switches and displays the number of movable points displayed on the display unit in accordance with the switching instruction.

[0086] (13) A method for controlling an information processing terminal, comprising the steps of: a receiving step of receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control step of displaying a three-dimensional model of the model on a display unit and displaying a movable point of the three-dimensional model in an operable manner; Including, A control method for an information processing terminal, characterized in that, in the display control process, when a switching instruction to switch the number of movable points to be displayed on the display unit is received in the receiving process, the number of movable points to be displayed on the display unit is switched and displayed in accordance with the switching instruction. [Explanation of symbols]

[0087] 101a, b, c: information processing terminal, 102: application server, 103: network, 201: CPU, storage unit 202, 203: communication control unit, 204: display unit, 205: operation unit, 206: camera, 211: CPU, 212: ROM, 213: RAM, 214: display unit, 215: operation unit, 216: communication control unit, 217: HDD

Claims

1. The computer of the information processing terminal, a receiving means for receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control means for displaying a three-dimensional model of the model on a display unit and displaying movable points of the three-dimensional model in an operable manner; It functions as The display control means is a computer program characterized in that, when the receiving means receives a user operation to operate the movable points displayed on the display unit, the display control means moves and displays the positions of the movable points displayed on the display unit in accordance with the user operation without changing the distance between the movable points.

2. the computer of the information processing terminal further functions as a determination unit that determines a movement of at least one part of the three-dimensional model in response to the user operation; 2. The computer program product according to claim 1, wherein the display control means updates the display of the three-dimensional model in accordance with the movement of the at least one part determined by the determining means.

3. The determining means further determines a motion of a part that follows the motion of the at least one part determined in response to the user operation; 3. The computer program according to claim 2, wherein the display control means updates the display of the three-dimensional model in accordance with the movement of the at least one part determined by the determination means as well as the movement of the following part.

4. The computer program according to claim 2, characterized in that the user operation for manipulating the movable point is an operation of pressing one of the movable points displayed on the display unit and, while pressing it, swiping in a direction that rotates the corresponding part.

5. The computer program according to claim 2, characterized in that the user operation for manipulating the movable point is an operation of selecting one movable point displayed on the display unit and then swiping in a direction to rotate the corresponding part.

6. the accepting means is capable of accepting a user operation to fix the movable point, The computer program according to claim 2 , wherein the determining means determines the movement of at least one part of the three-dimensional model, excluding a part associated with the movable point that is fixed in response to the user operation.

7. The computer program according to claim 2, further comprising causing the computer of the information processing terminal to function as a notification means that, when the reception means receives a save instruction to save the attitude of the edited three-dimensional model, notifies an application server that can communicate via a network of a save request together with attitude information of the edited three-dimensional model.

8. 3. The computer program according to claim 2, wherein the display control means rotates the three-dimensional model displayed on the display unit in multiple directions in response to a user operation received by the receiving means.

9. 3. The computer program according to claim 2, wherein the display control means displays an object indicating the movable point superimposed on the position of the movable part of the three-dimensional model displayed on the display unit.

10. A computer program described in any one of claims 1 to 9, characterized in that when the display control means receives a switching instruction to switch the number of movable points to be displayed on the display unit by the receiving means, it switches and displays the number of movable points to be displayed on the display unit in accordance with the switching instruction.

11. 11. The computer program according to claim 10, wherein the display control means increases or decreases the number of the movable points in a stepwise manner in accordance with the switching instruction.

12. 12. The computer program of claim 11, wherein the maximum number of movable points displayed is equal to the maximum number of movable parts in a real-life model corresponding to the three-dimensional model.

13. An information processing terminal, a receiving means for receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control means for displaying a three-dimensional model of the model on a display unit and displaying movable points of the three-dimensional model in an operable manner; Equipped with The information processing terminal is characterized in that, when the display control means receives a user operation to operate the movable points displayed on the display unit through the receiving means, the display control means moves and displays the positions of the movable points displayed on the display unit in accordance with the user operation without changing the distance between the movable points.

14. A control method for an information processing terminal, comprising: a receiving step of receiving a user operation via a screen displayed on a display unit of the information processing terminal; a display control step of displaying a three-dimensional model of the model on a display unit and displaying movable points of the three-dimensional model in an operable manner; Including, In the display control process, when a user operation to operate the movable points displayed on the display unit is received in the reception process, the positions of the movable points displayed on the display unit are moved and displayed in accordance with the user operation without changing the distance between the movable points.

Images