Information processing system and information processing method

Abstract

[Problem] To improve the review efficiency of a 3D virtual model using an HMD. [Solution] An information processing system comprising a control circuit for controlling review of a 3D virtual model on the basis of review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and the 3D virtual model displayed by the HMD, wherein: the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model; and the control circuit starts the review at the first viewpoint on the basis of the review information, and transitions to the review at the second viewpoint when a trigger is detected.

Description

Information processing system and information processing method

[0001] The present disclosure relates to an information processing system and an information processing method.

[0002] In recent years, technologies have been developed to support the review of designs such as 3D (three-dimensional) virtual models. For example, Patent Literature 1 discloses a technology for realizing a walkthrough of a building using CG images.

[0003] Japanese Patent Application Laid-Open No. 2000-113212

[0004] However, with the technology disclosed in Patent Document 1, it may be difficult to provide sufficient support for changes in the user's viewpoint.

[0005] According to one aspect of the present disclosure, there is provided an information processing system including: a control circuit that controls a review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, the review information including at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, the control circuit starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected.

[0006] According to another aspect of the present disclosure, there is provided an information processing method including: a processor controlling a review of a 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, the review information including at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model; and the controlling includes starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected.

[0007] FIG. 1 is a diagram for describing an overview of an embodiment of the present disclosure. FIG. 1 is a block diagram illustrating an example of a functional configuration of an information processing system 1 according to the embodiment. FIG. 2 is a diagram for describing a review transition triggered by a user's operation on a 3D virtual model 50 according to the embodiment. FIG. 3 is a diagram for describing an example of guidance control by a control unit 110 according to the embodiment. FIG. 4 is a diagram for describing an example of guidance control by a control unit 110 according to the embodiment. FIG. 5 is a diagram for describing saving and restoring a viewpoint according to the embodiment. FIG. 6 is a diagram for describing an editing operation that does not involve operation of the editing device 20 according to the embodiment. FIG. 7 is a diagram for describing an editing operation that does not involve operation of the editing device 20 according to the embodiment. FIG. 8 is a diagram for describing an editing operation that does not involve operation of the editing device 20 according to the embodiment. FIG. 9 is a sequence diagram illustrating an example of a flow of viewpoint adjustment based on acquired information according to the embodiment. FIG. 10 is a sequence diagram illustrating an example of a flow of review information update based on acquired information according to the embodiment. FIG. 11 is a diagram for describing review control including interaction between a 3D virtual model 50 according to the embodiment and an object in the real world. FIG. 12 is a diagram for describing a review in a case where the information processing system 1 according to the embodiment includes a plurality of HMDs 10 worn by each of a plurality of users. FIG. 13 is a block diagram illustrating an example of a hardware configuration of an information processing device 90 according to the embodiment.

[0008] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0009] In addition, in this specification and drawings, when multiple identical components are to be described separately, letters or the like may be added to the end of the reference numerals. On the other hand, when it is not necessary to distinguish between multiple identical components, the letters or the like may be omitted and a description common to all of the multiple identical components may be given.

[0010] The description will be given in the following order: 1. Embodiment 1.1. Overview 1.2. Example of functional configuration 1.3. Functional details 2. Example of hardware configuration 3. Summary

[0011] <1. Embodiment> <<1.1. Overview>> First, an overview of one embodiment of the present disclosure will be described.

[0012] Recently, 3D virtual models designed using a head mounted display (HMD) have been reviewed.

[0013] A user wearing an HMD, for example, displays a designed 3D virtual model on the HMD, reviews the 3D virtual model from various viewpoints, and edits (modifies) the 3D virtual model.

[0014] However, in general, design work for a 3D virtual model, including editing, is performed using a PC (Personal Computer) or the like.

[0015] Therefore, when repeatedly reviewing and editing a 3D virtual model, the user is required to alternate between the HMD and the PC or the like.

[0016] Furthermore, since a review using an HMD can take a variety of viewpoints, the user is required to adjust the viewpoint each time in order to ensure the same viewpoint before and after the editing work.

[0017] Furthermore, even when a user wishes to perform a small amount of editing, they may forgo the task in order to avoid the above-mentioned complication, which reduces the efficiency of the review.

[0018] The technical idea according to an embodiment of the present disclosure has been conceived in light of the above points, and aims to improve the efficiency of reviewing a 3D virtual model using an HMD.

[0019] For this reason, an information processing method according to one embodiment of the present disclosure includes a processor controlling a review of a 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of a user wearing the HMD 10 (see FIG. 2) and the 3D virtual model 50 (see FIG. 1) displayed by the HMD 10.

[0020] The review information also includes at least the positional relationship between a first viewpoint and the 3D virtual model 50 and the positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model 50 .

[0021] Furthermore, the controlling starts a review in a first perspective based on the review information, and transitions to a review in a second perspective when a trigger is detected.

[0022] An outline of one embodiment of the present disclosure will be described below with reference to FIG.

[0023] FIG. 1 shows an example in which the positional relationships between each of viewpoints A, B, and C and a 3D virtual model 50 are recorded in the review information.

[0024] In the example shown in FIG. 1, at least the context of viewpoints A, B, and C is recorded in the review information.

[0025] Here, it is assumed that a review sequence for sequentially performing a review from viewpoint A, a review from viewpoint B, and a review from viewpoint C is recorded in the review information.

[0026] In this case, in the information processing method according to this embodiment, first, a review at viewpoint A is started based on the review information, as shown in the first row of FIG.

[0027] The user wearing the HMD 10 can temporarily suspend the review as needed, edit the 3D virtual model 50 using a PC or the like, and then put the HMD 10 back on and resume the review.

[0028] In this case, in the information processing method according to this embodiment, the review is resumed from the same viewpoint A as before the pause, as shown in the second row of FIG.

[0029] In this way, in the information processing method according to this embodiment, when a review is paused, the viewpoint at the time of the pause is saved, and when the review is resumed, the review is resumed from the saved viewpoint.

[0030] According to the above control, the user does not need to adjust the viewpoint each time after editing, and review efficiency is greatly improved.

[0031] Furthermore, in the information processing method according to this embodiment, when a trigger is detected in a review, a transition to a review from the next viewpoint is made based on the review information.

[0032] For example, if a trigger is detected in the situation shown in the second row of Figure 1, a transition from review at viewpoint A to review at viewpoint B is performed based on the trigger information, as shown in the third row of Figure 1.

[0033] In this case, viewpoint A is an example of a first viewpoint, and viewpoint B is an example of a second viewpoint.

[0034] Furthermore, for example, if a trigger is detected in the situation shown in the third row of Figure 1, a transition from a review at viewpoint B to a review at viewpoint C is performed based on the trigger information, as shown in the fourth row of Figure 1.

[0035] In this case, viewpoint B is an example of a first viewpoint, and viewpoint C is an example of a second viewpoint.

[0036] According to the above control, the user does not need to adjust the viewpoint each time to match the specified scenario, and review efficiency is greatly improved.

[0037] The information processing method according to this embodiment has been outlined above. An example of the functional configuration of an information processing system 1 that implements the information processing method according to this embodiment will now be described.

[0038] <<1.2. Functional Configuration Example>> FIG. 2 is a block diagram showing an example of the functional configuration of the information processing system 1 according to this embodiment.

[0039] As shown in FIG. 2, the information processing system 1 according to this embodiment includes, for example, an HMD 10 and an editing device 20.

[0040] The HMD 10 and the editing device 20 communicate information via a wireless or wired network 30 .

[0041] (HMD 10 ) The HMD 10 according to this embodiment is an example of a control device that controls the review of the 3D virtual model 50 .

[0042] As shown in FIG. 1, the HMD 10 according to this embodiment may include a control unit 110, a display unit 120, an audio output unit 130, an imaging unit 140, a sensor unit 150, an audio input unit 160, a memory unit 170, and a communication unit 180, etc.

[0043] (Control Unit 110) The control unit 110 according to this embodiment is an example of a control circuit that controls each component included in the HMD 10.

[0044] In addition, the control unit 110 according to this embodiment controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the display unit 120.

[0045] In particular, the control unit 110 according to this embodiment starts a review from a first viewpoint based on review information, and when a trigger is detected, performs control to transition to a review from a second viewpoint.

[0046] In this embodiment, the viewpoint refers to a standpoint.

[0047] The functions of the control unit 110 according to this embodiment are realized by cooperation between various processors and memories.

[0048] The functions of the control unit 110 according to this embodiment will be described in detail later.

[0049] (Display Unit 120) The display unit 120 according to this embodiment displays information under the control of the control unit 110.

[0050] The display unit 120 according to this embodiment displays various objects including, for example, a 3D virtual model 50 .

[0051] For this purpose, the display unit 120 according to the present embodiment includes various displays.

[0052] The display includes a see-through display.

[0053] (Audio Output Unit 130) The audio output unit 130 according to this embodiment outputs audio under the control of the control unit 110.

[0054] For this reason, the audio output unit 130 according to this embodiment includes an amplifier, a speaker, and the like.

[0055] (Photographing Unit 140) The photographing unit 140 according to this embodiment photographs an image from the user's viewpoint under the control of the control unit 110.

[0056] For this reason, the image capturing unit 140 according to this embodiment includes an image capturing element that is arranged corresponding to the user's viewpoint.

[0057] (Sensor Unit 150) The sensor unit 150 according to this embodiment acquires various types of sensor information under the control of the control unit 110.

[0058] For this reason, the sensor unit 150 according to this embodiment includes sensors according to the sensor information to be acquired.

[0059] Examples of sensors included in the sensor unit 150 include an acceleration sensor, a gyro sensor, a geomagnetic sensor, an illuminance sensor, a temperature sensor, and a radar.

[0060] (Audio Input Unit 160) The audio input unit 160 according to this embodiment collects audio around the HMD 10 under the control of the control unit 110.

[0061] For this purpose, the audio input unit 160 according to this embodiment includes at least one microphone.

[0062] (Storage Unit 170) The storage unit 170 according to this embodiment stores various types of information used in the HMD 10.

[0063] In particular, the storage unit 170 stores review information.

[0064] (Communication Unit 180) The communication unit 180 according to this embodiment communicates information with other devices via the network 30 under the control of the control unit 110.

[0065] The other devices include, for example, an editing device 20 .

[0066] (Editing Device 20 ) The editing device 20 according to this embodiment is a computer such as a PC used to edit the 3D virtual model 50 .

[0067] Editing software 210 used to edit the 3D virtual model 50 is installed in the editing device 20 according to this embodiment.

[0068] The functional configuration example of the information processing system 1 according to the present embodiment has been described above. Note that the functional configuration described above with reference to Fig. 2 is merely an example, and the functional configuration of the information processing system 1 according to the present embodiment is not limited to this example.

[0069] For example, a control unit that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the display unit 120 does not necessarily need to be provided in the HMD 10.

[0070] The control unit may be provided in, for example, the editing device 20 or a server (not shown), etc. In this case, the editing device 20 or the server (not shown) provided with the control unit controls the review using the HMD 10 via the network 30.

[0071] Furthermore, for example, as will be described later, editing of the 3D virtual model 50 does not necessarily require operations using an editing device. For this reason, the editing software 210 may be installed in the HMD 10.

[0072] Furthermore, for example, the information processing system 1 may further include a controller that is held or worn by the user. In this case, information acquired by the controller is transmitted to the HMD 10 via the network 30. Note that the controller may be, for example, a grip-type controller that is held by the user, or a ring-type controller that is worn on the user's finger or wrist.

[0073] The functional configuration of the information processing system 1 according to this embodiment can be flexibly modified according to specifications, operation, and the like.

[0074] <<1.3. Detailed Functions>> Next, the functions of the information processing system 1 according to this embodiment will be described in detail.

[0075] First, the detection of a trigger according to this embodiment and the transition to a review from a second viewpoint based on the detected trigger will be described.

[0076] As described above, the control unit 110 according to this embodiment starts a review from a first viewpoint based on review information, and when a trigger is detected, performs control to transition to a review from a second viewpoint.

[0077] The trigger according to this embodiment may include, for example, the passage of a predetermined time.

[0078] In the example shown in FIG. 1, the control unit 110 may detect that a predetermined time has elapsed after the start of reviewing from viewpoint A, and control a transition to reviewing from viewpoint B.

[0079] Similarly, in the example shown in FIG. 1, the control unit 110 may detect that a predetermined time has elapsed after the transition to review at viewpoint B, and control the transition to review at viewpoint C.

[0080] In the above case, the predetermined time for the transition from viewpoint A to viewpoint B may be different from the predetermined time for the transition from viewpoint B to viewpoint C. Each predetermined time is recorded in the review information.

[0081] Furthermore, the trigger according to this embodiment may include, for example, an explicit instruction from the user.

[0082] The user may be able to instruct the transition of the review by, for example, operating a button or the like provided on the HMD 10, operating the controller described above, using a gesture with the hand or the like, or by voice or the like.

[0083] The control unit 110 can detect the above-described gestures based on the image captured by the image capturing unit 140 .

[0084] Furthermore, the control unit 110 can detect instructions given by the user's voice based on the voice recognition results of the user's voice collected by the voice input unit 160. Note that the voice recognition may be performed by the HMD 10 or by a device other than the HMD 10, such as a server.

[0085] Furthermore, the trigger according to this embodiment includes, for example, a predetermined interaction between the user and the 3D virtual model 50 .

[0086] The predetermined interaction may be, for example, a user's operation on the 3D virtual model 50 .

[0087] FIG. 3 is a diagram for explaining the transition of a review triggered by a user's operation on the 3D virtual model 50 according to this embodiment.

[0088] In the example shown in FIG. 3, the user uses his / her hand H to perform an operation on the 3D virtual model 50 .

[0089] The above operation may be, for example, a rotation operation for rotating the 3D virtual model 50 .

[0090] The control unit 110 according to this embodiment executes a transition to a review from the second viewpoint, triggered by the rotation operation described above.

[0091] Generally, it is difficult to adjust the amount of rotation, angle, etc. when rotating the 3D virtual model 50. However, in the information processing method according to this embodiment, the first viewpoint and the second viewpoint are determined in advance, which makes it possible to rotate the 3D virtual model 50 easily and smoothly without requiring delicate rotation operations.

[0092] The control unit 110 can detect an operation on the 3D virtual model 50 based on the image captured by the imaging unit 140 .

[0093] The 3D virtual model 50 may be operated using a controller held by or attached to the user's hand H.

[0094] Furthermore, the operation on the 3D virtual model 50 may be performed by voice.

[0095] Furthermore, the predetermined interaction may be, for example, the viewpoint of the user wearing the HMD 10 substantially matching the second viewpoint.

[0096] In this case, the control unit 110 according to this embodiment may perform control to guide the viewpoint of the user wearing the HMD 10 so that it substantially coincides with the second viewpoint.

[0097] 4 and 5 are diagrams for explaining an example of guidance control by the control unit 110 according to this embodiment.

[0098] For example, the control unit 110 according to this embodiment may realize guidance by displaying an object 60 such as an arrow on the display unit 120 as shown in FIG.

[0099] Furthermore, for example, the control unit 110 according to this embodiment may realize guidance by causing the audio output unit 130 to output audio 70 such as "Please look from the front," as shown in FIG.

[0100] Furthermore, for example, the control unit 110 according to this embodiment may achieve guidance by displaying an object 61 such as an avatar at a position corresponding to the second viewpoint on the display unit 120, as shown in FIG. 5 .

[0101] Furthermore, the control unit 110 according to this embodiment may realize guidance by causing the audio output unit 130 to output audio 71 such as "Please look from this location," in addition to controlling the display of the object 61.

[0102] The above describes the transition of review based on the detection of a trigger according to this embodiment. The above-described control eliminates the need for the user to adjust the viewpoint each time to match a defined scenario, significantly improving review efficiency.

[0103] Furthermore, as described above, the control unit 110 according to this embodiment may perform control such that when a review is paused, the viewpoint at the time of the pause is saved, and when the review is resumed, the review is resumed from the saved viewpoint.

[0104] Such control eliminates the need for the user to adjust the viewpoint each time after editing, greatly improving review efficiency.

[0105] Furthermore, the control unit 110 according to this embodiment may store the viewpoint at the time when the instruction is detected based on the user's instruction during review, and may perform control to return to the stored viewpoint based on the user's instruction.

[0106] FIG. 6 is a diagram for explaining saving and restoring of a viewpoint according to this embodiment.

[0107] In the example shown in FIG. 6, the control unit 110 causes the display unit 120 to display an object 62 for the user to instruct saving of the viewpoint.

[0108] For example, when the control unit 110 detects that the user's finger F has come into contact with the object 62, the control unit 110 stores the viewpoint at the time of detection.

[0109] Thereafter, the control unit 110 causes the display unit 120 to display an object 63 for instructing the user to return to the viewpoint saved.

[0110] For example, when the control unit 110 detects that the user's finger F has come into contact with the object 63 , the control unit 110 performs control to return to the viewpoint corresponding to the object 63 .

[0111] According to the above control, the user can save a viewpoint at any time and then return to the saved viewpoint, which greatly improves review efficiency.

[0112] The user may be able to save a viewpoint at any point in time together with any comment, and may also be able to save any comment separately from the viewpoint.

[0113] Next, editing of the 3D virtual model 50 using the HMD 10 according to this embodiment will be described.

[0114] In the information processing system 1 according to this embodiment, the user can edit the 3D virtual model 50 without operating the editing device 20 .

[0115] When a user performs an editing operation on the 3D virtual model 50 during a review, the control unit 110 according to this embodiment displays the 3D virtual model 50 reflecting the editing operation on the display unit 120 while maintaining the review.

[0116] 7 to 10 are diagrams for explaining an editing operation that does not involve operating the editing device 20 according to this embodiment.

[0117] In the example shown in FIG. 7 , the control unit 110 according to this embodiment causes the display unit 120 to display a simple editing screen 64 for simply editing various parameters related to the 3D virtual model 50 .

[0118] On the other hand, FIG. 8 shows an example of a standard editing screen 74 provided by the editing software 210 installed in the editing device 20.

[0119] The standard editing screen 74 is a main screen for designing and editing the 3D virtual model 50. As shown in Fig. 8, the standard editing screen 74 displays, for example, boxes for editing the style, color, scale, object candidates, etc. of the 3D virtual model 50 all together on one screen.

[0120] In contrast, the simplified editing screen 64 may be a user interface that includes only some of the boxes described above, or a simplified user interface that allows editing of some of the parameters that can be edited on the standard editing screen 74.

[0121] Examples of parameters that can be edited on the simple editing screen 64 include parameters related to the scale, style, color, texture, material, elasticity, etc. of each part of the 3D virtual model 50. The above styles include various styles such as saturation, brightness, sharpness, contrast, etc. in addition to the example shown in FIG.

[0122] The user may be able to edit any parameter related to the 3D virtual model 50 by operating a slider included in the simple editing screen 64 with a finger F or the like, for example.

[0123] In addition to the sliders, the simple edit screen 64 may also include fields for inputting parameter values, buttons for increasing or decreasing parameter values, and the like.

[0124] In addition, the control unit 110 may normally perform control such as not displaying the simple editing screen 64 or displaying it in a minimized form, and when a user instruction is detected, may perform control such as displaying the simple editing screen 64 or displaying it in an enlarged form.

[0125] On the other hand, the control unit 110 may control a user interface that allows the user to perform direct editing operations on the 3D virtual model 50 .

[0126] In the example shown in FIG. 9, the user edits the height of the lens of the 3D virtual model 50 by pulling the lens of the 3D virtual model 50 with the hand H or the like.

[0127] On the other hand, the control unit 110 may detect an editing operation performed by the user's voice.

[0128] In the example shown in FIG. 10, the control unit 110 detects an editing operation based on the results of speech recognition of a user's voice 72 such as "increase the lens height by 15 mm."

[0129] In this way, the user interface for the user to perform editing operations may be one that uses voice.

[0130] As described above with the specific examples, in the information processing system 1 according to this embodiment, the user can perform editing operations without operating the editing device 20 by using various user interfaces.

[0131] Furthermore, in order to achieve the above, the control unit 110 according to this embodiment transmits information relating to the acquired editing operation to the editing software 210 used to edit the 3D virtual model 50, and performs control to acquire the 3D virtual model 50 reflecting the editing operation from the editing software 210.

[0132] The transmission of the acquired information about the editing operation and the acquisition of the 3D virtual model 50 reflecting the editing operation may be realized via an API (Application Programming Interface) of the editing software 210 .

[0133] The acquired information about the editing operation includes various parameters related to the 3D virtual model 50 .

[0134] The editing operation that does not involve operation of the editing device 20 according to this embodiment has been described.

[0135] According to the control described above, the user does not need to alternate between using the HMD 10 and the editing device 20, and review efficiency is greatly improved.

[0136] Next, control based on acquired information according to this embodiment will be described.

[0137] The acquired information includes sensor information acquired by the sensor unit 150, information related to editing operations, user information, and the like.

[0138] For example, the control unit 110 according to this embodiment may adjust both or either one of the first viewpoint and the second viewpoint based on the acquired information.

[0139] FIG. 11 is a sequence diagram showing an example of the flow of viewpoint adjustment based on acquired information according to this embodiment.

[0140] In the example shown in FIG. 11, the control unit 110 first acquires sensor information and information related to editing operations (S101).

[0141] The control unit 110 estimates the user's posture based on the sensor information and information related to the editing operation acquired in step S101, and user information acquired in advance (S102).

[0142] The user information includes, for example, information about the height of the user.

[0143] The user's posture estimated in step S102 includes, for example, standing, sitting, crouching, and the like.

[0144] The control unit 110 adjusts both or either one of the first viewpoint and the second viewpoint based on the user's posture estimated in step S102 (S103).

[0145] The processes in steps S101 to S103 may be repeatedly executed from the start to the end of the review.

[0146] A more specific example will be described below.

[0147] For example, assume that the user is standing when the review from the first viewpoint starts, and the user becomes sitting midway through the review from the first viewpoint.

[0148] In this case, the control unit 110 may detect the change in the user's posture as described above, and adjust the first viewpoint lower so that the 3D virtual model 50 is easier to observe when the user is sitting.

[0149] Similarly, the control unit 110 may adjust the second viewpoint to a lower value.

[0150] According to the above control, it is possible to realize a more efficient review according to the user's posture.

[0151] Furthermore, for example, the control unit 110 according to this embodiment may update the review information based on the acquired information.

[0152] FIG. 12 is a sequence diagram showing an example of the flow of review information update based on acquired information according to this embodiment.

[0153] In the example shown in FIG. 12, the control unit 110 first acquires sensor information and information related to editing operations (S201).

[0154] The control unit 110 performs an analysis based on the sensor information and information related to the editing operation acquired in step S201, and the user information acquired in advance (S202).

[0155] The control unit 110 updates the review information based on the analysis results in step S202 (S203).

[0156] The processes in steps S201 to S203 may be repeatedly executed from the start to the end of the review.

[0157] A more specific example will be described below.

[0158] For example, assume that the 3D virtual model 50 is a space including seats, such as an aircraft cabin, and that the space is to be reviewed.

[0159] In this case, the control unit 110 can analyze the acquired information to obtain, as an analysis result, for example, that the user made many edits to the size (height) of the seat during the review.

[0160] In the above case, the control unit 110 may update the review information so that a scene in which the user sits in the seat is added to the review scenario.

[0161] The update of the review information as described above may be realized, for example, by learning based on the acquired information.

[0162] For example, the control unit 110 may update the review information based on the learning result that tall users tend to edit seats more frequently.

[0163] According to the above control, it is possible to provide more efficient reviews to the user.

[0164] Next, review control including interaction between the 3D virtual model 50 according to this embodiment and an object in the real world will be described with reference to FIG.

[0165] In the example shown in FIG. 13, the 3D virtual model 50 is a camera lens, and the object 55 in the real world is a mock-up of the camera housing.

[0166] In this case, the control unit 110 controls the review including the interaction between the 3D virtual model 50 and the object 55 in the real world, for example, based on review information corresponding to a review scenario that checks the attachability of the 3D virtual model to the object 55.

[0167] In this way, according to the information processing system 1 of this embodiment, it is possible to perform editing work while checking the interaction between the 3D virtual model 50 and the object 55 in the real world, thereby greatly improving review efficiency.

[0168] Next, a case will be described in which the information processing system 1 according to this embodiment includes a plurality of HMDs 10 worn by a plurality of users.

[0169] Although the above description has been given mainly as an example of a case where there is one user, there may be multiple users according to this embodiment.

[0170] FIG. 14 is a diagram for explaining a review in a case where the information processing system 1 according to this embodiment includes a plurality of HMDs 10 each worn by a plurality of users.

[0171] In the example shown in FIG. 14, the information processing system 1 includes a first HMD 10A worn by a first user 80A and a second HMD 10B worn by a second user 80B.

[0172] In this case, the first HMD 10A is equipped with a first control unit 110A that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the first user 80A and the 3D virtual model 50 displayed by the first display unit 120A provided in the first HMD 10A.

[0173] Similarly, the second HMD 10B has a second control unit 110B that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the second user 80B and the 3D virtual model 50 displayed by the second display unit 120B provided in the second HMD 10B.

[0174] The review information stored in the first HMD 10A and the review information stored in the second HMD 10B may be the same or different from each other.

[0175] In the above configuration, in addition to the 3D virtual model 50, the first user 80A (or an avatar corresponding to the first user 80A) may be included in the field of view of the second user 80B.

[0176] Similarly, in addition to the 3D virtual model 50, the field of view of the first user 80A may include the second user 80B (or an avatar representing the second user 80B).

[0177] Therefore, with the above configuration, the second user 80B can check the interaction between the first user 80A and the 3D virtual model 50.

[0178] Similarly, the first user 80A can see the interaction between the second user 80B and the 3D virtual model 50.

[0179] Also, in this respect, the first user 10A and the second user 10B can be considered to be objects in the real world.

[0180] As described above, the information processing system 1 according to this embodiment may include a plurality of HMDs 10 worn by a plurality of users.

[0181] According to the above configuration, each user can check the interaction between other users and the 3D virtual model 50, which, for example, greatly improves review efficiency.

[0182] Next, control relating to restriction of movement of the viewpoint according to this embodiment will be described.

[0183] Viewing images using the HMD 10A may cause so-called 3D motion sickness.

[0184] Therefore, the control unit 110 according to this embodiment may impose restrictions on the movement of the viewpoint during review in order to prevent (reduce) 3D motion sickness.

[0185] For example, the control unit 110 may set an upper limit on the speed and acceleration of movement in continuous movement from one viewpoint to the next.

[0186] Furthermore, for example, the control unit 110 may set an upper limit on the angular velocity and angular acceleration of the change in direction during the movement.

[0187] Furthermore, for example, the control unit 110 may limit the directional changes that can be made at one time to only one of the yaw, pitch, or roll directions, and achieve movement by combining these directions continuously in a time series.

[0188] Furthermore, for example, the control unit 110 may express the movement as discrete rather than continuous.

[0189] The main functions of the information processing system 1 according to this embodiment have been described above.

[0190] In the above description, the positional relationship between the viewpoint and the 3D virtual model 50 is defined in the review information. However, various conditions other than the positional relationship can be defined in the review information.

[0191] Other conditions defined in the review information include, for example, the size, texture, and surrounding environment (for example, background, lighting environment, etc.) of the 3D virtual model 50.

[0192] Furthermore, the control unit 110 may perform various controls other than those described above.

[0193] For example, the control unit 110 may perform control so that an explanation of the viewpoints to be reviewed is displayed before the review begins, immediately after the review begins, or the like.

[0194] Furthermore, for example, when resuming review after editing using the editing device 20, the control unit 110 may change the viewpoint or the posture of the 3D virtual model 50 so that the editing target is easier to see, and then resume review.

[0195] Furthermore, for example, the control unit 110 may automatically generate preset review information from the 3D virtual model 50 .

[0196] Furthermore, for example, the control unit 110 may update a preset series of review information by learning the review history (for example, updating to a collection of frequently used viewpoints).

[0197] 2. Hardware Configuration Example Next, a hardware configuration example common to the HMD 10 and the editing device 20 according to an embodiment of the present disclosure will be described.

[0198] Fig. 15 is a block diagram showing an example hardware configuration of an information processing device 90 according to an embodiment of the present disclosure. The information processing device 90 may be a device having a hardware configuration equivalent to that of the HMD 10 and the editing device 20. However, the HMD 10 and the editing device 20 may not have some of the configuration shown in Fig. 15 or may have configurations other than those shown in Fig. 15.

[0199] 15 , the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the information processing device 90 may include further components in addition to the components shown here.

[0200] (Processor 871) The processor 871 functions, for example, as an arithmetic processing device or a control device, and controls the overall operation of each component or part of it based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable storage medium 901.

[0201] (ROM 872, RAM 873) The ROM 872 is a means for storing programs to be read into the processor 871, data to be used for calculations, etc. The RAM 873 temporarily or permanently stores, for example, the programs to be read into the processor 871 and various parameters that change as appropriate when the programs are executed.

[0202] (Host bus 874, bridge 875, external bus 876, interface 877) The processor 871, ROM 872, and RAM 873 are connected to one another via, for example, a host bus 874 that is capable of high-speed data transmission. On the other hand, the host bus 874 is connected to, for example, an external bus 876 that has a relatively low data transmission speed via a bridge 875. Furthermore, the external bus 876 is connected to various components via an interface 877.

[0203] (Input Device 878) For example, a mouse, keyboard, touch panel, button, switch, lever, etc. are used as the input device 878. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 878. The input device 878 also includes an audio input device such as a microphone.

[0204] (Output Device 879) The output device 879 is a device capable of visually or audibly notifying the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. The output device 879 according to the present disclosure also includes various vibration devices capable of outputting tactile stimulation.

[0205] (Storage 880) The storage 880 is a device for storing various types of data. For example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device is used as the storage 880.

[0206] (Drive 881) The drive 881 is a device that reads information recorded on a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable storage medium 901.

[0207] (Removable storage medium 901) The removable storage medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.

[0208] (Connection Port 882) The connection port 882 is a port for connecting an external device 902, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0209] (Externally Connected Device 902) The externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

[0210] (Communication device 883) The communication device 883 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0211] 3. Summary As described above, the information processing method implemented by the information processing system 1 according to an embodiment of the present disclosure includes a processor controlling the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the HMD 10.

[0212] The review information also includes at least the positional relationship between a first viewpoint and the 3D virtual model 50 and the positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model 50 .

[0213] Furthermore, the controlling starts a review in a first perspective based on the review information, and transitions to a review in a second perspective when a trigger is detected.

[0214] According to the above configuration, it is possible to improve the efficiency of reviewing a 3D virtual model using an HMD.

[0215] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0216] Furthermore, the steps of the processes described in this disclosure do not necessarily have to be processed in chronological order according to the order shown in the flowcharts or sequence diagrams. For example, the steps of the processes of each device may be processed in an order different from the order shown, or may be processed in parallel.

[0217] Furthermore, the series of processes performed by each device described in this disclosure may be realized by a program stored in a non-transitory computer-readable storage medium. Each program is, for example, loaded into RAM when executed by a computer and executed by a processor such as a CPU. The storage medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The program may also be distributed, for example, via a network, without using a storage medium.

[0218] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0219] The following configurations also fall within the technical scope of the present disclosure. (1) An information processing system including: a control circuit that controls a review of a 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, the review information including at least a positional relationship between a first viewpoint and the 3D virtual model and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, the control circuit starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected. (2) The information processing system described in (1), wherein the trigger includes a predetermined interaction between the user and the 3D virtual model. (3) The information processing system described in (2), wherein the predetermined interaction includes an operation by the user on the 3D virtual model. (4) The information processing system described in (2), wherein the predetermined interaction includes the user's viewpoint substantially coinciding with the second viewpoint. (5) The information processing system according to (4), wherein the control circuit performs control to guide the user's viewpoint so that it substantially coincides with the second viewpoint. (6) The information processing system according to any one of (1) to (5), wherein, when the user performs an editing operation on the 3D virtual model during the review, the control circuit causes the HMD to display the 3D virtual model reflecting the editing operation while maintaining the review. (7) The information processing system according to (6), wherein the control circuit controls a user interface that enables the user to perform the editing operation. (8) The information processing system according to (7), wherein the control circuit causes the HMD to display an object that enables the user to edit parameters of the 3D virtual model. (9) The information processing system according to any one of (6) to (8), wherein the control circuit transmits the acquired information about the editing operation to software used to edit the 3D virtual model, and performs control to acquire the 3D virtual model reflecting the editing operation from the software.(10) The information processing system according to any one of (1) to (9), wherein the control circuit adjusts at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review. (11) The information processing system according to any one of (1) to (10), wherein the control circuit updates the review information based on information acquired by the HMD during the review. (12) The information processing system according to any one of (10) or (11), wherein the information acquired by the HMD includes sensor information and information related to an editing operation of the 3D virtual model. (13) The information processing system according to any one of (1) to (12), wherein the control circuit controls the review including an interaction between the 3D virtual model and an object in the real world. (14) The information processing system according to any one of (1) to (13), wherein the control circuit saves a viewpoint at the time of the review being paused when the review is paused, and resumes the review from the saved viewpoint when the review is resumed. (15) The information processing system according to any one of (1) to (14), comprising: a first control circuit that controls the review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and a 3D virtual model displayed by the first HMD, and a second control circuit that controls the review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD.(16) An information processing method including: a processor controlling a review of the 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, wherein the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, and the controlling includes starting the review from the first viewpoint based on the review information and, when a trigger is detected, transitioning to the review from the second viewpoint. (17) The information processing method according to (16), wherein the trigger includes a predetermined interaction between the user and the 3D virtual model. (18) The information processing method according to (17), wherein the predetermined interaction includes an operation by the user on the 3D virtual model. (19) The information processing method according to (17), wherein the predetermined interaction includes the user's viewpoint substantially coinciding with the second viewpoint. (20) The information processing method according to (19), wherein the controlling includes controlling to guide the user so that a viewpoint of the user substantially coincides with the second viewpoint. (21) The information processing method according to any one of (16) to (20), wherein the controlling includes, when the user performs an editing operation on the 3D virtual model in the review, displaying the 3D virtual model reflecting the editing operation on the HMD while maintaining the review. (22) The information processing method according to (21), wherein the controlling includes controlling a user interface for the user to perform the editing operation. (23) The information processing method according to (22), wherein the controlling includes displaying an object for the user to edit parameters of the 3D virtual model on the HMD. (24) The information processing method according to any one of (16) to (23), wherein the controlling includes transmitting the acquired information about the editing operation to software used to edit the 3D virtual model, and performing control to acquire the 3D virtual model reflecting the editing operation from the software.(25) The information processing method according to any one of (16) to (24), wherein the controlling adjusts at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review. (26) The information processing method according to any one of (16) to (25), wherein the controlling updates the review information based on information acquired by the HMD during the review. (27) The information processing method according to any one of (25) or (26), wherein the information acquired by the HMD includes sensor information and information regarding an editing operation of the 3D virtual model. (28) The information processing method according to any one of (16) to (27), wherein the controlling controls the review including an interaction between the 3D virtual model and an object in the real world. (29) The information processing method according to any one of (16) to (28), wherein the controlling includes, when the review is paused, saving a viewpoint at the time of the pause, and when the review is resumed, resuming the review from the saved viewpoint. (30) The information processing method according to any one of (16) to (29), comprising: controlling the review of a 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and a 3D virtual model displayed by the first HMD; and controlling the review of a 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD.

[0220] REFERENCE SIGNS LIST 1 Information processing system 10 HMD 110 Control unit 120 Display unit 130 Audio output unit 140 Photography unit 150 Sensor unit 160 Audio input unit 170 Storage unit 180 Communication unit 20 Editing device 210 Editing software 50 3D virtual model

Claims

1. An information processing system comprising: a control circuit that controls a review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, wherein the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, and the control circuit starts the review from the first viewpoint based on the review information, and transitions to the review from the second viewpoint when a trigger is detected.

2. The information processing system according to claim 1, wherein the trigger includes a predetermined interaction between the user and the 3D virtual model.

3. The information processing system according to claim 2, wherein the predetermined interaction includes an operation by the user on the 3D virtual model.

4. The information processing system according to claim 2, wherein the predetermined interaction includes the user's viewpoint substantially matching the second viewpoint.

5. The information processing system according to claim 4, wherein the control circuit performs control to guide the user's viewpoint so that it substantially coincides with the second viewpoint.

6. The information processing system according to claim 1, wherein, when the user performs an editing operation on the 3D virtual model during the review, the control circuit causes the HMD to display the 3D virtual model reflecting the editing operation while maintaining the review.

7. The information processing system according to claim 6, wherein the control circuit controls a user interface for the user to perform the editing operation.

8. The information processing system according to claim 7, wherein the control circuit causes the HMD to display an object for the user to edit parameters of the 3D virtual model.

9. The information processing system according to claim 6, wherein the control circuit transmits the acquired information about the editing operation to software used to edit the 3D virtual model, and performs control to acquire from the software the 3D virtual model reflecting the editing operation.

10. The information processing system according to claim 1, wherein the control circuit adjusts at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review.

11. The information processing system according to claim 1, wherein the control circuit updates the review information based on information acquired by the HMD in the review.

12. The information processing system according to claim 10, wherein the information acquired by the HMD includes sensor information and information related to editing operations of the 3D virtual model.

13. The information processing system of claim 1, wherein the control circuitry controls the review to include interactions between the 3D virtual model and objects in the real world.

14. The information processing system according to claim 1, wherein the control circuit, when the review is paused, saves the viewpoint at the time of the pause, and when the review is resumed, resumes the review from the saved viewpoint.

15. The information processing system of claim 1, comprising: a first control circuit that controls the review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and a 3D virtual model displayed by the first HMD; and a second control circuit that controls the review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD.

16. An information processing method, comprising: a processor controlling a review of the 3D virtual model based on review information recording a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, wherein the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, and the controlling includes starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected.

17. The information processing method according to claim 16, wherein the trigger includes a predetermined interaction between the user and the 3D virtual model.

18. The information processing method according to claim 16, wherein the controlling includes, when the user performs an editing operation on the 3D virtual model during the review, displaying the 3D virtual model reflecting the editing operation on the HMD while maintaining the review.

19. The information processing method according to claim 16, wherein the controlling comprises adjusting at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review.

20. The information processing method of claim 16, wherein the controlling includes controlling the review to include interactions between the 3D virtual model and objects in the real world.

Images