Information processing device and computer program

Abstract

When photographing an object and generating a video in which the 3D model data of the photographed object is placed in a virtual space, it is possible to apply different effects depending on the type of object. [Solution] An information processing device comprising: scan control means that controls the position and angle of a scanner to scan an object that has been placed on it and generate a scanned image; data generation means that generates three-dimensional model data from the scanned image; determination means that determines the type of object based on a rendered image of the object generated based on the three-dimensional model data; data generation means that generates output data including an image of the three-dimensional model based on the three-dimensional model data placed in a virtual space; and output means that outputs the output data to an external output device, wherein the data generation means generates output data including object-specific effects according to the type of object determined by the determination means.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a computer program.

Background Art

[0002] Patent Document 1 describes a technique for generating a virtual space image visible from a virtual camera in a virtual space by mapping a texture generated from a group of captured images obtained by capturing an object from a plurality of imaging directions to a primitive in the virtual space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the generation of a virtual space image as in the above technique, when performing video output by arranging a three-dimensional model on a virtual space using three-dimensional model data of a photographed object, it is expected to be able to perform different effects according to the type of the object.

[0005] Therefore, an object is to be able to perform different effects according to the type of an object when generating a video by capturing the object and arranging a three-dimensional model on a virtual space using the three-dimensional model data of the photographed object.

Means for Solving the Problems

[0006] One embodiment for solving the above problem is an information processing device comprising: scan control means for controlling the position and angle of a scanner to scan the appearance of an object placed on a mounting stand and generate a scanned image; data generation means for generating three-dimensional model data from the scanned image; determination means for determining the type of the object from among a plurality of types based on a rendering image of the object generated based on the three-dimensional model data; data generation means for generating output data including an image of the three-dimensional model based on the three-dimensional model data placed in a virtual space; and output means for outputting the output data to an external output device, wherein the data generation means generates the output data to include a rendering for each object according to the type of object determined by the determination means. [Effects of the Invention]

[0007] When photographing an object and generating a video in which the 3D model data of the photographed object is placed in a virtual space, different effects can be applied depending on the type of object. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example configuration of the image processing system 10 corresponding to the embodiment. [Figure 2] This figure shows an example of the hardware configuration of the information processing device 100, an example of the hardware configuration of the headset 150, and an example of the hardware configuration of the operating device 160, corresponding to the embodiment. [Figure 3A] A diagram showing an example of the appearance of a doll-shaped model corresponding to the embodiment. [Figure 3B] A figure showing another example of the appearance of a doll-shaped model corresponding to the embodiment. [Figure 3C] A figure showing yet another example of the appearance of a doll-shaped model corresponding to an embodiment. [Figure 4A] A diagram showing an example of the implementation of the image processing system 10. [Figure 4B] A diagram showing an example configuration of the image processing system 10 corresponding to the embodiment during scanning. [Figure 4C] A diagram showing an example of the external configuration of the operating device 160 corresponding to the embodiment. [Figure 5] A flowchart illustrating an example of processing in the image processing system 10 corresponding to the embodiment. [Figure 6] A diagram showing an example of a display screen for the type determination result corresponding to the embodiment. [Figure 7A] A diagram showing an example of a data table configuration related to the type of presentation corresponding to the embodiment. [Figure 7B] A diagram showing an example of the configuration of a data table that manages the data of reference images corresponding to the embodiment. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features from the multiple features described in the embodiments may be arbitrarily combined. Also, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted. Furthermore, in each figure, the top, bottom, left, right, front, and back directions relative to the paper will be used as the top, bottom, left, right, front, and back directions of the parts (or components) in this embodiment, and will be used in the descriptions within the text.

[0010] First, the configuration of the image processing system corresponding to this embodiment will be described. Figure 1 is a diagram showing an example of the configuration of the image processing system 10 corresponding to this embodiment. The image processing system 10 is configured by connecting an information processing device 100, a scanner 110, a support arm 120, a model support device 130, a display device 140, a headset 150, an operating device 160, etc. to it. Note that the system configuration is not limited to that shown in Figure 1, and the information processing device 100 may be further connected to an external server, cloud server, etc. via a network. Such external server, etc., can execute at least some of the processing of the embodiment described below.

[0011] The information processing apparatus 100 controls the operations of at least one of the scanner 110, the support arm 来 120, and the model support device 130, captures an item to be photographed from an arbitrary angle to generate a plurality of images, and generates three-dimensional model data (this data) from the plurality of images. When the item to be photographed has a configuration in which it can be separated into a plurality of component items, it is also possible to perform photographing for each component item to generate three-dimensional model data, and integrate them to generate three-dimensional model data of the target item. Further, the information processing apparatus 100 can perform a process (type determination process) of analyzing a rendering image obtained from the three-dimensional model data and determining the type of the subject to be scanned. Details of the type determination process will be described in detail by referring to the flowchart of FIG. 5 and the like.

[0012] By using the generated three-dimensional model data as virtual space data, the information processing apparatus 100 can also function as a video generation apparatus that generates a video displayed on a virtual space. In the present embodiment, the subject to be photographed is an item such as an assemblable plastic model, an action figure (a figure having a movable joint part), a toy, a doll, etc., and these are collectively referred to as "models" hereinafter. The generated video is provided to the headset 150 and the operation device 160 and displayed to the user.

[0013] Next, the scanner 110 is a three-dimensional scanner device that photographs (scans) the three-dimensional shape of the model to be photographed according to the control of the information processing apparatus 100 and outputs the three-dimensional shape and color information of the imaging target. In the present embodiment, the scan signals output from the scanner 110 are collectively referred to as "scan images". For the scanner 110, for example, Space Spider manufactured by Artec can be used. In the present embodiment, for example, 3D scan data of the entire toy can be obtained by acquiring about 500 to 800 frames of scan images. Further, as the scanner 110, for example, a smartphone with a camera installed with an application for photographing a three-dimensional shape may be used.

[0014] The support arm 120 is a position and orientation control device that moves the scanner 110 to a predetermined shooting position and orientation according to the control of the information processing device 100. The support arm 120 may be configured to allow manual changes to the shooting position and orientation, and to maintain the changed position and orientation in a fixed position, or it may be configured to be controllable by the information processing device 100. When the support arm 120 is configured to be controllable by the information processing device 100, for example, the xArm 7 manufactured by UFACTORY can be used. The xArm 7 consists of seven joints and can move in a manner similar to a human arm. Alternatively, the scanner 110 may be positioned manually instead of using the support arm 120. Furthermore, for shooting positions and orientations that cannot be covered by the support arm 120, the scanner 110 may be operated manually to perform scanning.

[0015] The model support device 130 is a support stand (or a mounting stand for the model) that supports a model with a fixed pose. The model support device 130 may be configured to rotate when the model is mounted on the support stand or on the tip of a support rod. In this embodiment, the scanner 110 is positioned at an arbitrary shooting position and angle using the support arm 120, and then the model support device 130 is rotated to take a photograph. The model support device 130 is rotatable in both clockwise and counterclockwise directions. The model support device 130 can also be stopped at any rotation angle within the range of 0 to 360 degrees, and the model support device 130 can be stopped at a specific rotation angle and positioned at an arbitrary shooting position and angle within the movable range of the scanner 110 to take a photograph. By doing this at multiple rotation angles, shooting positions, and shooting angles, an image of the entire model can be obtained. Note that by driving the support arm 120 and the model support device 130 in synchronous order, the shooting process can be performed more simply and accurately. Alternatively, instead of using the model support device 130, the scanner 110 may be manually moved around the model to scan from any desired shooting position and angle.

[0016] The display device 140 is a display device such as a liquid crystal display (LCD), and can display the processing result in the information processing device 100. Specifically, it can display the image acquired by the scanner 110, display the original data of the three-dimensional model data generated from the photographed image, display the video (VR video) reconstructed using the original data, etc. Also, it can display the progress meter while the scanner processing is in progress. Further, it can display information regarding the type of the subject to be scanned and specific parameter values corresponding to the present embodiment. An example of the display screen displayed here will be described later with reference to FIG. 6. The display device 140 can include an operation unit 140A that receives an operation from the user who is an observer of the displayed video, and the user can operate the operation unit 140A to perform an operation input corresponding to the content of the video displayed on the display device 140.

[0017] The headset 150 is composed of a head-mounted display 150A and a controller 150B, which will be described later. In particular, it may be configured to be able to provide a moving image corresponding to the posture and inclination of the user as an observer as a VR headset. A predetermined application is installed in the headset 150, and the application data executed in the application can be downloaded from the information processing device 100 and executed. In the present embodiment, similar to the information processing device 100, the headset 150 can use the three-dimensional model data as data for the virtual space to generate a video to be displayed on the virtual space, and function as a video generation device that provides the generated video as a VR video. Note that the application data includes output data for outputting a VR video or the like.

[0018] The operating device 160 consists of an output unit 160A and an operation unit 160B. It provides the user with images displayed in a virtual space using three-dimensional model data provided by the information processing device 100 as virtual space data via the output unit 160A, and can receive user operations on the displayed images via the operation unit 160B. The received operation results are transmitted to the information processing device 100, and the displayed images are updated.

[0019] Next, the hardware configuration of the information processing device 100 corresponding to this embodiment will be described. Figure 2(A) shows an example of the hardware configuration of the information processing device 100. The CPU 101 is a device that performs overall control of the information processing device 100 and performs data calculation, processing, and management. For example, it controls the timing and number of images captured by the scanner 110, and also controls the joint of the support arm 120 so that the scanner 110 can be positioned at any shooting position and angle. For example, after the shooting position and angle of the scanner 110 are determined, the model support device 130 can be rotated and the scanner 110 can perform the shooting operation. Alternatively, after the model support device 130 is stopped at a predetermined rotation angle, the scanner 110 can perform the shooting operation at any shooting position and angle.

[0020] Furthermore, the CPU 101 can also function as an image processing unit that processes images output from the scanner 110. Specifically, the CPU 101 can use the three-dimensional model data generated based on the scan signal obtained from the scanner 110 as virtual space data to generate an image that displays the three-dimensional model in the virtual space. The CPU 101 also generates a rendered image from the three-dimensional model data acquired from the scanner 110, determines the values ​​of predetermined parameters regarding the appearance of the subject to be scanned, and determines the type of subject based on the determined parameter values. Based on the processing results, the CPU 101 generates output data for output to the display device 140 and the output unit 160A of the operating device 160. The output data output to the operating device 160 includes various effect data such as video data, audio data, vibration data, airflow data, and light emission data.

[0021] RAM 102 is a volatile memory and is used as the main memory and temporary storage area of ​​the CPU 101, such as the work area. ROM 103 is a non-volatile memory and stores image data, other data, and various programs for the operation of the CPU 101 in predetermined areas. The CPU 101 controls various parts of the information processing device 100 using RAM 102 as work memory, for example, according to a program stored in ROM 103. Note that the program for the operation of the CPU 101 is not limited to what is stored in ROM 103, but may also be stored in the storage device 104.

[0022] The storage device 104 is composed of, for example, a magnetic disk such as an HDD or flash memory. The storage device 104 stores application programs, the OS, control programs, related programs, game programs, etc. The storage device 104 can read and write data based on the control of the CPU 101. The storage device 104 may be used instead of RAM 102 or ROM 103. Furthermore, the storage device 104 stores information from the tables shown in Figures 7A and 7B, which will be described later.

[0023] The communication device 105 is a communication interface for communicating with the scanner 110, support arm 120, model support device 130, display device 140, headset 150, and operating device 160, based on the control of the CPU 101. The communication device 105 may also be configured to communicate with an external server or the like. The communication device 105 may include a wireless communication module, which may include a well-known circuit mechanism including an antenna system, RF transceiver, one or more amplifiers, tuners, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identification module card, memory, etc. Here, communication between the information processing device 100 and the scanner 110, support arm 120, model support device 130, display device 140, headset 150, and operating device 160 may be performed by wireless communication.

[0024] Furthermore, the communication device 105 may also include a wired communication module for wired connection. The wired communication module enables communication with other devices, including the display device 140 and the operating device 160, via one or more external ports. It may also include various software components for processing data. The external ports connect to other devices directly or indirectly via a network, such as via Ethernet, USB, or IEEE 1394. In addition, software that provides equivalent functionality to each of the above devices can be configured to replace the hardware devices.

[0025] The operation unit 106 is composed of, for example, buttons, a keyboard, a touch panel, a controller, etc., and accepts user input. The operation unit 106 may be shared with the operation unit 140A or the operation unit 160B of the operation device 160, or it may be independent of the operation unit 140A and the operation unit 160B. For example, if the operation unit 140A is assumed to be an operation unit composed of a keyboard and mouse, etc., it can be shared with the operation unit 106. On the other hand, if the operation unit 140A is assumed to be an operation unit composed of a touch panel or a controller, etc., it can be a separate operation unit from the operation unit 106.

[0026] The display control unit 107 functions as a control unit for displaying information on the display device 140 and the output unit 160A of the operating device 160, which are connected to the information processing device 100, and controls the operation of the display device 140 and the output unit 160A. Some functions of the operating unit 106 may be provided on the display device 140 and the output unit 160A. For example, the display device 140 may be configured as a device equipped with a touch panel, such as a tablet terminal.

[0027] Next, the hardware configuration of the headset 150 corresponding to this embodiment will be described. Figure 2(B) shows an example of the hardware configuration of the headset 150. The headset 150 consists of a head-mounted display (HMD) 150A and a controller 150B. The HMD 150A provides the right eye image and the left eye image to the user's left and right eyes respectively, and enables a virtual reality (VR) experience by creating a sense of depth through the parallax between them. The controller 150B is provided in a housing independent of the HMD 150A. The controller 150B is assumed to be configured as a pair of two controllers to be held in the user's left and right hands for operation, but it may also be a single controller.

[0028] The CPU 151 is a device that performs overall control of the headset 150 and performs calculations, processing, and management of data. For example, it can execute application data downloaded from the information processing device 100 and display VR images on the display 156. It can also switch the VR images to be displayed, change the viewpoint within the VR space, or change the position within the VR space based on operations received via the controller 150B or information detected by the detection unit 157.

[0029] RAM 152 is a volatile memory and is used as the main memory and temporary storage area of ​​the CPU 151, such as the work area. ROM 153 is a non-volatile memory and stores image data, other data, and various programs for the operation of the CPU 151 in predetermined areas. The CPU 151 controls various parts of the headset 150 using RAM 152 as work memory, for example, according to a program stored in ROM 153. Note that the program for the operation of the CPU 151 is not limited to what is stored in ROM 153, but may also be stored in the storage device 154.

[0030] The storage device 154 is composed of, for example, a magnetic disk such as an HDD or flash memory. The storage device 154 stores application programs, the OS, control programs, related programs, game programs, application data and output data downloaded from the information processing device 100, etc. The storage device 154 can read and write data based on the control of the CPU 151. The storage device 154 may be used instead of RAM 152 or ROM 153.

[0031] The communication device 155 is a communication interface for communicating with the information processing device 100 and the controller 150B based on the control of the CPU 151. The communication device 155 includes a wireless communication module that enables wireless communication based on Bluetooth or WiFi (IEEE802.11). The headset 150 connects to the information processing device 100 wirelessly and can download output data including data for displaying VR images. It also communicates with the controller 150B to receive information on user operation instructions for the controller 150B.

[0032] The display 156 is configured to provide the right-eye image and left-eye image generated by the CPU 151 to the user's right and left eyes, respectively. The detection unit 157 is a mechanism for detecting the gaze direction of the user's left and right eyes, as well as the tilt of the head-mounted display. The detection unit 157 includes, for example, a sensor for detecting the gaze direction or gaze direction of the user wearing the HMD 150A. It also includes a gyroscope, magnetometer, accelerometer, global positioning system (GPS), compass, etc., and can determine the position, orientation, tilt, etc. of the HMD 150A according to this detection information. The detection unit 157 detects information for determining the user's gaze direction and action in the HMD 150A and transmits it to the CPU 151. The CPU 151 determines the gaze direction and action based on the received detection information and adjusts the image presented on the display 156 of the HMD 150A to match the determined gaze direction and action.

[0033] The controller 150B is composed of, for example, multiple buttons, a directional pad, etc., and accepts user input such as selection operations and directional indication operations. The controller 150B is wirelessly connected to the HMD 150A via the communication device 155.

[0034] Next, the hardware configuration of the operating device 160 corresponding to this embodiment will be described. Figure 2(C) shows an example of the hardware configuration of the operating device 160. The operating device 160 consists of an output unit 160A and an operation unit 160B. The output unit 160A outputs video, sound, vibration, airflow, etc. to the user to enable an interactive experience. The operation unit 160B is a component in the operating device 160 that receives input from the user, and may be provided independently of the operating device 160 or integrated with the operating device 160.

[0035] The CPU 161 is a device that performs overall control of the operating device 160. For example, it processes video information, audio information, and other performance information acquired from the information processing device 100 for output via the output unit 166. It can also switch the video to be displayed, output audio, or switch performance information based on operations received via the operating unit 160B.

[0036] RAM 162 is a volatile memory and is used as the main memory and temporary storage area of ​​the CPU 161, such as the work area. ROM 163 is a non-volatile memory and stores image data, other data, and various programs for the operation of the CPU 161 in predetermined areas. The CPU 161 controls various parts of the operating device 160 using RAM 162 as work memory, for example, according to a program stored in ROM 163. Note that the program for the operation of the CPU 161 is not limited to what is stored in ROM 163, but may also be stored in the storage device 164.

[0037] The storage device 164 is composed of, for example, a magnetic disk such as an HDD or flash memory. The storage device 164 stores application programs, the OS, control programs, related programs, game programs, application data and output data downloaded from the information processing device 100, etc. The storage device 164 can read and write data based on the control of the CPU 161. The storage device 164 may be used instead of RAM 162 or ROM 163.

[0038] The communication device 165 is a communication interface for communicating with the information processing device 100 and the operation unit 160B based on the control of the CPU 161. The communication device 165 includes a wireless communication module that enables wireless communication based on Bluetooth or WiFi (IEEE802.11). The operation unit 160 connects to the information processing device 100 wirelessly and can download performance data such as video data and audio data. It also communicates with the operation unit 160B to receive information on user operation instructions for the operation unit 160B.

[0039] The output unit 166 has an effect output mechanism that provides various effects, such as a display that displays video based on video data provided by the CPU 161, a speaker that outputs audio based on audio data, a vibration unit that outputs vibration based on vibration data, a fan that outputs air based on airflow data, and a light-emitting unit that provides effects by emitting LED light based on light-emitting data.

[0040] The control unit 160B consists of, for example, multiple buttons, switches, a directional pad, a joystick, a lever, a pedal, a touch panel, a steering wheel, etc., and accepts user input such as selection operations and directional indication operations. The control unit 160B is connected to the output unit 160A of the control device 160 wirelessly or via a wired connection through the communication device 165.

[0041] Next, with reference to Figures 3A to 3C, an example of a model and its accessories that will be the subject of photography in this embodiment will be described. Model 300 is an example of a model having the appearance of a human (robot or human). This model may be assembled and painted as, for example, a plastic model. Alternatively, it may be a pre-assembled model, such as a figure (action figure) with movable joints. The model in Figure 3 is merely an example for illustrative purposes, and the shape of the model is not limited to those with the appearance of a human, but can be any shape, such as a general vehicle, racing car, military vehicle, aircraft, ship, animal, virtual life form, etc. It should be noted that the item to be photographed is not limited to a model, as long as the scanner 110 can capture its three-dimensional shape.

[0042] Model 300 comprises the following parts: head 301, right arm 302, left arm 303, torso 304, right leg 305, left leg 306, right foot 307, and left foot 308, which are joined together to form the model. At least a portion of each of the individual parts 301 to 308 is supported so as to be rotatable (or swingable) relative to an adjacent part. For example, the head 301 is supported so as to be rotatable relative to the torso 304, and the right arm 303 and left arm 304 are supported so as to be rotatable relative to the torso 304. In this way, each part of model 300 is provided with a joint structure, so that model 300 can assume any posture.

[0043] Furthermore, the model 300 may be accompanied by an accessory 309. The accessory 309 is an accessory part intended to be used with the model 300, and in Figure 3A, it is a weapon with a rifle shape. That is, it is assumed that the model 300 will hold the accessory 309 with both hands when using it. The weapon's form is not limited to a rifle, and can include any form of weapon, such as a sword, shield, bomb (e.g., grenade), or bazooka. Also, the accessory is not limited to a weapon, as long as it is an accessory part that can be used with the model 300, it may also be something like a decorative item. For example, if the model 300 is a doll-type model of a specific character, the accessory may be various items that the character carries (e.g., personal belongings, hat, bag, accessories, smartphone, or any other item that the character can carry).

[0044] Next, with reference to Figures 3B and 3C, another example of a model and its accessories that serve as the subject of photography in this embodiment will be described. Model 310 shown in Figure 3B is another example of a model having the appearance of a humanoid (robot or human). Model 320 shown in Figure 3C is yet another example of a model having the appearance of a humanoid (robot or human). Parts 311-318 and 321-328 that make up models 310 and 320, respectively, correspond to parts 301-308 of model 300, respectively. Although accessory 309 from Figure 3A is omitted in Figures 3B and 3C, accessories may be included as in Figure 3A. Compared to model 300, model 310 has a sleeker overall form and is equipped with flight wings 319, giving the impression of superior mobility. Model 320, on the other hand, is more heavily equipped than models 300 and 310, having artillery mechanisms 329 to 332, and gives the impression of being heavier overall.

[0045] In this embodiment, the subjects to be scanned can be classified into one of three types: a second type (mobile, speed type) with excellent mobility, such as model 310; a third type (heavyweight, power type) with a sense of weight, such as model 320; and a first type (general-purpose, multi-type) that is intermediate between the first two, such as model 300. The second type emphasizes a sense of speed in the video production, the third type emphasizes a sense of weight in the video production, and the first type has a sense of speed and weight that is intermediate between the second and third types in the video production. The classification of subject types will be explained in more detail later. In this embodiment, there are three types, but at least three types are sufficient, and more types may be provided.

[0046] Next, with reference to Figure 4, an overall implementation example of the image processing system 10 in this embodiment will be described. Figure 4 is a generalized diagram showing an implementation example capable of generating the actual data generated by photographing a model.

[0047] In Figure 4A, the case 401 contains the information processing device 100, a drive system for driving the support arm 402, a drive system for driving the turntable 406, and the like. The support arm 402 can also be manually adjusted for shooting direction and position. The surface of the case 401 is flat so that promotional posters can be attached.

[0048] The support arm 402 corresponds to the support arm 120 and can support and fix the position of the terminal 403, which functions as a scanner 110, either under the control of the information processing device 100 or manually. The support arm 402 can also operate to control the tilt of the terminal 403.

[0049] Terminal 403 is a touch-panel terminal with a built-in camera that can be used as a scanner 110. For example, a smartphone, tablet, or digital camera can be used. Instead of these terminals, an Artec Space Spider can also be used. Figure 4A is merely a diagram showing a generalized example of the system configuration, and the configuration can be adjusted according to the type of device used as the scanner 110. Terminal 403 can capture images of the model 300 and transmit them to the information processing device 100. The ring light 404 is a lighting device used when terminal 403 photographs the model 300, and it illuminates the model 300 evenly, minimizing shadows. In addition to the ring light 404, auxiliary lights such as a top light, left and right lights, and bottom lights may be installed as additional light sources. Instead of terminal 403, a three-dimensional scanner device as described above may be used.

[0050] The background sheet 405 is a background sheet for photography, and for example, a white sheet can be used. The turntable 406 can mount the model 300 and rotate the model. The turntable 406 and its drive system are configured to correspond to the model support device 130, and the turntable 406 rotates clockwise and counterclockwise and stops according to the control of the information processing device 100. Multiple predetermined markers 410 may be placed on the turntable 406. The markers 410 can be used to adjust the orientation and position of the photographed model.

[0051] In Figure 4A, the model is placed on a translucent (transparent) stand, but other support devices, such as those called "Action Base" (registered trademark), may also be used. An Action Base has a support column on a base that is bent into a "V" shape, and the model can be attached to the tip of the support column. In this case, the marker 410 mentioned above may be placed on the tip of the support column. The method of supporting the model can be changed depending on the pose. For example, in an upright position, the model can be placed on a transparent stand for photography. On the other hand, if you want to photograph the soles of the feet, such as in a flying pose, you may use an Action Base. Note that an Action Base may also be used to photograph an upright position.

[0052] The display device 407 is a device that corresponds to the display device 140 and may have a touch panel function. The user can perform predetermined selection operations using the touch panel function. Alternatively, operations can be accepted by a controller or mouse (not shown) that can function as the operation unit 140A instead of the touch panel. The VR headset 408 consists of an HMD 408A and a controller 408B, and corresponds to HMDs 150A and 150B, respectively. The user can operate the headset while viewing VR images by wearing the HMD 408A on their head and holding the controllers 408B in their left and right hands.

[0053] Next, with reference to Figure 4B, a more specific configuration of the image processing system 10 corresponding to this embodiment, which simultaneously scans the model 300 and the accessory 309, will be described. In Figure 4B, the model 300 and the accessory 309 are placed on the turntable 406. The model 300 is supported by an action base 411 and is positioned approximately in the center of the turntable 406. The accessory 309 is supported by a support member 412A within the frame 412 and is positioned adjacent to the model 300 and near the end of the turntable 406. The support member 412A can be, for example, a clip-shaped member. This allows the accessory 411 to be held in place by the clip and supported within the frame 412.

[0054] The scanner device 413 is supported by a support arm 402, and in this example, it is assumed that an Artec Space Spider scanner is used. The drive system for driving the support arm 402, the drive system for driving the turntable 406, etc., are the same as those described in Figure 4A, so their explanation is omitted. In this embodiment, the turntable 406 can rotate in both clockwise and counterclockwise directions.

[0055] In this embodiment, the model 300 and accessories 309 are placed together on the turntable 406 in the configuration shown in Figure 4B, and a scanner device 413 captures them together to generate a scanned image. The scanned image will include both the model 300 and accessories 309, but the three-dimensional model data is generated separately from them.

[0056] The frame 412 supporting the accessory 309 is configured as a frame surrounding the accessory 309 and is used as an indicator to identify the area where the accessory 309 is located. That is, the three-dimensional object located inside the frame 412 is recognized as the accessory 309. The thickness of the frame 412 should be as thin as possible while maintaining strength so as not to obstruct the line of sight of the scanner device 413. For example, it can be about 3 mm thick.

[0057] Furthermore, the model 300 and the frame 412 are positioned at a certain distance from each other. Depending on the rotation angle of the turntable 406, the frame 412 may be positioned in front of the model 300 relative to the scanner device 413, obstructing the imaging of the model 300. While it is necessary to compensate for such obstructed areas at a different rotation angle, the compensation accuracy decreases if the model 300 and the frame 412 are too close together. Therefore, a certain distance is maintained to ensure compensation accuracy.

[0058] In this case, to ensure a certain distance is maintained between the model 300 and the frame 412 (or to prevent them from getting too close), the sizes of the model 300 and the accessory 309 may be limited to predetermined sizes or smaller. For example, the size of the model 300 may be 15 cm in height, 15 cm in width, and 10 cm in depth or less, and the size of the accessory 309 may be 15 cm in height, 5 cm in width, and 2 cm in thickness or less.

[0059] Furthermore, in this embodiment, the mounting base 414 on which the action base 411 supporting the model 300 is installed is thicker than the mounting base 415 on which the frame 412 supporting the accessory 309 is installed. That is, the model 300 is installed so that it is positioned relatively higher than the accessory 309. For example, the model 300 can be installed so that at least its upper body, or upper half, is positioned above the frame 412. This reduces the proportion of the model 300 that is obscured by the accessory 309 when scanning, thereby increasing the success rate of the scan.

[0060] Next, with reference to Figure 4C, a more specific configuration of the operating device 160 corresponding to this embodiment will be described. In Figure 4C, the operating device 160 is shown as having an output unit 160A, an operation unit 160B, and a seat 160C. When a user sits in the seat 160C, the output unit 160A is positioned to cover the front, allowing the user to experience an immersive sensation. The output unit 160A has displays in at least three directions: the front and the left and right, and displays images to be displayed in the virtual space using three-dimensional model data provided by the information processing device 100 as virtual space data. The output unit 160A may also have a speaker for audio output. The user of the seat 160C can perform operation input by operating the operation unit 160B, which is configured as a lever or pedal, and the received operation results are transmitted to the information processing device 100, which updates the displayed image and output sound according to the operation content. The seat 160C is configured to vibrate in response to the displayed image and the operation of the operation unit 160B. Furthermore, a speaker for audio output may also be placed in seat 160C, allowing for audio output synchronized with the video or with the vibrations of seat 160C. In addition, a fan may be placed in output unit 160A and seat 160C to provide airflow synchronized with the video and audio. Furthermore, an LED light-emitting unit may be appropriately placed in output unit 160A to provide light output.

[0061] Next, with reference to Figure 5, an example of processing performed by the image processing system 10 corresponding to this embodiment will be described. At least a part of the processing corresponding to the flowchart is realized by the CPU 101 of the information processing device 100 executing a program stored in the ROM 103 or the storage device 104.

[0062] First, in S501, CPU 101 accepts user registration. It accepts input of the user's name and contact information. Each user is assigned a user identifier to uniquely identify them. CPU 101 stores the entered user information in storage device 104, associating it with the time the input was received and the user identifier.

[0063] Once user registration is complete, the user places their model in the model support device 130. The CPU 101 can determine whether the model 300 and accessories 309 have been placed in the model support device 130 based on the scanned image from the scanner 110. Alternatively, a switch that turns on when the model 300 and accessories 309 are placed in the model support device 130 may be provided, and the CPU 101 may detect the signal from the switch to make the determination. Alternatively, a button for accepting operation when the model 300 and accessories 309 have been placed may be displayed on the display device 140, and the CPU 101 may detect whether an operation in response to the button has been received. In S502, the CPU 101 detects that the model 300 and accessories 309 have been placed in the model support device 130 using one of the methods described above. In response to this detection, the process proceeds to S503.

[0064] Here, if a certain amount of time has elapsed after at least the model 300 has been set and the setting of the accessory 309 is not detected, the process may proceed to S503. Alternatively, the system may only detect whether the accessory 309 has been set if the frame 412 is set.

[0065] In S503, the scanner 110, support arm 120, and model support device 130 work together to perform scanning (photography) and generate a scanned image. Specifically, the CPU 101 controls the support arm 120 to move the scanner 110 to one of the registered shooting positions, and the scanner 110 can perform scanning while the model support device 130 rotates at that shooting position. After stopping the model support device 130 at a predetermined rotation angle, the scanner 110 can perform a shooting operation at any shooting position and angle. Specifically, with the front of the model 300 and accessory 309 positioned on the front side of the scanning device 413, the support arm 402 is moved to its limit of motion to scan the front, left and right sides, top, and bottom surfaces of the model 300 and accessory 309. After that, the turntable 406 is rotated to scan the model 300 and accessory 309 at a predetermined rotation angle. At that time, the scan is performed to complement the parts of the model 300's exterior that are obscured by the frame 412 and accessories 309 (the surfaces of the model 300 on which the frame 412 and accessories 309 are located). In S503, for example, the scan operation can be switched to change the scan range depending on the presence or type of accessories.

[0066] In the subsequent S504, the CPU 101 performs post-scan processing on the scanned images acquired through the scanning process to generate three-dimensional model data for both the model 300 and the accessory 309. When generating the three-dimensional model data, if three-dimensional model data for the accessory 309 is generated, the three-dimensional model data for the accessory 309 is associated with a predetermined position in the three-dimensional model data for the model 300. For example, if the accessory is a type that is held by hand, on the other hand, if three-dimensional model data for the accessory 309 is not generated, no such association is made.

[0067] In post-scan processing, an area is created in advance to define the range of unnecessary data, and the data is read and the faces included in the unnecessary area are deleted. This makes it possible to remove almost all faces other than the model 300 and accessories 309. The unnecessary data range includes the areas where the action base 411, mounting base 414, frame 412, support member 412A, mounting base 415, etc., are located.

[0068] In the subsequent S505, the CPU 101 performs a type determination process to determine the type of subject to be scanned based on the three-dimensional model data generated in S504. In this type determination process, the CPU 101 compares, for example, a rendered image obtained by rendering using the three-dimensional model data with a reference image stored in the storage device 104, and determines a type value related to the type of subject based on the degree of agreement with the reference image.

[0069] The reference images include images of models pre-classified as Type 1 and their renderings, images of models pre-classified as Type 2 and their renderings, and images of models pre-classified as Type 3 and their renderings. The renderings are generated using a viewpoint set from a diagonally upward angle, which can adequately cover models of different sizes and convey the overall atmosphere. The background color can be, for example, 50% gray.

[0070] The type value indicates the degree to which the subject belongs to that type, with each of the three types (Type 1, Type 2, and Type 3) being assigned a value from 0 to 10. 0 indicates the lowest degree of belonging to that type, while 10 indicates the highest degree of belonging to that type.

[0071] In the type determination process, if features of the first, second, and third type reference images are extracted from the rendering image being determined, the process is performed so that the value of the corresponding type is higher. For example, the second type has structural features that enable it to move quickly. Specifically, these include features such as limbs and torso that appear thinner compared to other types, the addition of wing-like parts such as thin plates or sharp shapes, and the attachment of many thrusters to obtain propulsion.

[0072] Furthermore, the third type has structural characteristics that prioritize equipment and superior defensive capabilities over speed, as seen in the second type. Specifically, it includes features such as limbs and torso that appear relatively thicker compared to other types, the addition of defensive components such as shields and armor, and the use of heavy weapons such as large-caliber cannons.

[0073] Type 1 is intermediate between Type 2 and Type 3, and is characterized by not exhibiting significant characteristics of either Type 2 or Type 3.

[0074] Reference images can include images of the model itself as well as images of accessories. If the rendering image includes images of accessories, the degree of similarity of the accessory features can be further referenced to determine the type of subject. For example, if the rendering image includes accessories that are similar to or match those included in the reference image of a third-type subject, the value for the third type can be added. Furthermore, reference images can be prepared as images taken from multiple viewpoints, and type determination processing can be performed using rendering images generated from multiple viewpoints. This is because the type of subject can be determined by referring to not only the features of the front but also the features of the back.

[0075] In this way, once the type values ​​for the first to third types of the subject to be judged are obtained, the CPU 101 determines the type of the subject based on the obtained values. Specifically, the type can be determined based on the type value that has the largest value.

[0076] For example, if the value of the third type is the largest, such as Td(Type value of the first type: T1, Type value of the second type: T2, Type value of the third type: T3)=(7, 6, 8), it is determined to be the third type. Also, if the values ​​of the first type, the second type, and / or the third type match, and that value is larger than the others, it can be determined to be the first type. For example, if the value of the first type is the largest and matches the other types, such as (T1, T2, T3)=(8, 8, 7), it is determined to be the first type. This is because the first type is assigned to a general-purpose type.

[0077] Once the type has been determined, the individual parameter values ​​(multiple first parameter values) for each type are then determined. In this embodiment, five types of parameters are assumed, but there may be fewer or more. The five types include, for example, Operability as parameter 1, Mobility as parameter 2, Rapid fire as parameter 3, Armor as parameter 4, and Attack power as parameter 5. These parameters can be calculated based on the type values ​​used in the type determination described above.

[0078] In this embodiment, the following reference values ​​can be set for each of the first, second, and third types. For Parameter 1, operability, (reference value for Type 1: R11, reference value for Type 2: R21, reference value for Type 3: R31) = (10, 3, 4). Similarly, for Parameter 2, mobility, (R12, R22, R32) = (4, 10, 1); for Parameter 3, attack speed, (R13, R23, R33) = (4, 8, 2); for Parameter 4, defense, (R14, R24, R34) = (4, 2, 8); and for Parameter 5, attack power, (R14, R24, R34) = (4, 1, 10).

[0079] Then, the values ​​of multiple first parameters, from parameter 1 to parameter 5 (Pn: P1 to P5), can be calculated, for example, using the following calculation formula. Pn=(R1n*T1 2 / 100)+(R2n*T2 2 / 100)+(R3n*T3 2 / 100) (where n=1~5) (Equation 1)

[0080] For example, if it is determined to be type 1 and the type values ​​are (T1, T2, T3) = (7, 6, 5), the value of the operability parameter P1 is P1 = (10 × 49 / 100) + (3 × 36 / 100) + (4 × 25 / 100) = 6.98. Parameters P2 through P5 can be calculated in the same way. The results obtained in this way are normalized so that the sum is 30.

[0081] For the type determination process described above, a generative AI can also be used. This can be achieved by generating and inputting prompts that instruct the generative AI on the conditions for type determination as described above, and then having the generative AI execute them.

[0082] In the subsequent S506, the result of the type determination process in S505 is output. For example, the determination result display screen can be displayed on the display device 140. On this display screen, the type of the subject determined in S505 and each of the multiple first parameter values ​​from parameter 1 to parameter 5 for that determined type can be displayed using a pentagonal graph (radar chart). At this time, the rendering image obtained by rendering using the three-dimensional model data generated in S504 may be included on this screen. For example, the determination result display screen can be displayed next to the rendering image. This allows the user to check the rendering image of the model captured by scanning and determine whether the appropriate type classification has been made.

[0083] Figure 6 shows an example of a display screen for the type determination result. As described above, there are three types for the type determination result, from the first type to the third type. Display screen 600 shows an example of the display when the first type, general-purpose type, is determined. Here, the values ​​of each parameter are displayed in the radar chart 601, and the determined type is displayed in the type display area 602. Display screen 600 displays "General-purpose type (multi-type)". Display screen 610 shows an example of the display when the second type, mobile type, is determined. The values ​​of each parameter are displayed in the radar chart 611, and the determined type is displayed in the type display area 612 as "Mobile type (speed type)". Display screen 620 shows an example of the display when the third type, heavy-duty type, is determined. The values ​​of each parameter are displayed in the radar chart 621, and the determined type is displayed in the type display area 622 as "Heavy-duty type (power type)".

[0084] Each screen displays OK buttons 603, 613, and 623, which the user operates to accept the judgment result. The user operates the OK button if they accept the judgment result. If the user does not accept the judgment result, for example, the user may be allowed to change the displayed type. Specifically, the user may be able to change the type by selecting the type display area 602, etc. The user confirms the type by selecting the OK button 603, etc., with the changed content.

[0085] In the subsequent S507, output processing is performed. In this output processing, if performed in the information processing device 100, the CPU 101 generates application data for video display (display data) that includes effects (or information specifying the effects) according to the type of subject, based on the three-dimensional model data of the model 300 and accessories 309 generated in S504 and the type of subject determined in S506. At this time, in addition to the generated display data, sound data, vibration data, air blowing data, light emission data, etc., can also be generated to include effects (or information specifying the effects) according to the type of subject. These are collectively called output data, and when executed by a specific application in the information processing device 100, the generated output data can play back video, sound, etc. Furthermore, when executed by the same specific application in the head-mounted display 150A, it can play back VR video, as well as provide vibration output, sound output, etc. The generated three-dimensional model data and output data are stored in the storage device 104 in association with user information, etc. Furthermore, the CPU 101 executes a specific application corresponding to the output data and outputs it to the display device 140 or the output unit 160A of the operating device 160 via the display control unit 107. The output data may be generated to include at least one of the above-mentioned display data, audio data, vibration data, airflow data, and light emission data.

[0086] In addition, in S507, the CPU 101 may control the communication device 105 to transmit output data to the HMD 150A, and the CPU 151 of the HMD 150A may perform output processing including video display by executing the received output data in the corresponding application to display VR video. Alternatively, output processing may be performed by transmitting output data to the operating device 160, and the CPU 161 of the operating device 160 executing the received output data in the corresponding application to perform processing such as video display and audio output.

[0087] Thus, in the video display, the effects on the model and accessories can be switched according to the type determined in S506. For example, if the subject type is the second type, a mobile type, the effects can be displayed to emphasize the speed of the model's movement. Also, if the subject type is the third type, a heavily armed type, when displaying the effect of bullets being fired from a shooting weapon, the size of the bullets may be made larger than for other types. On the other hand, the number of bullets fired per unit time can be made higher for the second type, lower for the third type, and somewhere in between for the first type. In this case, the bullets may be displayed in different colors depending on the type.

[0088] Furthermore, the movements (motions) in attack actions may differ depending on the type. For example, the second type may have movements that emphasize speed, the third type may have movements that emphasize weight, and the first type may have movements that balance speed and weight.

[0089] In this embodiment, information related to the performance for each type can be stored in the storage device 104 of the information processing device 100 in a table as shown in Figure 7A. In the table 700 of Figure 7A, multiple second parameter values ​​common to each of the first type, second type, and third type can be registered. The multiple second parameter values ​​include movement speed, boost energy, rate of fire, hit score, damage reduction, and damage durability, and different values ​​(some may be the same) can be set for each type. In the video display processing, the CPU 101 can read the information from the table 700 from the storage device 104 and determine the parameters for the performance processing by taking the information from the table 700 into consideration.

[0090] Table 700 in Figure 7A can be referenced, for example, when the CPU 101 executes a specific application corresponding to the output data described above, and generates game footage in which the user manipulates a three-dimensional model based on the output data. In Table 700, the parameter value for the first type is set to a baseline value of 5, with a value of 7 for higher performance and 3 for lower performance; however, this value assignment is merely an example and is not limited to this example.

[0091] Movement speed represents the movement speed of the 3D model, with Type 2 having high movement speed (7) and Type 3 having low movement speed (3). Boost energy indicates the amount of "boost gauge" consumed within a certain period of time (when it reaches zero, boosting becomes impossible), with Type 2 having high (high gauge consumption: 7) and Type 3 having low (low gauge consumption: 3). Rate of fire represents the number of attacks within a certain period of time, with Type 2 having high (high-speed rapid fire possible: 7) and Type 3 having low (poor at rapid fire: 3). Hit score represents the score obtained when an attack hits, with Type 2 having low (low score due to rapid fire capability: 3) and Type 3 having high (high score due to poor rapid fire capability: 7). Damage deduction represents the degree of deduction relative to the amount of damage taken (damage evasion performance). Type 2 has a high deduction (high ability to evade attacks and reduce damage due to high movement speed: 7), and Type 3 has a low deduction (low ability to evade attacks and reduce damage due to slow movement speed: 3). Damage durability represents the degree of durability when taking damage. Type 2 has low durability (3), and Type 3 has high durability (7). Both damage deduction and damage durability may be used, or only one of them may be used. For example, a base deduction amount can be set for each type of attack, and the amount of deduction when taking damage can be determined using the values ​​of damage deduction and damage durability (or one of them).

[0092] The parameter values ​​shown in Figure 7A are merely examples, and additional parameters can be registered. For example, vibration parameters can be added, and the vibration parameter values ​​can be set to decrease in the order of Type 3, Type 1, and Type 2. This allows for a larger vibration output for Type 3 in the performance processing. Similarly, values ​​for sound, airflow, and light emission can be set according to the type.

[0093] Furthermore, in the flowchart of Figure 5, the scanning process begins when it is detected that a model has been set in S502. However, it is also possible to accept information input from the user before the scanning process begins. Specifically, the type and name of the model to be scanned, the type and name of the accessories, and the desired type can be accepted. For example, models and accessories may each have names, and this information can be used for type determination. Also, if the user wishes to experience the performance multiple times using the same model, it would be uninteresting to always have the same type determined. Therefore, it is possible to accept the user's preference for different types in advance.

[0094] When additional information is received from the user in this manner, information such as the table 710 shown in Figure 7B is stored in the storage device 104, and the CPU 101 can determine the type of subject to be scanned by comparing it with the input information in S504. The table 710 in Figure 7B shows an example of the configuration of a data table for managing reference image data. The table 710 in Figure 7B includes the name of the model and the name of the accessories shown in the reference image, associated with the reference image data for each type. If there are multiple types of accessories, they may be registered separately. In that case, each accessory may be registered under a different type. The reference image may include images of the model itself as well as images of multiple accessories.

[0095] When the user provides information about the model's name and accessories, CPU101 can search the registered contents of Table 710, identify the corresponding model, obtain a reference image, and compare it with the rendered image. This process reduces the processing load and allows for more efficient identification of the type of model being scanned.

[0096] If the user provides input of a desired type, the CPU 101 can, in principle, proceed with processing to adopt the type specified by the user. At this time, the CPU 101 can adjust the value of the determined type Td, which was determined in the type determination process S505 described above, so that the value of the type corresponding to the type desired by the user is maximized. Furthermore, in determining the parameter values ​​P1 to P5 for each type, the adjusted value of determined type Td may be used to calculate each parameter value.

[0097] Furthermore, if the user provides input of a desired type, the CPU 101 may adopt the type specified by the user under certain conditions. For example, the condition may be that the difference between the value of the specified type and the values ​​of other types in the determination type Td determined in the type determination process S505 described above is not greater than a threshold. Specifically, if the difference between the value of the specified type and the values ​​of other types is, for example, 3 or more, the user specification will not be accepted due to bias, and if the difference is, for example, 2 or less, the user specification will be accepted. In addition, if the value of the specified type in the determination type Td is the lowest value (the third lowest value), the user specification may not be accepted, and if it is the second lowest value, the user specification may be accepted. Furthermore, the user specification may be accepted only under other predetermined conditions not mentioned above.

[0098] Furthermore, in this embodiment, a notification unit can be provided to notify the status of the processing shown in Figure 5. For example, an LED tape may be attached to at least one of the turntable 406, action base 411, mounting base 414, mounting base 415, etc., and notification may be given by illuminating the LED tape in different colors corresponding to the processing steps during the processing shown in Figure 6. For example, during the processing in S501 and S502, the LED tape may be illuminated in a first light color (e.g., red). After that, when scanning starts, the LED tape may be illuminated in a second light color (e.g., white). After the scanning process is completed, in the processing from S504 onwards, the LED tape may be illuminated in a third light color (e.g., blue). In each case, the illumination may be continued throughout the processing, or it may be illuminated for a certain period of time (e.g., 5 seconds or 10 seconds) and then turned off.

[0099] As described above, in this embodiment, when generating display data for displaying an image by placing three-dimensional model data generated from images obtained by scanning a model in a virtual space, the model and its accessories are simultaneously photographed to generate three-dimensional model data, and the type of subject can be determined based on the rendering image obtained from the generated three-dimensional model data. The determined type of subject can be used to switch various effects when outputting an image to the user. In other words, in this embodiment, the user can be provided with effects corresponding to the type of scanned subject. Furthermore, the type of subject can be not only determined by the information processing device 100 but also specified by the user, so the user can experience the effects they expect.

[0100] In the embodiments described above, the case in which the subject type is classified into three types—general-purpose, mobile, and heavily armored—was explained, but the embodiments of the invention are not limited to these. The type can be changed depending on the type of subject to be scanned. For example, if the subject to be scanned is a model or doll of an animal or virtual life form, the type may be set from different perspectives, such as herbivorous, carnivorous, and omnivorous, or cute, imposing, and gentle. In these cases as well, similar to the embodiments described above, a rendering image can be generated from the three-dimensional model data obtained for the subject to be scanned, and the corresponding type can be determined by comparing it with a reference image for each classification. Furthermore, the presentation can be switched according to the determined type.

[0101] Furthermore, although the above-described embodiment assumed weapons as accessories, the type of accessories is not limited to weapons. For example, if the model is a specific humanoid character, the accessories may be clothing, accessories worn by that character, etc. It is also possible to have the model appear in events, concerts, sports, online meetings, etc., held in a virtual space. Moreover, the technology of this embodiment can also be applied to video technologies that merge the real world and the virtual world, such as cross-reality (XR), making it possible to perceive things that do not exist in the real world.

[0102] In this embodiment, three-dimensional model data is generated from images of the exterior of the model and its accessories, and it becomes possible to view a video in which the three-dimensional model is placed in a virtual space. Some models and accessories, such as assembly-type plastic models, are finished by the user through painting and other processes to become unique works of art, and their individuality can be reflected in the video and character representation in the virtual space, thereby significantly improving the user experience.

[0103] <Summary of Embodiments> The above embodiments disclose at least the following information processing device and computer program. (1) An information processing device, A scan control means that controls the position and angle of the scanner to scan the appearance of an object placed on a mounting stand and generate a scanned image, A data generation means for generating three-dimensional model data from the aforementioned scanned image, A determination means for determining the type of the subject from among multiple types based on a rendering image of the subject generated based on the three-dimensional model data, A data generation means that generates output data including an image of a three-dimensional model placed in a virtual space based on the aforementioned three-dimensional model data, Output means for outputting the aforementioned output data to an external output device Equipped with, The data generation means generates the output data so as to include a performance for each subject according to the type of subject determined by the determination means. (2) The determination means determines the type of the subject based on a comparison between the rendered image and a plurality of reference images, The information processing apparatus according to (1), wherein each of the aforementioned multiple reference images is associated with one of the aforementioned multiple types. (3) The information processing apparatus according to (2), wherein the determination means determines the type of the subject based on the type associated with a reference image that has a high degree of agreement with the rendered image. (4) The subject includes a first subject and a second subject, the second subject being an accessory to the first subject, If the rendering image includes a first rendering image of the first subject and a second rendering image of the second subject, the determination means determines the type of the subject based on the degree of agreement with the first rendering image and the type associated with a reference image that has a high degree of agreement with the second rendering image, according to (3). (5) When the determination means receives input from a user regarding the subject, it further refers to the input information to determine the type of the subject, the information processing device according to any one of (2) to (4). (6) Each of the aforementioned reference images is associated with information about the corresponding subject, The information processing apparatus according to (5), wherein, upon receiving input of information relating to the subject, the determination means further determines the type of the subject based on the degree of agreement between the input information relating to the subject and the corresponding information relating to the subject. (7) The information processing apparatus according to (5) or (6), wherein the information relating to the subject includes information relating to the names of the first subject and the second subject. (8) The plurality of types includes at least three types, The information processing apparatus according to any one of (1) to (7), wherein the determination means determines a type value indicating the degree to which each of the three types belongs to that type, and determines the type of the subject based on the type value. (9) The above-mentioned three types include the first type, the second type, and the third type, The second type is one in which the first characteristic is emphasized in the direction of the video, and the second characteristic, which is different from the first characteristic, is suppressed. The third type is one in which the second characteristic is emphasized and the first characteristic is suppressed in the direction of the video. The information processing apparatus according to (8), wherein the first type is positioned between the second type and the third type in terms of the presentation of the first and second characteristics in the image. (10) The information processing apparatus according to (9), wherein the first characteristic is a sense of speed in the performance, and the second characteristic is a sense of weight in the performance. (11) The information processing apparatus according to (9) or (10), wherein the determination means determines the type of the subject based on the type value which has the largest value. (12) The information processing apparatus according to (11), wherein the determination means determines the type of the subject to be the second type if there are multiple type values ​​with the largest value. (13) The determination result in the determination means is displayed on the display unit, The information processing apparatus according to (9), wherein the display screen of the display unit includes a display of the determined type of subject. (14) The determination means calculates a plurality of first parameter values ​​common to each of the three types based on the type value, The information processing apparatus according to (13), wherein the display screen further includes displaying a plurality of first parameter values ​​calculated for the type of subject being displayed. (15) The information processing device described in (14), wherein the display of the plurality of first parameter values ​​is shown using a radar chart. (16) Further comprising means for receiving operations from the user, The information processing apparatus according to any one of (1) to (15), wherein, when an operation to change the type of subject is received via the operation means in response to the display of the type of subject shown on the display unit, the data generation means generates output data that includes a performance for each subject corresponding to the changed type of subject received via the operation means. (17) The information processing device according to any one of (1) to (16), wherein the effect includes at least one of video, sound, vibration, airflow, and light emission. (18) The aforementioned performance uses a plurality of second parameter values ​​that are common to the plurality of types and different for each of the plurality of types, The information processing apparatus according to (17), wherein the data generation means generates the output data to include a performance based on a plurality of second parameter values ​​selected according to the type of subject determined by the determination means. (19) The information processing apparatus according to any one of (1) to (18), wherein the output device includes at least one of a head-mounted display device and an operating device including an output unit and an operating unit. (20) A computer program that causes a computer to function as an information processing device as described in any one of items (1) through (19).

[0104] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention.

Claims

1. An information processing device, A scan control means that controls the position and angle of the scanner to scan the appearance of an object placed on a mounting stand and generate a scanned image, A data generation means for generating three-dimensional model data from the aforementioned scanned image, A determination means for determining the type of the subject from among multiple types based on a rendering image of the subject generated based on the three-dimensional model data, A data generation means that generates output data including an image of a three-dimensional model placed in a virtual space based on the aforementioned three-dimensional model data, Output means for outputting the aforementioned output data to an external output device Equipped with, The data generation means generates the output data so as to include a performance for each subject according to the type of subject determined by the determination means.

2. The determination means determines the type of the subject based on a comparison between the rendered image and a plurality of reference images. The information processing apparatus according to claim 1, wherein each of the plurality of reference images is associated with one of the plurality of types.

3. The information processing apparatus according to claim 2, wherein the determination means determines the type of the subject based on the type associated with a reference image that has a high degree of agreement with the rendered image.

4. The subject includes a first subject and a second subject, the second subject being an accessory to the first subject. If the rendering image includes a first rendering image of the first subject and a second rendering image of the second subject, the determination means determines the type of the subject based on the degree of agreement with the first rendering image and the type associated with a reference image that has a high degree of agreement with the second rendering image, according to claim 3.

5. The information processing apparatus according to claim 4, wherein, upon receiving input from a user regarding the subject, the determination means further refers to the input information to determine the type of the subject.

6. Each of the aforementioned reference images is associated with information about the corresponding subject, The information processing apparatus according to claim 5, wherein, upon receiving input of information relating to the subject, the determination means further determines the type of the subject based on the degree of agreement between the input information relating to the subject and the corresponding information relating to the subject.

7. The information processing apparatus according to claim 6, wherein the information relating to the subject includes information relating to the names of the first subject and the second subject.

8. The aforementioned multiple types include at least three types, The information processing apparatus according to claim 6, wherein the determination means determines a type value indicating the degree to which each of the three types belongs, and determines the type of the subject based on the type value.

9. The aforementioned three types include the first type, the second type, and the third type. The second type is one in which the first characteristic is emphasized in the direction of the video, and the second characteristic, which is different from the first characteristic, is suppressed. The third type is one in which the second characteristic is emphasized and the first characteristic is suppressed in the direction of the video. The information processing apparatus according to claim 8, wherein the first type is positioned between the second type and the third type in the presentation of the first and second characteristics in the image.

10. The information processing apparatus according to claim 9, wherein the first characteristic is a sense of speed in the performance, and the second characteristic is a sense of weight in the performance.

11. The information processing apparatus according to claim 9, wherein the determination means determines the type of the subject based on the type value which has the largest value.

12. The information processing apparatus according to claim 11, wherein the determination means determines the type of the subject to be the second type if there are multiple type values ​​with the largest value.

13. The determination result in the determination means is displayed on the display unit. The information processing apparatus according to claim 9, wherein the display screen of the display unit includes a display of the determined type of subject.

14. The determination means calculates a plurality of first parameter values ​​common to each of the three types based on the type value, The information processing apparatus according to claim 13, wherein the display screen further includes displaying a plurality of first parameter values ​​calculated for the type of subject being displayed.

15. The information processing apparatus according to claim 14, wherein the display of the plurality of first parameter values ​​is shown using a radar chart.

16. It further includes a means of operation that accepts input from the user, The information processing apparatus according to claim 14, wherein when an operation to change the type of subject is received via the operation means in response to the display of the type of subject shown on the display unit, the data generation means generates output data that includes a performance for each subject corresponding to the changed type of subject received via the operation means.

17. The information processing apparatus according to claim 16, wherein the aforementioned effect includes at least one of video, sound, vibration, airflow, and light emission.

18. The aforementioned performance uses a plurality of second parameter values ​​that are common to the plurality of types, and different for each of the plurality of types. The information processing apparatus according to claim 17, wherein the data generation means generates the output data to include a performance based on the plurality of second parameter values ​​selected according to the type of subject determined by the determination means.

19. The information processing apparatus according to claim 18, wherein the output device includes at least one of a head-mounted display device or an operating device including an output unit and an operating unit.

20. A computer program for causing a computer to function as an information processing device according to any one of claims 1 to 19.

Images