Information processing device and computer program

Abstract

The invention relates to an information processing apparatus and a computer program. When an object is imaged and an image in which a three-dimensional model is disposed in a virtual space is generated using three-dimensional model data of the imaged object, the generated three-dimensional model data can be associated with attribute information of the imaged object to generate an image corresponding to the attribute information. This information processing device is provided with: a scanning control means for controlling the position and angle of a scanner, scanning the appearance of a subject, and generating a scanned image; a data generation mechanism which generates three-dimensional model data according to the scanning image; an image generation means for generating an image in which a three-dimensional model based on the three-dimensional model data is arranged in a virtual space; and a receiving means for receiving input of attribute information relating to the object to be scanned and the performance of the object. The image generating means generates the image so as to include the performance of each of the subjects corresponding to the attribute information.

Description

[0001] Information about divisional applications

[0002] This case is a divisional application. The parent case of the divisional application is an invention patent application with an application date of February 7, 2025, application number 202510138072.0, and invention name "Information processing device and computer program". Technical Field

[0003] The present invention relates to an information processing device and a computer program. Background Art

[0004] Patent document 1 describes a technology for mapping textures generated by a group of photographic images of an object photographed from multiple photographic directions by a photographic unit onto graphic elements in a virtual space, thereby generating a virtual space image that can be viewed from a virtual camera in the virtual space.

[0005] [Background Technology Literature]

[0006] [Patent Document]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2020-107251 Summary of the invention

[0008] [Problems to be solved by the invention]

[0009] In the generation of a virtual space image such as the above-described technology, it is expected that when an object is photographed and the three-dimensional model data of the photographed object is used to generate an image in which the three-dimensional model is arranged in a virtual space, the generated three-dimensional model data can be associated with the attribute information of the photographed object to generate an image corresponding to the attribute information.

[0010] Therefore, its purpose is to, when photographing an object and using the three-dimensional model data of the photographed object to generate an image in which the three-dimensional model is arranged in a virtual space, be able to associate the generated three-dimensional model data with the attribute information of the photographed object to generate an image corresponding to the attribute information.

[0011] [Technical means to solve the problem]

[0012] One aspect for solving the above problem is an information processing device comprising:

[0013] A scanning control mechanism controls the position and angle of the scanner to scan the appearance of the object set on the setting table and generate a scanned image;

[0014] A data generating mechanism for generating three-dimensional model data according to the scanned image;

[0015] An image generating mechanism for generating an image in which a three-dimensional model based on the three-dimensional model data is arranged in a virtual space;

[0016] an output mechanism for outputting the image generated by the image generating mechanism to a display device; and

[0017] An accepting unit accepts input of attribute information related to a subject to be scanned and a performance of the subject; and

[0018] The image generation unit generates the image in a manner that includes a performance of each of the subjects corresponding to the attribute information.

[0019] [Effects of the Invention]

[0020] When photographing an object and using the three-dimensional model data of the photographed object to generate an image of the three-dimensional model arranged in a virtual space, it is possible to associate it with the attribute information of the photographed object, generate the three-dimensional model data, and generate an image corresponding to the attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a diagram showing a configuration example of an image processing system 10 according to the embodiment.

[0022] Figure 2 (A) and (B) in FIG. 1 are diagrams showing an example of the hardware configuration of the information processing device 100 and an example of the hardware configuration of the headphones 150 according to the embodiment.

[0023] Figure 3 It is a diagram showing an example of the appearance of a doll-type model corresponding to the embodiment.

[0024] Figure 4A It is a diagram showing an example of implementation of the image processing system 10 .

[0025] Figure 4B It is a diagram showing a configuration example of the image processing system 10 according to the embodiment when performing scan imaging.

[0026] Figure 5 (A) to (F) in FIG. 1 are diagrams for explaining the relationship between the rotational positions of the scanner device 413 and the turntable 406 during scanning imaging according to the embodiment.

[0027] Figure 6 This is a flowchart corresponding to an example of processing in the image processing system 10 according to the embodiment.

[0028] Figure 7 It is a diagram showing an example of an attribute designation screen according to the embodiment. DETAILED DESCRIPTION

[0029] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In addition, the following embodiments do not limit the invention to which the claims relate, and all combinations of the features described in the embodiments are not limited to those required by the invention. Two or more of the multiple features described in the embodiments may also be arbitrarily combined. In addition, the same reference number is given to the same or identical configurations, and repeated descriptions are omitted. In addition, in each figure, the up-down, down-down, left-right, front-back directions relative to the paper are used as the up-down, down-down, left-right, front-back directions of the parts (or components) in this embodiment, and are used in the description of this article.

[0030] First, the configuration of an image processing system according to the present embodiment will be described. Figure 1 1 is a diagram showing an example of the configuration of an image processing system 10 corresponding to the present embodiment. The image processing system 10 is configured by connecting a scanner 110, a support arm 120, a model support device 130, a display device 140, a headset 150, etc. to an information processing device 100. The system configuration is not limited to Figure 1 In the configuration shown, the information processing device 100 may be further connected to an external server, a cloud server, etc. via a network. In the external server, etc., at least a part of the processing of the embodiments described below can be executed.

[0031] The information processing device 100 controls the operation of at least one of the scanner 110, the support arm 120, and the model support device 130, photographs the object of the photographing object from any angle to generate a plurality of images, and generates three-dimensional model data (true data) based on the plurality of images. In the case where the object of the photographing object is a structure that can be separated into a plurality of constituent objects, the three-dimensional model data can be generated by photographing each constituent object, and the three-dimensional model data of the object object can be generated by integrating them.

[0032] The information processing device 100 can also function as an image generating device that generates images displayed in a virtual space by using the generated three-dimensional model data as data for the virtual space. In this embodiment, the subject of the photographic object is an assembleable plastic model, an action figure (a doll with an active joint), a toy body, a doll, etc., which are collectively referred to as "models" below.

[0033] Next, the scanner 110 is a three-dimensional scanner device that captures (scans) the three-dimensional shape of the model of the photographed object under the control of the information processing device 100 to output the three-dimensional shape and color information of the photographed object. In the present embodiment, the scanning signals output from the scanner 110 are collectively referred to as "scanned images". The scanner 110 can use, for example, Space Spider manufactured by Artec. In the present embodiment, for example, 3D scanning data of the entire toy body can be obtained by obtaining about 500 to 800 frames of scanning images. In addition, as the scanner 110, for example, a smartphone with a camera installed with an application (Application) for capturing three-dimensional shapes can also be used.

[0034] The support arm 120 is a position and posture control device that moves the scanner 110 to a specified photographing position and posture according to the control of the information processing device 100. The support arm 120 can be configured to manually change the photographing position or posture and to fixedly maintain the changed position and posture, or it can be configured to be controlled by the information processing device 100. In the case where the support arm 120 is configured to be controlled by the information processing device 100, for example, xArm7 manufactured by UFACTORY can be used. xArm7 includes 7 joints and can perform the same movements as a human arm. In addition, the scanner 110 can also be positioned manually instead of the support arm 120. In addition, for photographing positions and postures that cannot be covered by the support arm 120, the scanner 110 can also be manually operated to perform scanning.

[0035] The model support device 130 is a support table (or a setting table for setting the model) for supporting a model with a fixed posture. The model support device 130 can also be configured to be rotatable when the model is set on the support table or at the front end of the support rod. In this embodiment, after the scanner 110 is located at any photographic position and photographic angle by the support arm 120, the model support device 130 is rotated to perform photography. The model support device 130 can rotate in both clockwise and counterclockwise directions. In addition, the model support device 130 can be stopped at any rotation angle within the range of 0 degrees to 360 degrees, and the model support device 130 can be stopped at a specific rotation angle so that it is located at any photographic position and photographic angle within the active range of the scanner 110 for photography. By performing the operation at multiple rotation angles, photographic positions, and photographic angles, an image of the entire model can be obtained. In addition, by driving the support arm 120 and the model support device 130 synchronously, photography processing can be performed more simply and with high precision. In addition, the scanner 110 may be manually moved around the model to perform scanning from any arbitrary imaging position and imaging angle instead of the model supporting device 130 .

[0036] The display device 140 is a display device such as a liquid crystal display (LCD) and can display the processing results in the information processing device 100. Specifically, it can display the image obtained by the scanner 110, the real data of the three-dimensional model data generated based on the photographic image, or the image reconstructed using the real data (VR (Virtual Reality) image). In addition, the progress during the scanning process can be displayed on a meter. The display device 140 can have an operation unit 140A that accepts operations from the observer of the displayed image, that is, the user, and the user can operate the operation unit 140A to perform operation inputs corresponding to the content of the image displayed by the display device 140.

[0037] The headset 150 is composed of a head-mounted display 150A and a controller 150B described later. In particular, as a VR headset, it can also be configured to provide dynamic images corresponding to the posture or inclination of the user as an observer. The headset 150 is installed with a specified application, and can download and execute application data executed in the application from the information processing device 100. In this embodiment, the headset 150, like the information processing device 100, can function as an image generating device, and the image generating device uses three-dimensional model data as data for virtual space, generates images displayed in the virtual space, and provides the generated images as VR images. In addition, the application data includes display data for displaying VR images.

[0038] Next, the hardware configuration of the information processing device 100 according to the present embodiment will be described. Figure 2 (A) in the figure is an example of the hardware structure of the information processing device 100. The CPU (Central Processing Unit) 101 is a device that performs overall control of the information processing device 100 or calculation, processing, and management of data. For example, it is possible to control the imaging timing and the number of images in the scanner 110, and control the joints of the arm of the support arm 120 to configure the scanner 110 at any imaging position and imaging angle. For example, after determining the imaging position and imaging angle of the scanner 110, the model support device 130 can be rotated, and the scanner 110 performs the imaging action. In addition, after the model support device 130 is stopped at a specified rotation angle, the imaging action can be performed by the scanner 110 at any imaging position and imaging angle.

[0039] Furthermore, the CPU 101 can also function as an image processing unit that processes an image output from the scanner 110. Specifically, the CPU 101 can generate an image that displays a three-dimensional model in the virtual space using three-dimensional model data generated based on a scan signal obtained by the scanner 110 as virtual space data.

[0040] RAM (Random Access Memory) 102 is a volatile memory used as a temporary storage area such as a main memory and a work area of ​​CPU 101. ROM (Read Only Memory) 103 is a non-volatile memory, and image data or other data, various programs for the operation of CPU 101, etc. are stored in predetermined areas. CPU 101 uses RAM 102 as a work memory, for example, to control various parts of information processing device 100 according to the program stored in ROM 103. In addition, the program for the operation of CPU 101 is not limited to being stored in ROM 103, but may also be stored in storage device 104.

[0041] The storage device 104 is composed of a disk such as a HDD (Hard Disc Drive) or a flash memory. The storage device 104 stores application programs, OS (Operating System), control programs, related programs, game programs, etc. The storage device 104 can read or write data based on the control of the CPU 101. The storage device 104 can be used instead of the RAM 102 or the ROM 103.

[0042] The communication device 105 is a communication interface for communicating with the scanner 110, the support arm 120, the model support device 130, the display device 140, and the headset 150 based on the control of the CPU 101. The communication device 105 can also be configured to be able to communicate with an external server, etc. The communication device 105 can include a wireless communication module, and the module can include a well-known circuit mechanism, and the well-known circuit mechanism includes an antenna system (Antenna System), an RF (Radio Frequency: Radio Frequency) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identification module card, a memory, etc. Here, the communication between the information processing device 100 and the scanner 110, the support arm 120, the model support device 130, the display device 140, and the headset 150 can also be performed through wireless communication.

[0043] In addition, the communication device 105 can also include a wired communication module for wired connection. The wired communication module can communicate with other devices including the display device 140 via one or more external ports. In addition, various software components for processing data can be included. The external port is directly connected to other devices via Ethernet, USB (Universal Serial Bus) or IEEE1394, or indirectly via a network. In addition, it can also be constituted as a substitute for the hardware device by software that realizes the same functions as the above devices.

[0044] The operation unit 106 is composed of, for example, a button, a keyboard, a touch panel, a controller, etc., and receives operation input from the user. The operation unit 106 may be shared with the operation unit 140A or may be independent of the operation unit 140A. For example, if the operation unit 140A is an operation unit configured as a keyboard and a mouse, it can be shared with the operation unit 106. On the other hand, if the operation unit 140A is an operation unit configured as a touch panel or a controller, it can be set as an operation unit different from the operation unit 106.

[0045] The display control unit 107 functions as a control unit for displaying information on the display device 140 connected to the information processing device 100, and controls the operation of the display device 140. Some functions of the operation unit 106 may also be provided in the display device 140. For example, the display device 140 may be configured as a device having a touch panel such as a tablet terminal.

[0046] Next, the hardware configuration of the headphones 150 according to the present embodiment will be described. Figure 2 (B) in the figure represents an example of the hardware configuration of the headset 150. The headset 150 is composed of a head-mounted display (HMD: Head-Mounted Display) 150A and a controller 150B. The HMD 150A provides right-eye images and left-eye images to the left and right eyes of the user, respectively, so that the user can feel a three-dimensional sense through their parallax, thereby experiencing virtual reality (VR). The controller 150B is provided in a housing independent of the HMD 150A. Although the controller 150B is assumed to be a pair of two controllers so that they can be held and operated by the left and right hands of the user respectively, it can also be a single controller.

[0047] The CPU 151 is a device that performs overall control of the headset 150 or calculation, processing, and management of data. For example, it is possible to execute application data downloaded from the information processing device 100 and display a VR image on the display 157. In addition, based on an operation received via the controller 156 or information detected by the detection unit 158, it is possible to switch the VR image itself to be displayed, switch the viewpoint in the VR space, or change the position in the VR space.

[0048] The RAM 152 is a volatile memory used as a temporary storage area such as a main memory and a work area of ​​the CPU 151. The ROM 153 is a non-volatile memory, and image data or other data, various programs for the operation of the CPU 151, etc. are stored in predetermined areas. The CPU 151 uses the RAM 152 as a work memory to control various parts of the headset 150 according to the program stored in the ROM 153, for example. In addition, the program for the operation of the CPU 151 is not limited to being stored in the ROM 153, but may also be stored in the storage device 154.

[0049] The storage device 154 is composed of a magnetic disk such as an HDD or a flash memory. The storage device 154 stores application programs, OS, control programs, related programs, game programs, application data downloaded from the information processing device 100, display data, etc. The storage device 154 can read or write data based on the control of the CPU 151. The storage device 154 can be used instead of the RAM 152 or the ROM 153.

[0050] The communication device 155 is a communication interface for communicating with the information processing device 100 or the controller 150B based on the control of the CPU 151. The communication device 155 includes a wireless communication module capable of wireless communication based on Bluetooth or WiFi (IEEE802.11). The headset 150 is connected to the information processing device 100 through wireless communication and can download data for displaying VR images. In addition, it communicates with the controller 150B and receives information on the user's operation instructions to the controller 150B.

[0051] The display 156 is configured to provide the right eye image and the left eye image generated by the CPU 151 to the right eye and the left eye of the user, respectively. The detection unit 157 is a mechanism for detecting the sight lines of the left and right eyes of the user and detecting the inclination of the head mounted display. The detection unit 157 includes, for example, a sensor for detecting the sight line direction or the gaze direction of the user wearing the HMD 150A. In addition, it includes a gyroscope, a magnetometer, an accelerometer, a global positioning system (GPS), a compass, etc., and can specify the position, posture, inclination, etc. of the HMD 150A according to these detection information. The detection unit 157 detects information for determining the gaze direction or activity of the user in the HMD 150A and sends it to the CPU 151. The CPU 151 determines the gaze direction or activity based on the received detection information, and adjusts the image presented by the display 156 of the HMD 150A in a manner consistent with the determined gaze direction or activity.

[0052] The controller 150B is composed of, for example, a plurality of buttons, a cross key, etc., and receives operation inputs such as selection operations and direction instruction operations from the user. The controller 150B is wirelessly connected to the HMD 150A via the communication device 155 .

[0053] Next, refer to Figure 3 , an example of a model of a subject to be photographed in this embodiment and its accessories is described. Model 300 is a model with a doll-like (robot, or human) appearance. The model can be assembled and painted as a plastic model, for example. Alternatively, it can be a completed model such as a doll (an action figure) with an active joint. Figure 2 The model is only an example for explanation, and the shape of the model is not limited to the appearance of a doll, but can also be a model of any shape such as a general vehicle, a racing vehicle, a military vehicle, an airplane, a ship, an animal, a hypothetical life form, etc. In addition, the object to be photographed can be any object whose three-dimensional shape can be photographed by the scanner 110, and is of course not limited to a model.

[0054] The model 300 includes parts of a head 301, a chest 302, a right wrist 303, a left wrist 304, a right torso 305, a left torso 306, a right leg 307, a left leg 308, a right foot 309, and a left foot 310, and these parts are combined to form a structure. At least a part of each part 301 to 310 is supported so as to be able to rotate (or swing) relative to an adjacent part. For example, the head 301 is supported so as to be able to rotate relative to the chest 302, and the right wrist 303 and the left wrist 304 are supported so as to be able to rotate relative to the chest 302. In this way, since joint structures are provided at each part of the model 300, the model 300 can take any posture.

[0055] In addition, the model 300 is provided with an accessory 311. The accessory 311 is an accessory part that is supposed to be used together with the model 300. Figure 3 The weapon in the figure is a rifle-shaped weapon. That is, it is assumed that the model 300 uses the accessory 311 by holding it with both hands. The shape of the weapon is not limited to the shape of a rifle, and can include weapons of any shape such as a sword (knife) shape, a shield (shield) shape, a bomb shape (such as a grenade), a bazooka shape, etc. In addition, the accessories can be any accessory parts that can be used with the model 300, and are not limited to weapons, but can also be decorations. For example, in the case where the model 300 is a puppet-type model of a specific character, it can also be various props carried by the character (such as luggage, hats, bags, accessories, smartphones, and other items that the character can carry).

[0056] Next, refer to Figure 4A , an overall installation example of the image processing system 10 in this embodiment is described. Figure 4AThis is a diagram that generalizes an implementation example that can generate real data generated by photographing a model.

[0057] exist Figure 4A In the embodiment, the information processing device 100, a driving system for driving the support arm 402, a driving system for driving the turntable 406, etc. are contained in the housing 401. In addition, the support arm 402 can manually adjust the shooting direction and position. The surface of the housing 401 is a flat structure so that a poster for promotion can be attached.

[0058] The support arm 402 corresponds to the support arm 120 and can support and fix the position of the terminal 403 functioning as the scanner 110 according to the control of the information processing device 100 or manually. In addition, the support arm 402 can also operate to control the inclination of the terminal 403.

[0059] The terminal 403 is a touch panel terminal with a built-in camera that can be used as the scanner 110, and can be, for example, a smartphone, a tablet terminal, a digital camera, etc. Instead of these terminals, Space Spider manufactured by Artec Corporation can be used. Figure 4A The above diagrams are diagrams showing examples of generalized system configurations, and can be configured to correspond to the type of device used as the scanner 110. The terminal 403 can capture an image of the model 300 and transmit it to the information processing device 100. The ring light 404 is a lighting device used when the terminal 403 captures the image of the model 300, and can evenly illuminate the model 300 without causing light and shadow. In addition, as an additional light source, in addition to the ring light 404, auxiliary lights can be provided on the top light or on the left, right, or bottom. The three-dimensional scanner device can be used instead of the terminal 403.

[0060] The background plate 405 is a background plate for photography, and a white plate can be used, for example. The turntable 406 can carry the model 300 and rotate the model. The structure including the turntable 406 and its driving system corresponds to the model support device 130, and the turntable 406 is rotated clockwise and counterclockwise and stopped according to the control of the information processing device 100. A plurality of predetermined marks 410 can also be arranged on the turntable 406. The marks 410 can be used to adjust the direction or position of the model to be photographed.

[0061] exist Figure 4A In the present invention, the model is placed on a semi-transparent (transparent) table, but other supporting devices such as a "movable base" (registered trademark) may also be used. The movable base is provided on the table with a structure that is curved. The model can be mounted on the front end of the support in the shape of a letter "U". In this case, the mark 410 can also be arranged on the front end of the support. The method of indicating the model can be changed according to the posture. For example, in the case of an upright posture, it can be set on a transparent table and photographed. On the other hand, in the case of wanting to photograph the soles of the feet like a flying posture, a movable base can also be used. In addition, a movable base can also be used to photograph an upright posture.

[0062] The display device 407 is a device corresponding to the display device 140, and may also have a touch panel function. The user can use the touch panel function to perform a predetermined selection action. In addition, the operation can also be accepted by a controller or mouse not shown in the figure that can function as the operation unit 140A instead of the touch panel. The VR headset 408 includes an HMD 408A and a controller 408B, which correspond to HMDs 150A and 150B, respectively. The user wears the HMD 408A on the head and holds the controller 408B with the left and right hands respectively to operate, and can operate while watching the VR image.

[0063] Next, refer to Figure 4B , a more specific configuration of the image processing system 10 corresponding to the present embodiment in the case of simultaneously scanning the model 300 and the accessory 311 is described. Figure 4B In the embodiment, the model 300 and the accessory 311 are arranged on the turntable 406. The model 300 is supported by the movable base 411 and is arranged substantially in the center of the turntable 406. The accessory 311 is supported by the support member 412A in the frame 412 and is adjacent to the model 300 and arranged near the end of the turntable 406. The indicator member 412A can be set as a clamp-shaped member, for example. Thus, the accessory 411 can be clamped by the clamp and supported in the frame 412.

[0064] The scanner device 413 is supported by the support arm 402, and it is assumed here that the Space Spider manufactured by Artec Corporation is used. The drive system for driving the support arm 402, the drive system for driving the turntable 406, etc. are similar to Figure 4A In addition, in the present embodiment, the turntable 406 can rotate in both clockwise and counterclockwise directions.

[0065] In this embodiment, Figure 4B In the illustrated embodiment, the model 300 and the accessory 311 are collectively placed on a turntable 406, and are collectively photographed by a scanner device 413 to generate a scanned image. Although the scanned image includes both the model 300 and the accessory 311, the three-dimensional model data is generated by separating them.

[0066] In addition, the frame 412 supporting the accessory 311 is configured as a frame body surrounding the accessory 311, and is used as an indicator for identifying the range in which the accessory 311 is arranged. That is, the three-dimensional object existing inside the frame 412 is recognized as the accessory 311. In order not to block the line of sight of the scanner device 413 as much as possible, it is desirable that the thickness of the frame 412 is as thin as possible within the range of maintaining strength. For example, the thickness can be set to about 3 mm.

[0067] In addition, the model 300 and the frame 412 are arranged to be spaced apart to ensure a constant distance. Depending on the rotation angle of the turntable 406, the frame 412 is located near the front side of the model 300 relative to the scanner device 413, and the imaging of the model 300 may be blocked. In this way, the blocked part needs to be supplemented at another rotation angle, but if the model 300 and the frame 412 are too close, the supplement accuracy decreases, so a constant distance is ensured to ensure the supplement accuracy.

[0068] At this time, in order to ensure a constant distance between the model 300 and the frame 412 (or to prevent them from being too close), for example, the sizes of the model 300 and the accessory 311 can be limited to sizes below the preset sizes. For example, the size of the model 300 can be set to be 15 cm in length, 15 cm in width, and 10 cm in depth, and the size of the accessory 311 can be set to be 15 cm in length, 5 cm in width, and 2 cm in thickness.

[0069] In addition, in the present embodiment, the thickness of the installation platform 414 is thicker than that of the installation platform 415 on which the movable base 411 supporting the model 300 is installed. In other words, the model 300 is installed so as to be located higher than the accessories 311. As an example, at least the upper body or upper part of the model 300 can be installed so as to be located higher than the frame 412. As a result, when scanning the model 300, the proportion of being blocked by the accessories 311 is reduced, and the success rate of scanning can be improved.

[0070] Next, refer to Figure 5 , the cooperative action of the turntable 406 and the support arm 120 during scanning corresponding to this embodiment is described. Figure 5 4 is a diagram showing the relationship between the rotation direction and amount of the turntable 406 and the installation tables 414 and 415 with respect to the scanner device 413. Figure 5 In the figure, the model 300 placed on the installation table 414 is arranged so that the direction written as the front is the front. The accessories 311 placed on the installation table 415 can also be set in the same way.

[0071] Figure 5(A) in FIG. 4 shows the relationship between the scanner device 413 and the installation platforms 414 and 415 at the start position of the scanning operation. The rotation angle of the turntable 406 at this point is set as 0 degrees as the reference. When the turntable 406 rotates 90 degrees to the right (clockwise) from the above state, it becomes Figure 5 The state shown in (B) in the figure. Figure 5 (A) in the transformation to Figure 5 During (B) in FIG. 4 , the scanner device 413 is located on the side of the turntable 406 and scans the model 300 and the accessory 311. Then, Figure 5 In the state shown in (B), when the front faces of the model 300 and the accessory 311 are located on the front side of the scanner device 413, the support arm 402 is moved within the limits of the active area to scan the front face, left and right sides, flat surface, and bottom surface of the model 300 and the accessory 311.

[0072] Next, from Figure 5 In the state (B) in the figure, the turntable 406 is rotated rightward from the position of 90 degrees to the position of 270 degrees. During this period, the model 300 and the accessory 311 are also scanned. Then, it becomes Figure 5 In the state shown in (C), when the back of the model 300 and the accessory 311 are located on the front side of the scanner device 413, the support arm 402 is moved within the limit of the active area to scan the back, left and right sides, flat surface, and bottom surface of the model 300 and the accessory 311.

[0073] Next, from Figure 5 In the state (C) in the figure, the turntable 406 is further rotated to the right from the position of 270 degrees to the position of 330 degrees. During this period, the model 300 and the accessory 311 are also scanned. Then, it becomes Figure 5 In the state shown in (D) in FIG. 1 , the support arm 402 is moved within the limit of the active area from the right oblique rear of the model 300 and the accessory 311 to scan the model 300 and the accessory 311. By scanning as described above, the model 300 can be scanned in a manner that complements the portion of the model 300 that is blocked by the frame 412 or the accessory 311 (the surface of the model 300 on which the frame 412 or the accessory 311 is disposed). In addition, the scanning from the right oblique rear is described here, but the scanning from the right oblique front can also be performed.

[0074] Next, from Figure 5 In the state (D) in the figure, the turntable 406 is rotated to the left (counterclockwise) from the position of 330 degrees to the position of 30 degrees. During this period, the scanner device 413 is configured to scan from the oblique upper side of the model 300 and the accessory 311 as a bird's-eye view. Then, Figure 5 Then, from Figure 5 In the state (E) in the figure, the turntable 406 is rotated rightward (clockwise) from the position of 30 degrees to the position of 360 degrees. During this period, the scanner device 413 is configured to scan from the oblique lower side of the model 300 and the accessory 311 as an upward viewpoint. Then, Figure 5 The state shown in iF).

[0075] In addition, when "fighting / boxing" is selected in the setting of the attribute 720 of the accessory described later, the accessory 311 is set without including the frame 412. In this case, since the accessory 311 is not set, it is also possible to omit the frame 412 from the above. Figure 5 (E) returns Figure 5 The turntable can be rotated by the method (F) in the figure. Figure 5 In (E), the scanning is terminated halfway instead of rotating from 330 degrees to 30 degrees. For example, the rotation may be terminated at 360 degrees or at an angle less than 30 degrees (e.g., 5 degrees, 10 degrees, etc.). In this way, the first scanning range when the accessory 311 is provided is different from the second scanning range when the accessory 311 is not provided, and the first scanning range is larger (or wider) than the second scanning range. In addition, when the frame 412 remains, it is not necessary to omit the frame 412 from the above. Figure 5 (E) returns Figure 5 The operation of rotating the turntable is performed in the manner (F).

[0076] Assume that the scanner device 413 used in the present embodiment performs scanning at a position approximately in the middle (e.g., the distance between the scanner device 413 and the model 300 is about 130 mm) relative to the scannable range of the scanner device 413 (e.g., the distance between the scanner device 413 and the model 300 is about 130 mm). In this case, if the accessory 311 passes between the scanner device 413 and the model 300, there is a concern that the accessory 311 will block the model 300 and the scanned object will be lost. However, as described above, by rotating the turntable 406 in the clockwise direction and also rotating it in the counterclockwise direction and performing scanning, and by performing scanning in a manner that supplements the blocked range, the three-dimensional model data of each of the model 300 and the accessory 311 can be reliably acquired without losing the scanned object.

[0077] Next, refer to Figure 6 An example of processing performed by the image processing system 10 according to the present embodiment is described below. At least a portion of the processing corresponding to the flowchart is implemented by the CPU 101 of the information processing device 100 executing a program stored in the ROM 103 or the storage device 104 .

[0078] First, in S601 , CPU 101 accepts user login. It accepts input of a user's name or contact information. It assigns a user identifier for uniquely identifying each user to each user. CPU 101 associates the input user information with the time of accepting the input or the user identifier in advance and stores them in storage device 104 .

[0079] When the user login is completed, the user installs his own model on the model support device 130. CPU101 can determine whether the model 300 and the accessories 311 are installed on the model support device 130 based on the scanned image of the scanner 110. Alternatively, a switch that becomes turned on when the model 300 and the accessories 311 are installed on the model support device 130 can be pre-configured, and CPU101 detects and determines the signal from the switch. Alternatively, a button for accepting an operation when the installation of the model 300 and the accessories 311 is completed can be pre-displayed on the display device 140, and CPU101 detects whether an operation for the button operation is accepted. In S602, CPU101 detects whether the model 300 and the accessories 311 have been installed on the model support device 130 by any of the above methods. Based on the detection, the processing enters S603.

[0080] In the following Figure 7 In the attribute 720 of the accessory, since there is a case where the attack performance is performed without using the accessory 311, the process may proceed to S603 if the attachment of the accessory 311 is not detected after a certain time has passed after at least the model 300 is attached. Alternatively, it may be detected whether the accessory 311 is attached only when the frame 412 is attached.

[0081] In S603, a process of accepting the specification of the attributes of the model 300 and the accessory 311 is executed. Figure 7 The screen 700 shown receives selections of the attributes (types) of the model to be scanned next, the attributes (types) of the accessories, and the colors (rendering colors) used for rendering the subject and accessories.

[0082] Can Figure 7 The screen 700 is displayed on the display device 140, for example, and accepts operation input from the user. The user can select either realism or deformation for the model attribute 710 in the display of the screen 700. Because the appearance characteristics of the model are different in realism and deformation, the scanning method can also be selected in accordance with the selected model attribute. For example, in the case of deformation, the range in which the scanner device 413 operates can be set narrower than in the case of realism. In addition, in the model attribute 710, the scale value of the model (1 / 144, 1 / 100, 1 / 60) and the like can also be specified. In this way, the operating range of the scanner device 413 can be further set in accordance with the scale of the model.

[0083] Next, regarding the attribute 720 of the accessory, here assuming that the accessory is a weapon, one can select from "fighting / boxing" that uses a body part such as a fist as a weapon, "rifle" as a shooting weapon, "machine gun", "bazooka", and "melee weapon" such as a sword. In addition, as the attribute of the weapon, Figure 7 The above is only an example, and more detailed attributes can be specified. The attribute 720 of the accessory in this embodiment is equivalent to the specification of the type of performance using the accessory, and the specification related to the presence or absence of the accessory. In other words, as a performance method, it is possible to specify any one of a performance using shooting weapons such as "rifles", "machine guns", and "bazookas", a performance using "melee weapons" such as swords, or a performance using boxing without using weapons. In addition, if "fighting / boxing" is specified, it means that there are no accessories, and if an attribute other than this is specified, it means that there are corresponding accessories. In this embodiment, different corresponding performances are performed according to these specified attributes.

[0084] When using the body parts, in addition to punching, kicking, rotating the wrist or foot, and attacking using any other body parts can be performed, and these can also be selected as weapon attributes. In addition, when "fighting / boxing" is specified as the attribute 720 of the accessory, the scanning of the accessory 311 described later can be omitted, and the scanning of the accessory 311 can be performed in the same way as when a shooting weapon or a close combat weapon is specified. In addition, although Figure 7 Only attacking weapons are shown, but defensive weapons, shields, and other attributes can also be specified. For the performance color 730, any one of red, yellow, yellow-green, blue, and pink can be selected. Figure 7 The color shown here is just an example, and colors other than those shown here can be specified.

[0085] In addition, when the attachment 311 is not detected in S602, only "Fighting / Boxing" may be selected. Alternatively, when the attachment 311 is detected in S602, "Fighting / Boxing" may not be selected or may be selected.

[0086] The scan start button 740 is displayed on the screen 700. When the attributes of 710 to 730 are specified, the user can instruct the start of scanning. When the start of scanning is instructed, the process moves to S604. Figure 7The attribute information input on the screen is associated with the user information and stored in the storage device 104. When the scan start button 740 is operated, a screen showing the progress of the scan is displayed on the display device 140. The user can understand the progress of the scan by referring to the screen. The screen includes a message (scanning, post-scanning processing, etc., a message notifying the current status), a meter showing the progress (a pie chart or a bar chart showing the progress as a percentage), etc. In addition, a scan button for stopping the scan may also be included.

[0087] In S604, as shown in reference Figure 5 As described above, the scanner 110, the support arm 120, and the model support device 130 cooperate to perform scanning processing (photographic processing) to generate a scanned image. Specifically, the CPU 101 controls the support arm 120 to move the scanner 110 to any registered photographing position, and rotates the model support device 130 at the photographing position while the scanner 110 performs scanning. In addition, after the model support device 130 stops at a predetermined rotation angle, the scanner 110 can perform a photographing action at any photographing position and photographing angle. In S604, the scanning action can be switched according to the type of attribute accepted by the attribute 720 of the accessory in S603. That is, when an attribute other than "fighting / boxing" is selected, since the accessory 311 is set, the first scanning action corresponding to the above situation is performed. On the other hand, when "fighting / boxing" is selected, since the accessory 311 is not set, the second scanning action corresponding to the above situation is performed. The first scanning action is performed in the first scanning range, and the second scanning action is performed in the second scanning range. The captured scanned image is transmitted to the information processing device 100 , and the CPU 101 associates the scanned image with a user identifier and the like and stores the image in a table in the storage device 104 .

[0088] In the next S605, the CPU 101 performs post-scanning processing on the scanned image obtained by the scanning process, generates the three-dimensional model data of the model 300 and the accessory 311, and generates application data (display data) for image display. Here, when "fighting / boxing" is specified as the attribute 720 of the accessory, the three-dimensional model data of the accessory 311 is not generated, and when "rifle", "machine gun", "bazooka" and "melee weapon" are specified, the three-dimensional model data of the accessory 311 is generated. In addition, in the generation of the three-dimensional model data, when the three-dimensional model data of the accessory 311 is generated, the three-dimensional model data of the accessory 311 is associated with the position of the hand of the three-dimensional model data of the model 300. On the other hand, when the three-dimensional model data of the accessory 311 is not generated, such an association is not established.

[0089] Specifically, for rifles, machine guns, and bazookas, the handle portion is associated in a manner that reaches the position of the hand of the model 300. At this time, the angle at which the model 300 holds the shooting weapon is also set. In addition, for melee weapons such as sideknife or knife, the handle portion is associated in a manner that reaches the position of the hand of the model 300, and the angle at which the model 300 holds the melee weapon is also set. In this way, three-dimensional model data in a state where the model 300 holds the accessory 311 can be generated. When "fighting / boxing" is specified as the attribute 720 of the accessory, the above-mentioned association process is omitted.

[0090] In the post-scanning process, a region defining the range of useless data is created in advance, and the faces included in the region where data reading is not required are deleted. In this way, faces other than the model 300 and the accessory 311 can be roughly removed. The useless data range includes the region where the movable base 411, the setting table 414, the frame 412, the support member 412A, the setting table 415, etc. are located.

[0091] In the information processing device 100, when the generated application data is executed in a specific application, an image can be reproduced. In addition, in the head mounted display 150A, when the specific application is executed, a VR image can be reproduced. The generated three-dimensional model data and display data are associated with the user information together with the attribute information received in S603 and stored in the storage device 104.

[0092] In the next S606, the image display process is performed. When the image display process is performed in the information processing device 100, the CPU 101 executes the display data in the corresponding specific application, and displays the image on the display device 140 via the display control unit 107. In addition, in S606, the CPU 101 controls the communication device 105 to send the display data to the HMD 150A, and the CPU 151 of the HMD 150A executes the received display data in the corresponding application to perform the process of displaying the VR image, thereby performing the image display process.

[0093] When displaying an image, the performance of the subject or accessories can be switched according to the attribute information. Figure 7 When the attribute 720 of the attribute 720 is any one of a machine gun, a rifle, and a bazooka, the main body is moved by the behavior corresponding to each shooting weapon to perform a performance display of a projectile fired from the shooting weapon. At this time, different performances can be performed according to each attribute information such as the size of the projectile in the performance display of the machine gun, rifle, and bazooka, the number of projectiles fired per unit time, and the speed of the projectile. In addition, the display of the projectile at this time is performed in the color specified by the performance color.

[0094] In addition, when the attribute is "melee weapon", for example, the main body is moved by a behavior corresponding to a melee weapon such as a saber or an axe, and a performance display of a slash is performed from the melee weapon according to the swinging of the weapon. The display of the slash at this time is performed in the color specified by the performance color. In addition, when the attribute is "fighting / boxing", the main body is made to perform a boxing action, and according to the boxing action, a performance display of an energy bullet or an attack wave equivalent to the slash in the melee weapon is performed from the fist. In addition, the same is true for the case where a kick can be specified, and a performance display of an energy bullet or an attack wave can be performed from the fist according to the kicking action. The display of the attack at this time is performed in the color specified by the performance color.

[0095] exist Figure 6 In the description, the case where the specification of the attribute information is accepted before the start of the scanning process is described, but the embodiment is not limited to this, and the specification of the attribute information may be accepted after the scanning process of S604 or the post-scanning process of S605.

[0096] In this embodiment, it is possible to set a Figure 6 For example, an LED (Light Emitting Diode) strip may be installed in advance on at least one of the turntable 406, the movable base 411, the setting table 414, the setting table 415, etc., so that the LED strip is connected to Figure 6 The LED strip may be notified by emitting light in different colors corresponding to the processing steps in the processing. For example, during the processing in S601 and S602, the LED strip may be emitted in the first emission color (e.g., red), and during the period of accepting input from the user in S603, the LED strip may be emitted in the second emission color (e.g., blue). Thereafter, when the user operates the scanning start button 740, the LED strip may be emitted in the third emission color (e.g., white). The LED strip may continue to emit light during the processing, or may emit light for a constant time (e.g., 5 seconds or 10 seconds) and then turn off.

[0097] As described above, in this embodiment, when the three-dimensional model data generated from the image obtained by photographing the model with a scanner is configured in a virtual space to generate display data for displaying the image, the three-dimensional model data can be generated by photographing its accessories simultaneously with the model, and used in the virtual space together with the three-dimensional model data of the model.

[0098] In addition, in this embodiment, since it is possible to accept the designation of attribute information for the type or presence of an accessory, when using the three-dimensional model of the accessory in the virtual space, a performance corresponding to the accepted attribute can be performed. As a result, since the user can reflect his or her own image on the performance related to the accessory, there is no sense of disharmony. In addition, since it is also possible to accept the designation as attribute information for the type of model, it is possible to obtain a more accurate scanned image by controlling the scanning range according to the type of model.

[0099] In the above embodiment, although weapons are assumed to be accessories for explanation, the types of accessories are not limited to weapons. For example, in the case where the model is a specific humanoid character, the accessories may also be clothing, accessories, etc. worn by the character. In addition, the model can also appear in events, concerts, sports, online meetings, etc. that take place in a virtual space. In addition, the technology of this embodiment can also be applied to the technology of merging the real world with the virtual world, such as Cross Reality (XR), to perceive images that do not exist in the real space.

[0100] Thus, in this embodiment, three-dimensional model data can be generated from an image of the appearance of a model and its accessories, and an image of the three-dimensional model arranged in a virtual space can be enjoyed. Models and accessories, such as assembled plastic models, can be processed as individual works by users through painting, etc., and their personalities can be reflected in dynamic images or character expressions in a virtual space, which can significantly increase the fun.

[0101] <Summary of Implementation Methods>

[0102] The above-described embodiments disclose at least the following information processing devices and computer programs.

[0103] (1) An information processing device comprising:

[0104] A scanning control mechanism controls the position and angle of the scanner to scan the appearance of the object set on the setting table and generate a scanned image;

[0105] A data generating mechanism for generating three-dimensional model data according to the scanned image;

[0106] An image generating unit generates an image in which a three-dimensional model based on the three-dimensional model data is arranged in a virtual space; and

[0107] An accepting unit accepts input of attribute information related to the subject to be scanned and the performance of the subject; and

[0108] The image generation unit generates the image in a manner that includes a performance of each of the subjects corresponding to the attribute information.

[0109] (2) The information processing device according to (1), wherein the attribute information includes first attribute information indicating the type of the subject and second attribute information related to the performance in the image generation mechanism; and

[0110] The image generation unit generates the image in a manner that includes a performance of each of the subjects corresponding to the first attribute information and the second attribute information.

[0111] (3) The information processing device according to (2), wherein the second attribute information includes information indicating a type of the performance.

[0112] (4) The information processing device according to (3), wherein the type of performance includes a plurality of different types of performance, and the image generation unit includes different performance images for each type.

[0113] (5) The information processing device according to any one of (2) to (4), wherein the subject includes a first subject and a second subject, and the second subject is an accessory of the first subject; and

[0114] The first attribute information indicates a type of the first object;

[0115] The second attribute information includes a type of the second subject used for the performance.

[0116] (6) The information processing device according to (5), wherein the second attribute information indicates the presence of the second subject by indicating the type of the accessory; and

[0117] The second attribute information indicates that the second subject does not exist by indicating that a part of the first subject is used for the performance.

[0118] (7) An information processing device according to (5) or (6), wherein the data generating unit performs processing to associate the second subject with a specified position of the first subject based on a first scanned image of the first subject and a second scanned image of the second subject, and generates the three-dimensional model data in a manner in which the first subject holds the second subject.

[0119] (8) The information processing device according to (7), wherein the second attribute information further indicates the presence or absence of the second subject;

[0120] When the second attribute information indicates that the second object does not exist, the scanned image of the second object is not generated; and

[0121] The data generating unit generates the three-dimensional model data based on the first scan image without performing a process of associating the second object with a predetermined position of the first object.

[0122] (9) The information processing device according to any one of (5) to (8), wherein when the second attribute information indicates the presence of the second subject, the scanned image of the second subject is generated.

[0123] (10) An information processing device according to any one of (5) to (9), wherein the scanning control mechanism causes a first scanning range when the second object is set on the setting table to be different from a second scanning range when the second object is not set on the setting table, and the first scanning range is larger than the second scanning range.

[0124] (11) The information processing device according to (5) or (10), further comprising: a detection unit for detecting whether the second object is placed on the setting table; and

[0125] When the second subject is not detected, the accepting unit does not accept the second attribute information indicating the type of the accessory.

[0126] (12) The information processing device according to any one of (5) to (11), wherein the first subject is a doll-type model, and the second subject is an accessory part of the model.

[0127] (13) The information processing device according to any one of (2) to (12), wherein the second attribute information further includes information on a color used for the performance.

[0128] (14) The information processing device according to any one of (1) to (13), further comprising: a notification unit for notifying the operation status of the information processing device; and

[0129] The notification unit notifies the information processing device of at least one of a state of accepting input of the attribute information and a state of performing the scanning.

[0130] (15) A computer program for causing a computer to function as the information processing device described in any one of 1 to 14.

[0131] The present invention is not limited to the above-described embodiments, and various changes and modifications can be made within the scope of the gist of the invention.

Claims

1. An information processing device comprising: A scanning control mechanism controls the position and angle of the scanner to scan the appearance of the object set on the setting table and generate a scanned image; A data generating mechanism for generating three-dimensional model data according to the scanned image; An image generating unit generates an image in which a three-dimensional model based on the three-dimensional model data is arranged in a virtual space; and An accepting unit accepts input of attribute information related to the subject to be scanned and the performance of the subject; and The image generation unit generates the image in a manner that includes a performance of each of the subjects corresponding to the attribute information.

2. The information processing device according to claim 1, wherein the attribute information includes first attribute information indicating the type of the subject, and second attribute information related to the performance in the image generation mechanism; and The image generation unit generates the image in a manner that includes a performance of each of the subjects corresponding to the first attribute information and the second attribute information. 3 . The information processing device according to claim 2 , wherein the second attribute information includes information indicating a type of the performance.

4. The information processing device according to claim 3, wherein the types of performances include a plurality of different types of performances, and the image generating means includes different performance images for each type.

5. The information processing device according to claim 4, wherein the subject comprises a first subject and a second subject, the second subject being an accessory of the first subject; and The first attribute information indicates a type of the first object; The second attribute information includes a type of the second subject used for the performance.

6. The information processing device according to claim 5, wherein the second attribute information indicates the presence of the second subject by indicating the type of the accessory; and The second attribute information indicates that the second subject does not exist by indicating that a part of the first subject is used for the performance.

7. An information processing device according to claim 6, wherein the data generating mechanism performs processing to associate the second subject with a specified position of the first subject based on the first scanned image of the first subject and the second scanned image of the second subject, and generates the three-dimensional model data in a manner in which the first subject holds the second subject.

8. The information processing device according to claim 7, wherein the second attribute information further indicates whether the second subject exists; When the second attribute information indicates that the second object does not exist, the scanned image of the second object is not generated; and The data generating unit generates the three-dimensional model data based on the first scan image without performing a process of associating the second object with a predetermined position of the first object. 9 . The information processing device according to claim 8 , wherein when the second attribute information indicates the presence of the second object, the scanned image of the second object is generated.

10. The information processing device according to claim 9, wherein the scanning control mechanism makes a first scanning range when the second object is set on the setting table different from a second scanning range when the second object is not set on the setting table, and the first scanning range is larger than the second scanning range.

11. The information processing device according to claim 10, further comprising: a detection unit for detecting whether the second object is placed on the setting table; and When the second subject is not detected, the accepting unit does not accept the second attribute information indicating the type of the accessory. 12 . The information processing device according to claim 11 , wherein the first object is a doll-type model, and the second object is an accessory part of the model. 13 . The information processing device according to claim 12 , wherein the second attribute information further includes information on a color used for the performance.

14. The information processing device according to claim 13, further comprising: a notification unit for notifying an operation status of the information processing device; and The notification unit notifies the information processing device of at least one of a state of accepting input of the attribute information and a state of performing the scanning.

15. A computer program for causing a computer to function as the information processing device according to any one of claims 1 to 14.

Images