Information processing apparatus, image processing apparatus, computer program, and storage medium

Abstract

The present application provides an information processing device, an image processing device, a computer program, and a storage medium, which acquire information about the appearance of an article with high precision and enable the information to be utilized in a virtual space. An information processing device generates virtual space data from scan information about the appearance of an article acquired by a three-dimensional scanner device, the information processing device accepts an operation instruction for first scan information displayed in a first screen for adjusting the coordinate system of the first scan information obtained by scanning the article in a manner consistent with the coordinate system of frame data of the article, and generates virtual space data from the frame data and second scan information obtained after the operation instruction.

Description

Technical Field

[0001] This invention relates to an information processing device, an image processing device, and a computer program. Background Technology

[0002] Patent Document 1 describes the following technology: In an online game in which robots fight each other in a virtual space operated by two or more terminals connected via a network, information of a plastic model is read in each terminal to generate robot information and the robot information is used.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-087007 Summary of the Invention

[0006] The problem the invention aims to solve

[0007] In Patent Document 1, the information of the plastic model is read in the terminal where the game is implemented, with the aim of improving the accuracy of the reading.

[0008] The purpose of this invention is to acquire information related to the appearance of an object with high precision and to enable that information to be used in virtual space.

[0009] Solution for solving the problem

[0010] This invention relates to an information processing apparatus that generates virtual space data usable in a virtual space based on scanned information of the appearance of an object acquired by a 3D scanner configured to acquire 3D shape and color information of the object. The information processing apparatus includes: a determination unit that determines the category of the object; an acquisition unit that acquires frame data corresponding to the determined category of the object, the frame data being configured to map texture information obtained based on the scanned information; a display control unit that causes a display unit to display a first screen, the first screen being used to adjust a coordinate system set for first scanned information in a manner consistent with a coordinate system set for the frame data, and including the display of the first scanned information generated by scanning the object using the 3D scanner; a receiving unit capable of receiving an instruction, including at least one of movement and rotation, for the first scanned information displayed in the first screen; and a first generation unit that generates the virtual space data including texture information mapped onto the frame data based on the frame data and second scanned information obtained after the display of the first screen ends in response to the instruction.

[0011] The effects of the invention

[0012] According to the present invention, it is possible to acquire information related to the appearance of an object with high precision and make that information available in virtual space. Attached Figure Description

[0013] Figure 1 These are diagrams showing an example of the structure of a game system corresponding to the implementation method and an example of the hardware structure of the information processing device 100.

[0014] Figure 2 This is a diagram showing an example of the appearance of a toy body corresponding to the embodiment.

[0015] Figure 3 This is a diagram showing an example of the appearance of a specific fastener corresponding to the embodiment.

[0016] Figure 4 This is a diagram illustrating an example of the data structure of a data table corresponding to an implementation method.

[0017] Figure 5 This is a flowchart illustrating an example of the processing for generating game data corresponding to the implementation method.

[0018] Figure 6 This is a flowchart illustrating an example of the scanning process corresponding to the implementation method.

[0019] Figure 7 This is a flowchart illustrating an example of game data generation processing corresponding to the implementation method.

[0020] Figure 8 This is a diagram showing an example of an operation screen displayed during game data generation processing, corresponding to the implementation method.

[0021] Figure 9 This is a diagram showing other display examples of the operation screen displayed during game data generation processing, corresponding to the implementation method.

[0022] Figure 10 This is a flowchart illustrating an example of game processing corresponding to an implementation method. Detailed Implementation

[0023] The embodiments will now be described in detail with reference to the accompanying drawings. Furthermore, the embodiments described below do not limit the invention as defined in the claims, and the combinations of features described in the embodiments are not limited to those essential to the invention. Two or more features from the plurality of features described in the embodiments may be combined arbitrarily. Additionally, the same or identical structures are labeled with the same reference numerals, and repeated descriptions are omitted. Furthermore, in each figure, the up, down, left, right, front, and back directions relative to the plane of the paper are used as the up, down, left, right, front, and back directions of the component (or accessory) in this embodiment for use in the description herein.

[0024] First, the structure of the game system corresponding to this implementation method will be explained. Figure 1 Figure (A) is an example of the structure of a game system 10 corresponding to this embodiment. The game system 10 is configured by connecting an information processing device 100 to a scanner 110, a robotic arm 120, a rotary table 130, and a display device 140.

[0025] The information processing device 100 controls the movements of the scanner 110, the robotic arm 120, and the rotary table 130 to scan the object from any angle to generate three-dimensional data of the object. The obtained three-dimensional data is then applied as a texture to frame data corresponding to the toy body, thereby generating game data that can be used in a game. Furthermore, the information processing device 100 can function as a game device that performs game processing using the generated game data, or it can function as an image processing device that generates images displayed in a virtual space by using the game data as virtual space data. Alternatively, a game device and an image processing device can be prepared separately from the information processing device 100. In this embodiment, the object to be scanned is a toy body such as a plastic model or a doll.

[0026] Next, the scanner 110 is a 3D scanner device that, under the control of the information processing device 100, captures (scans) the 3D shape of the toy as the subject of the image and outputs the 3D shape and color information of the subject. In this embodiment, the scanning signals output from the scanner 110 are collectively referred to as "scanned images". The scanner 110 can be, for example, the Space Spider manufactured by Artec. In this embodiment, for example, 3D scan data of the entire toy can be acquired by acquiring scanned images at approximately 500 to 800 frames per second.

[0027] The robotic arm 120 is a position and posture control device that moves the scanner 110 to a predetermined scanning position and posture under the control of the information processing unit 100. The robotic arm 120 can, for example, use the xArm 7 manufactured by UFACTORY (a registered trademark). The xArm 7 consists of seven joints and is capable of performing the same movements as a human arm.

[0028] The rotary table 130 is a rotating device configured to rotate under the control of the information processing device 100 while the toy is mounted. In this embodiment, after the scanner 110 is positioned at any shooting position and angle by the robotic arm 120, the rotary table 130 is rotated one revolution and a scan is performed. By performing this scan at multiple shooting positions and angles, 3D scan data of the entire toy is acquired. Furthermore, it is preferable that the scanning process can be performed more easily and with higher precision by synchronously driving the rotary table 130 and the robotic arm 120.

[0029] The display device 140 is a display device such as a liquid crystal display (LCD) that displays the 3D scan data acquired by the scanner 110, or displays such as Figure 8 , Figure 9 The operation screen shown, or the game screen that uses the generated game data.

[0030] Figure 1 (B) shows an example of the hardware structure of the information processing device 100. The CPU 101 is a device that performs overall control, data calculation, processing, and management of the information processing device 100. Specifically, it can control the timing and number of images captured in the scanner 110, and can control the joint of the robot arm 120 to configure the scanner 110 to any capturing position and angle. In addition, after determining the capturing position and angle of the scanner 110, the rotary table 130 is rotated and the scanning action is performed through the scanner 110. Furthermore, the CPU 101 can also function as an image processing unit that compresses and encodes the digital image signals output from the scanner 110 to generate image data.

[0031] RAM 102 is volatile memory, used as the main memory and temporary storage area for the CPU 101, such as its working area. ROM 103 is non-volatile memory, storing image data, other data, and various programs used by the CPU 101 for operation in designated areas. For example, the CPU 101 uses RAM 102 as its working memory according to the programs stored in ROM 103 to control various parts of the information processing device 100. Furthermore, the programs used by the CPU 101 for operation are not limited to being stored in ROM 103; they can also be stored in storage device 104.

[0032] Storage device 104 may be composed of a disk such as an HDD or flash memory. Storage device 104 stores applications, operating system, control programs, associated programs, game programs, etc. Storage device 104 can read or write data based on the control of CPU 101. Storage device 104 may also be used as a replacement for RAM 102 and ROM 103.

[0033] The communication device 105 is a communication interface used for communication with the scanner 110, robot arm 120, and rotary table 130 based on the control of the CPU 101. The communication device 105 may include a wireless communication module, which may include known circuitry including an antenna system, RF transmitter and receiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a participant identification module card, and memory. Here, communication between the information processing device 100 and the scanner 110, robot arm 120, and rotary table 130 can also be achieved wirelessly.

[0034] Additionally, the communication device 105 may also include a wired communication module for wired connections. The wired communication module enables communication with other devices, including the display device 140, via one or more external ports. Furthermore, it may include various software components for data processing. The external ports can be coupled directly or indirectly to other devices via Ethernet, USB, IEEE 1394, etc. Moreover, it may be a hardware replacement device that implements equivalent functions to the aforementioned devices.

[0035] The operation unit 106 is composed of, for example, buttons, keyboards, touch panels, etc., and accepts operation input from the user. The display control unit 107 functions as an interface for displaying information on the display device 140 connected to the information processing device 100, and controls the operation of the display device 140.

[0036] When the game device is installed independently of the information processing device 100, the structure of the game device can also be configured to be the same as... Figure 1(B) is the same.

[0037] Next, refer to Figure 2 To illustrate a specific example of a toy that is the object of scanning in this embodiment, the toy body 200 is a toy body with a humanoid (robot or human) appearance. This toy body can, for example, be a toy body that can be assembled and painted as a plastic model. Figure 2 The toy body mentioned is merely an example for illustration. The shape of the toy body is not limited to a humanoid appearance; it can be any shape, such as a general vehicle, racing car, military vehicle, airplane, ship, animal, or virtual life form. Furthermore, the item to be scanned is not limited to a toy body as long as it can be scanned by the scanner 110.

[0038] The toy body 200 comprises a head 201, a chest 202, a right arm 203, a left arm 204, a right torso 205, a left torso 206, a right leg 207, a left leg 208, a right foot 209, and a left foot 210, which together constitute the toy body 200. At least a portion of each part 201 to 210 is supported in a manner that allows it to rotate (or swing) relative to adjacent parts. For example, the head 201 is supported in a manner that allows it to rotate relative to the chest 202, and the right arm 203 and left arm 204 are supported in a manner that allows them to rotate relative to the chest 202. Because joint structures are provided in various parts of the toy body 200 in this way, the toy body 200 can assume any posture. On the other hand, in order to scan the toy body 200 using the scanner 110 to obtain three-dimensional shape and color information of the outer surface, a posture suitable for obtaining the three-dimensional shape is required.

[0039] Therefore, using such Figure 3 The fixture 300 shown determines the posture of the toy body 200 during scanning. The fixture 300 has parts corresponding to the head 201, chest 202, right arm 203, left arm 204, right torso 205, left torso 206, right leg 207, left leg 208, right foot 209, and left foot 210 of the toy body 200. The head fixing part 301 is a component for fixing the head 201. The chest fixing part 302 is a component for fixing the chest 202. The right arm fixing part 303 and the left arm fixing part 304 fix the right arm 203 and left arm 204 respectively, so that they extend away from the chest 202. Furthermore, the right torso fixing part 305 and the left torso fixing part 306 fix the position of the right torso 205 and left torso 206 relative to the chest 202. The right leg fixing part 307 and the left leg fixing part 208 are components used to fix the right leg 207 and the left leg 208 to open the stride, respectively. The right foot fixing part 309 and the left foot fixing part 310 are components used to fix the right foot 209 and the left foot 210, respectively.

[0040] The fixture 300 is secured to the toy body 200 by covering its front side. After positioning the toy body 200 in a posture corresponding to the shape of the fixture 300, the toy body is placed on the rotary table 130. At this time, a component for securing the toy is provided on the rotary table 130. After the toy body is attached to this component, the fixture 300 is removed from the toy body 200. Thus, the toy body on the rotary table 130 can be positioned in a prescribed posture suitable for acquiring scanned images.

[0041] exist Figure 3 Only the fixture 300 used on the main body of the toy body 200 is shown in the figure. However, if the toy body 200 has an accessory and the mounting angle and shape can be changed when the accessory is installed, the fixture used to fix the posture of the accessory can also be used.

[0042] Next, refer to Figure 4 This will explain the data structure of the various tables stored in the storage device 104 of the information processing apparatus 100. Figure 4 In Table 400 of (A), associated with the specific model name 401 of the toy, are registered information such as model identification information 402 for determining the model name based on the appearance of the toy body, fixture information 403 for determining the fixture used, camera system control information 404 during scanning and shooting, and frame data (skeleton information for mapping the scanned image as a texture) 405 corresponding to the specific model of the toy.

[0043] First, the model name 401 contains identification information that uniquely identifies the toy entity capable of generating game data in this embodiment. Model identification information 402 is used to determine the model name or category of the toy entity based on its appearance; for example, information related to the shape of specific parts of the toy entity may also be registered. Specifically, information such as the shape, pattern, and number of holes on the soles of the toy entity's feet are registered. Based on this model identification information 402, the image of the soles of the toy entity 200 is processed, thereby identifying the model.

[0044] The fixture information 403 contains information used to identify the fixture installed on the toy, which has been identified as a model registered under model name 401. The fixture information includes the fixture's color, material, shape, and fixture number. The camera system control information 404 is used by the CPU 101 to control the movements of the scanner 110, robotic arm 120, and rotary table 130 according to the model during scanning and imaging.

[0045] Frame data 405 is template data used to map textures generated from scanner images acquired using scanner 110. The frame data contains vertex data for each vertex of the model (vertex position coordinates, texture coordinates, color data, normal vectors, or α values, etc.) and information about movable joints. The frame data has its own coordinate system. The frame data is (except for the unmapped textures) data that can be used in the game, has an appearance corresponding to various toy bodies, and can move in the game just like a real toy body. By mapping texture data obtained from a physical toy body onto the surface of the frame data, it can be displayed in the game as if the toy body were actually manipulated.

[0046] In this embodiment, each model has a different shape and movable joints. For example, there is a situation where, in model A, the ankle is movable but the hip joint is fixed, while in model B, the ankle is not movable, but the hip joint is movable instead. In this case, the ankle of the frame data 405 corresponding to model A is movable but the hip joint is fixed, while the ankle of the frame data 405 corresponding to model B is fixed, but the hip joint is movable. By incorporating the actual structural characteristics of the toy into the frame data in this way, the structural characteristics of the toy are reflected in the game based on the appearance information of the toy.

[0047] then, Figure 4 (B) shows a structural example of Table 410, which registers information obtained through scanning for each user. Table 410 registers user information 411, model name 412, scanning information 413, and game data 414. User information 411 is information used to uniquely identify the user (or player) as the owner of the game entity obtained through scanning. User information 411 can also be used as login information when playing the game. Model name 412 registers information indicating the category of the game entity owned by the user. Model name 412 corresponds to one of the information registered as model name 401 in Table 400.

[0048] Scan information 413 contains information about the scanned image obtained by scanning the toy owned by the user registered in username 411. Game data 414 contains game data generated based on scan information 413. For the method of generating game data, please refer to [reference needed]. Figure 7 The flowchart is described later.

[0049] Next, refer to Figure 5 Here is an example of the processing performed by the information processing apparatus 100 corresponding to this embodiment. At least a part of the processing corresponding to this flowchart is implemented by the CPU 101 of the information processing apparatus 100 executing the program stored in the ROM 103 and the storage device 104.

[0050] First, in S501, the CPU 101 determines the category of the toy body 200. In determining the category, for example, an image of the sole of the toy body 200 is input into the information processing device 100. The CPU 101 performs image processing on the image based on the model recognition information 402, thereby determining the category of the toy, i.e., the model name 401.

[0051] In the next step S502, the CPU 101 determines the corresponding fastener information 403 in table 400 according to the information of the determined model name 401. At this time, the CPU 101 can also display the determined fastener information 403 on the display device 140 via the display control unit 107. Specifically, it can display information necessary for determining the fastener, such as the color, material, shape, and fastener number of the fastener, which are meaningful for determining the fastener.

[0052] In the next step, S503, the user of the information processing device 100 selects the fixture 300 according to the information related to the displayed fixture and installs it onto the toy body 200. The fixture 300 can be used to fix the posture of the toy body 200. Then, in S504, after the user installs the toy body 200 onto the fixing member on the turntable, the fixture 300 is removed from the toy body 200. Furthermore, the processes in S503 and S504 can also be performed without human intervention, using a robotic arm or the like, under the control of the CPU 101.

[0053] In the next step S505, the CPU 101 retrieves camera system control information 404 from table 400 corresponding to the category of the toy body 200 determined in S501, performs scanning processing in S506, and generates scan information 413. In the next step S507, the CPU 101 generates game data 414 based on the scan information 413.

[0054] Next, refer to Figure 6 The details of the scanning process in S506 will now be explained. In this embodiment, the camera control information 404 records the drive information of the robot arm 120 used to determine the scanning position and the number of times the scanner 110 takes pictures at that scanning position. Furthermore, when taking pictures at each scanning position, scanning is performed by rotating the rotary table 130.

[0055] First, in S601, the CPU 101 controls the robot arm 120 to move the scanner 110 to any scanning position registered in the camera control information 404. In the following S602, at that scanning position, the CPU 101 rotates the turntable 130 while using the scanner 110 to capture 3D data, thereby acquiring the scan information 413 of the toy body 200. Alternatively, at this time, the turntable 130 can be stopped at a specific rotation position, and the robot arm 120 can be controlled to move the scanner 110 longitudinally while performing a specific scan.

[0056] In the next step, S603, the CPU 101 determines whether there is an unselected scan position in the camera control information 404. If there is an unselected scan position, it returns to S601 and continues processing. On the other hand, if there is no unselected scan position, it proceeds to S604. In S604, the CPU 101 stores the scan information obtained through the above steps as scan information 413 in table 410 into the storage device 104 and ends the scanning process.

[0057] Next, refer to Figure 7 The following describes the details of the processing in S507, which generates game data 414 based on the scan information obtained in S506. First, in S701, the CPU 101 reads frame data 405 from table 400, corresponding to the determined specific game category. Next, in S702, the display control unit 107 displays the scan information and frame data 405 on the display device 140, and corrects the position and angle of the scan information to superimpose it onto the frame data 405. At this time, the CPU 101 can also display a guide screen for aligning the scan information based on the frame data 405, and position the scan information in a manner consistent with the guide screen.

[0058] The processing in S702 is used to adjust the coordinate system set for the scanned information (the coordinate system set during the scanning process; even for toys of the same category, there will be deviations depending on their individual scanned information) to be consistent with the coordinate system of the frame data 405. The scanned information and frame data 405 originally contain information about toys of the same category, so they should be directly superimposed when displayed in a common coordinate system. However, sometimes a slight offset occurs in the coordinate system of the scanned information generated through the scanning process. Due to this offset, the displayed coordinate system will also be offset relative to the frame data 405. In S702, the scanned information itself is moved or rotated to adjust this coordinate system offset.

[0059] Figure 8 An example of a guide screen displayed on the display device 140 during processing in S702 is shown. Figure 8 (A) shows an example of an operation panel 800 in a guide screen. The operation panel 800 displays options for setting the viewing direction (display direction) for the scanned information: "Top," "Side," and "Front." When "Top" is selected, the following is displayed: Figure 8 The display screen 810 shown in (B) displays scan information 811 of the toy body 200 viewed from above, that is, an image of the head 201 of the toy body 200 viewed from above. At this time, for the scan information 811, the corresponding outline position of the frame data 405 is shown using guide information 812. By moving and rotating the position of the scan information 811 so that the guide information 812 is aligned with the outline of the chest of the scan information 811, the scan information 811 can be superimposed on the frame data 405.

[0060] exist Figure 8 In the example, scan information 811 is in Figure 8 In (B), the scan information 811 is shifted upwards and rotated counterclockwise. Therefore, the scan information 811 is moved downwards using the crosshair button and rotated clockwise using the rotary button. Thus... Figure 8 As shown in (C), the outline of the chest in scan information 811 is consistent with that in guide information 812.

[0061] In addition to the above, alignment is performed in the same way for "side" and "front" views, thereby superimposing the scan information 811 onto the frame data. For example, when "side" is selected, guide information for the side of the waist can be displayed to align with the scan information 811. Similarly, when "front" is selected, guide information for the front of the chest and waist can be displayed to align with the scan information 811. In the above, alignment is performed only if there is an offset between the scan information 811 and the guide information. Depending on the scanning method, there may be cases where no offset occurs, in which case no operation instruction for offset correction is required. After the alignment process is completed (which may also be completed without operation instruction), when the "Complete" button is pressed... Figure 8 Once the guide screen finishes displaying, the processing in S702 also ends.

[0062] Back Figure 7 As explained in S702, when the position and angle of the scanned information are adjusted, the process proceeds to S703, where the CPU 101 overlays a pre-prepared image onto the local area corresponding to the deformed part of the playable. S703 is the process for determining the position where the data (image) for the playable is overlaid onto the scanned information. This overlay position can be set to a pre-defined position in the frame data 405 corresponding to the playable's category, or it can utilize... Figure 9The guide screen determines the overlay position based on the specifications received from the scan information.

[0063] The deformable parts of a playable refer to movable parts such as bent joints like the knees and elbows. In this embodiment, since the scanning information is obtained while the playable's arms and legs are extended, if these joints are displayed in a later game setting, their appearance will become unnatural. Therefore, for deformable (movable) parts such as joints, pre-prepared joint data corresponding to the deformity is overlaid according to the playable's category. In this embodiment, regarding the deformable parts, a pre-prepared image corresponding to the deformity is overlaid on the texture data obtained from the scanning information. However, this is not a limitation; it is also possible to omit the texture data obtained from the scanning information.

[0064] Figure 9 This shows an example of a guide screen that overlays joint data onto scan information. Figure 9 Screen (A) 900 shows an example of an operation panel 900 in a guide screen. The operation panel 900 displays selection buttons for specifying which joint position to overlay joint data onto in the specific scan information. Figure 9 In (A), the state of selecting the upper left selection button 901 is shown. That is, the specific right arm part is selected, and the scan information display screen 910 at this time is... Figure 9 As shown in (B). Joint data 912 is overlaid on the right arm joint portion of scan information 911. However, in Figure 9 In (B), the angle between the right arm and joint data 912 is inconsistent. Therefore, the button for counterclockwise rotation on the operation panel 900 is operated, such as... Figure 9 As shown in (C), joint data 912 can be positioned on the right arm of the scan information 911. Furthermore, joint data 912 can be prepared for each specific model, or multiple models can share a single joint data 912. In the above, alignment of the scan information 911 and joint data 912 is performed only if there is an offset between them. Depending on the scanning method, there may be cases where no offset occurs, in which case no operation instruction for offset correction is required. After the alignment process is completed (which may also be completed without operation instruction), when the "Complete" button is pressed, Figure 9 Once the guide screen finishes displaying, the processing in S703 also ends.

[0065] In the next step, S704, CPU 101 generates game data. Specifically, CPU 101 generates texture information for mapping onto frame data based on the scan information generated through the above processing, and stores this texture information, along with the texture coordinates when mapping the texture information onto the frame data, as game data 414 in Table 410 in storage device 104. CPU 101 also includes information on the position for mapping joint data determined in S703 in the game data. The mapping position information is a pre-set position in the frame data 405 corresponding to the specific game category, or it is obtained using... Figure 9 The guide screen is positioned based on the specified information received from the scan. Through the above, game data 414 can be generated.

[0066] In addition, while the robotic arm 120 is used to control the position and posture of the scanner 110, it can also be used manually instead of the robotic arm 120 for positioning.

[0067] then, Figure 10 This describes an example of the process used to execute a game using game data generated through the above-described process. At least a portion of the process corresponding to this flowchart is implemented by the CPU 101 of the information processing device 100 executing programs stored in the ROM 103 and the storage device 104.

[0068] First, in S1001, the CPU 101 accepts user selection. For example, an input screen is displayed on the display device 140, and user information is accepted as input. The user information can also be a combination of a username and password. This information is registered in table 410 as user information 411.

[0069] When a user is identified in S1001 based on the input information, in the following S1002, the CPU 101 reads the information of the game data 414 registered with the user from table 410 and displays the information on the display device 140. In this embodiment, multiple game data can be registered for a user. When multiple game data 414 are registered, their icon images are displayed on the display device 140, and the user can make a selection. When game data 414 is selected, the process moves to S1003, and game processing begins. The game can be, for example, an online or offline battle game in which characters (robots, characters, vehicles, airplanes, etc.) generated based on the game data battle in a designated area in virtual space. The opponent can be the CPU of the game device or other users. Furthermore, the type of game is not limited to battle games; it can be applied to various types of games such as simulation games, role-playing games, and adventure games. Furthermore, in the case of multiple participants, S1003 is executed after repeating the processes of S1001 and S1002 according to the number of participants.

[0070] In S1003, a game character is generated by mapping the texture information contained in the game data 414 to the frame data 405 corresponding to the selected game data 414. At this time, regarding... Figure 7 The movable parts specified in S703 generate an overlay image corresponding to the pose of the game character based on data of a pre-prepared joint model that is overlaid on the texture information contained in the game data 414, and use the overlay image. As a result, a more natural expression can be achieved when the joint parts are bent.

[0071] Furthermore, the above description of the embodiments illustrates the case where game data is generated based on the appearance information of the toy body for use in games implemented in virtual space. However, the use of the game data generated in this embodiment is not limited to games. By using it as virtual space data, images displayed in virtual space can be generated, and it can be widely applied to processing the actions of characters in virtual space. For example, the character can appear in events, concerts, sports, online meetings, etc., conducted in virtual space. Moreover, the technology of this embodiment can also be applied to imaging technologies such as extended reality (XR), which merge the real world and the virtual world, enabling the perception of objects that do not exist in real space.

[0072] Thus, in this embodiment, information about the appearance of the toy can be acquired, and texture information can be generated based on this appearance information and mapped onto frame data for use as a character in the game or any virtual space. The toy, such as a plastic model, is a toy that users paint to create their own artwork. Because these individual characteristics can be reflected in the character's performance in the game or virtual space, it can significantly increase interest.

[0073] The invention is not limited to the above-described embodiments, and various modifications and alterations can be made within the scope of the invention's intent.

Claims

1. An information processing apparatus that generates virtual space data usable in virtual space based on scanned information of the appearance of an object acquired by a 3D scanner configured to acquire 3D shape and color information of the object, the information processing apparatus comprising: The determination unit determines the category of the item; The acquisition unit acquires frame data corresponding to the determined category of the item from a data structure that is associated with the registration of item categories and frame data. The frame data is configured to map texture information obtained based on the scanning information. The display control unit causes the display unit to display a first screen, the first screen being used to adjust the coordinate system set for the first scan information in a manner consistent with the coordinate system set for the frame data, and includes the display of the first scan information, which is generated by scanning the item using the three-dimensional scanner device; The receiving unit is capable of receiving instructions, including at least one of movement and rotation, for the first scan information displayed in the first screen. as well as The first generation unit generates virtual space data containing texture information for mapping onto the frame data, based on the frame data and second scan information obtained after the display of the first screen is terminated in response to the instruction.

2. The information processing apparatus according to claim 1, wherein, The first screen includes the display of the first scan information corresponding to a certain part of the item, and the display showing the position of the part in the frame data. The receiving unit receives an operation instruction to make the display position of the first scan information of the part consistent with the display position of the part.

3. The information processing apparatus according to claim 2, wherein, The first screen includes a switching operation unit for switching the display direction of the first scanned information. The first screen displays the first scan information of a part of the item in the display direction corresponding to the selected switching operation unit, as well as the position of the part in the frame data.

4. The information processing apparatus according to any one of claims 1 to 3, wherein, The first generation unit generates the virtual space data in a manner that includes positional information of an image superimposed on the second scan information corresponding to a specified portion of the item.

5. The information processing apparatus according to any one of claims 1 to 3, wherein, The display control unit further causes the display unit to display a second screen, the second screen being used to adjust the position of the image superimposed on the first scanned information corresponding to a predetermined portion of the item. The receiving unit is also capable of receiving a second operation instruction for any of the movement and rotation of the image displayed in the second screen. The first generation unit generates the virtual space data in a manner that also includes the location information of the image corresponding to the second operation instruction.

6. The information processing apparatus according to claim 4, wherein, The specified part refers to the movable part of the article.

7. The information processing apparatus according to claim 5, wherein, The specified part refers to the movable part of the article.

8. The information processing apparatus according to claim 6 or 7, wherein, In the case where the object is a humanoid toy, the movable part is a joint.

9. The information processing apparatus according to any one of claims 1 to 3, wherein, It also includes a selection unit that selects a fixture for specifying the posture of the item when it is scanned using the 3D scanner device, based on the determined category of the item.

10. The information processing apparatus according to claim 9, wherein, The first scan information is generated by scanning the object in a posture fixed by the fixture using the three-dimensional scanner device.

11. The information processing apparatus according to any one of claims 1 to 3, wherein, It also includes a position and posture control device, on which the 3D scanner device is disposed, and the position and posture control device controls the position and posture of the 3D scanner device. The first scan information is generated by scanning the object in a predetermined position and posture set by the position and posture control device using the three-dimensional scanner device.

12. The information processing apparatus according to any one of claims 1 to 3, wherein, It also includes a rotating device configured to hold and rotate the article. The first scan information is generated by scanning the article placed on the rotating device using the 3D scanner device.

13. The information processing apparatus according to any one of claims 1 to 3, wherein, The determination unit determines the category of the item based on information about a specified part of the item.

14. The information processing apparatus according to claim 13, wherein, In the case where the item is a humanoid toy, the designated part is the sole of the foot.

15. An image processing apparatus that uses virtual space data generated by the information processing apparatus according to any one of claims 4 to 8 to generate an image displayed in a virtual space. The image processing device includes a second generation unit, which generates display information for a specified character corresponding to the item by mapping texture information contained in the virtual space data to frame data. in, The second generation unit generates the display information based on the location information contained in the virtual space data, in a manner that displays a specified overlay image, excluding texture information, contained in the virtual space data at a specified location in the character's display information corresponding to a specified part of the item.

16. The image processing apparatus according to claim 15, wherein, The overlay image is a common image across multiple categories of the item.

17. The image processing apparatus according to claim 15, wherein, The overlay image is an image inherent to the category of the item.

18. A computer program product comprising a computer program for causing a computer to operate as a unit of an information processing apparatus according to any one of claims 1 to 13 or an image processing apparatus according to any one of claims 15 to 17.

19. A calculator-readable storage medium containing a computer program for causing the computer to operate as a unit of an information processing apparatus according to any one of claims 1 to 13 or an image processing apparatus according to any one of claims 15 to 17.

Images