Information Processing Apparatus, Information Processing Method, and Program

Abstract

Provided is an information processing device (300) comprising: a data acquisition unit (320) that acquires real object sensing data; a parameter acquisition unit (330) that acquires a prescribed parameter; and an object generation unit (340) that extracts a feature amount from the sensing data, and generates a virtual object on the basis of the feature amount, the object generation unit adjusting the virtual object in accordance with the prescribed parameter.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Recently, the opportunities to use online communication and business have been increasing. For example, as an online business, there is a technology for selling and buying second-hand goods between a seller and a buyer using an e-commerce trading platform without going through a store.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Unlike real communication that takes place in a store, office, etc., online communication as described above has limitations in the amount of information that can be transmitted, and thus has problems unique to online communication. Therefore, in order to solve the above problems, attention has been focused on technologies for communicating in a virtual space through an avatar or the like that serves as a user's alter ego.

[0005] However, heretofore, no studies have been made on technologies that can give further added value to avatars and the like used in communication in a virtual space.

[0006] Therefore, in view of the above situation, the present disclosure proposes an information processing apparatus, an information processing method, and a program that can give new added value to virtual objects such as avatars used in communication in a virtual space and the like.

Means for Solving the Problem

[0007] According to the present disclosure, there is provided an information processing apparatus including a data acquisition unit that acquires sensing data of a real object, a parameter acquisition unit that acquires a predetermined parameter, and an object generation unit that extracts a feature amount from the sensing data and generates a virtual object based on the feature amount, wherein the object generation unit adjusts the virtual object according to the predetermined parameter.

[0008] Further, according to the present disclosure, there is provided an information processing method including acquiring sensing data of a real object, acquiring a predetermined parameter, extracting a feature amount from the sensing data, and generating a virtual object based on the feature amount by an information processing apparatus, and adjusting the virtual object according to the predetermined parameter when generating the virtual object.

[0009] Furthermore, according to the present disclosure, there is provided a program for causing a computer to realize a function of acquiring sensing data of a real object, a function of acquiring a predetermined parameter, a function of extracting a feature amount from the sensing data, and a function of generating a virtual object based on the feature amount, and adjusting the virtual object according to the predetermined parameter when generating the virtual object.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, for components having substantially the same functional configuration, the same reference numerals are given to omit redundant explanations. Also, in this specification and the drawings, for similar components in different embodiments, different alphabets may be attached after the same reference numeral for distinction. However, when it is not particularly necessary to distinguish each of the similar components, only the same reference numeral is given.

[0012] In this specification, the term "icon" means a graphic that represents an object or the like by an illustration or the like while leaving an image of the shape of the object or the like. Also, in this specification, the term "avatar" means a character image that functions as a user's alter ego or a user identification graphic in a virtual space.

[0013] The description will be made in the following order. 1. Background leading to the creation of the embodiments according to the present disclosure 2. Embodiments 2.1 Overview of the information processing system 10 2.2 Detailed configuration of the user terminal 100 2.3 Detailed configuration of the server 300 2.4 Detailed configuration of the object generation unit 340 2.5 Information processing method 3. Examples 3.1 Example 1 3.2 Example 2 3.3 Example 3 4. Summary 5. Regarding Hardware Configuration 6. Supplementary Explanation

[0014] <<1. Background Leading to the Creation of Embodiments of the Present Disclosure>> First, before explaining the details of the embodiments of the present disclosure, the background leading to the creation of the embodiments of the present disclosure by the inventor will be described with reference to FIGS. 1 and 2. FIG. 1 is an explanatory diagram for explaining an example of avatar generation according to an embodiment of the present disclosure, and FIG. 2 is an explanatory diagram for explaining an example of icon generation according to an embodiment of the present disclosure.

[0015] Recently, due to the spread of the novel coronavirus (COVID-19), the opportunities to use online communication means have been increasing. Examples of such online communication means include video conferencing systems and chat tools. However, these only establish communication by sharing only voice of speech, face images, text, etc., and cannot mutually share communication elements such as the "atmosphere" and "reactions" of the other party, which are difficult to digitalize. That is, since the amount of information that can be transmitted by online communication means is limited compared to real communication, there are issues unique to online communication.

[0016] Therefore, in order to solve the above-described problems, as a means for establishing a richer (more information-rich) communication even online, means for establishing communication on a virtual space, for example, via an avatar that serves as the user's alter ego, etc. have attracted attention. Through the detailed actions of the avatar, emotions that cannot be expressed by words or the like can be conveyed to the other party, and by having the avatars of the participating users exist on the same virtual space, the participating users can share the feeling of "being in the same place, right there, now" with each other. In this way, by using an avatar or the like, the deficiencies of conventional online communication can be compensated for, and richer communication can be established. More specifically, the user can prepare an avatar that serves as their alter ego by drawing an illustration themselves or by selecting and combining pre-prepared parts (face, hairstyle, eyes, mouth, etc.). Furthermore, the user can freely move the avatar on the virtual space by their own operation, or can freely move the avatar based on the image of a camera that captures the user themselves or the sensing data of a motion sensor worn by the user.

[0017] Based on such a situation, the inventor has considered that by more strongly connecting the virtual (virtual space) and the real, the communication via an avatar or the like can be made closer to a more realistic feeling. Therefore, the inventor conceived of a technology that can automatically generate an avatar that closely resembles the user themselves or an icon that closely resembles an item actually possessed by the user and can be used on a virtual space. Specifically, according to the technology conceived by the inventor, as shown in FIG. 1, an avatar 900 that closely resembles the user 800 can be automatically generated based on an image of the user 800, or as shown in FIG. 2, an icon 910 that closely resembles the item 810 can be automatically generated based on an image of the item 810.

[0018] Furthermore, the inventor of the present invention has not merely generated avatars 900 and icons 910 (virtual objects) that simply resemble the user 800 and the article 810 (actual object) based on the image, but has independently created a technique for generating avatars 900 and icons 910 with their designs changed according to information (parameters) regarding the user 800 and the article 810, such as the position information and biological information of the user 800, and the sales volume and quality of the article 810. According to such a technique, it is possible to generate avatars 900 and icons 910 with certain features emphasized, or avatars 900 and icons 910 with designs that give a sense of surprise and freshness that the user 800 cannot anticipate. That is, according to the embodiment of the present disclosure created by the inventor, the user 800 can enjoy avatars 900 and icons 910 that not only resemble the user 800 and the article 810 but also vary in various ways depending on various information, and can add value to the generated avatars 900 and icons 910. The details of the embodiment of the present disclosure created by the inventor will be described below.

[0019] <<2. Embodiment>> <2.1 Outline of Information Processing System 10> First, with reference to FIG. 3, a configuration example of an information processing system 10 according to an embodiment of the present disclosure will be described. FIG. 3 is a configuration diagram of the information processing system 10 according to the embodiment of the present disclosure.

[0020] As shown in FIG. 3, the information processing system 10 according to the present embodiment includes a user terminal 100 and a server 300, which are communicably connected to each other via a communication network 500. Specifically, the user terminal 100 and the server 300 are connected to the communication network 500 via a base station or the like (not shown), such as a base station of a mobile phone or an access point of a wireless local area network (LAN). Note that any communication method can be applied to the communication network 500, whether wired or wireless, but it is desirable to apply a communication method that can maintain stable operation. The outlines of the respective devices included in the information processing system 10 according to the embodiment of the present disclosure will be described below.

[0021] (User terminal 100) The user terminal 100 is a terminal for receiving sensing data such as images by the operation of the user 800 and displaying virtual objects generated by the server 300 to the user 800. For example, the user terminal 70 can be a wearable device such as a tablet, a smartphone, a mobile phone, a laptop PC (Personal Computer), a notebook PC, a desktop PC, an HMD (Head Mounted Display), etc. The wearable device is a device that can be worn on a part of the body of the user 800. More specifically, various types of wearable devices such as HMD type, bracelet type, and eyewear type can be adopted. The details of the user terminal 100 will be described later.

[0022] (Server 300) The server 300 is configured by, for example, a computer or the like. The server 300 is owned by, for example, a service provider that provides services according to this embodiment and provides services to each user 800. Specifically, the server 300 provides virtual objects (avatars 900, icons 910, etc.) to each user 800. The details of the server 300 will be described later.

[0023] In addition, in FIG. 3, the information processing system 10 according to this embodiment is shown as including one user terminal 100, but it is not limited to this in this embodiment. For example, the information processing system 10 according to this embodiment may include a plurality of user terminals 100. Further, the information processing system 10 according to the embodiment may include other communication devices such as a relay device for transmitting and receiving sensing data and virtual objects between the user terminal 100 and the server 300.

[0024] <2.2 Detailed configuration of the user terminal 100> Next, the configuration of the user terminal 100 according to the embodiment of the present disclosure will be described with reference to FIG. 4. FIG. 4 is a block diagram of the user terminal 100 according to the embodiment of the present disclosure. As shown in FIG. 11, the user terminal 100 mainly includes a sensor unit 102, an input unit 104, a display unit 106, a control unit 110, and a communication unit 112. Details of each functional unit of the user terminal 100 will be described below.

[0025] (Sensor unit 102) The sensor unit 102 is provided in the user terminal 100 and can be an imaging device that captures images of the user 800 and the articles 810 (real objects) possessed by the user. The sensor unit 102 can output the acquired image data (sensing data) to the control unit 110 described later. The imaging device is not limited to a visible light camera and may be a depth sensor such as a ToF (Time of Flight) sensor that acquires depth information of a real object by reflection of infrared rays. Here, the depth information is depth information, which is one of the distance information from the ToF sensor to each point on the surface of the real object. By aggregating the depth information of a plurality of points on the surface, it can be treated as shape information regarding the uneven shape of the surface of the real object, that is, information on the outer shape of the real object.

[0026] Furthermore, the sensor unit 102 may include various sensors used for authentication such as fingerprint authentication, face authentication, iris authentication, gait authentication, and pulse authentication. In order to obtain the parameters according to the present embodiment, it may also include a motion sensor for detecting the operation of the user 800. For example, the motion sensor is realized by one or more sensor devices such as an acceleration sensor, a gyro sensor, and a geomagnetic sensor, and detects changes such as acceleration and angular velocity generated along with the operation of the user 800. In addition, the sensor unit 102 may include a positioning sensor for detecting the position of the user 800 in order to obtain the parameters according to the present embodiment. The positioning sensor can be, for example, a GNSS (Global Navigation Satellite System) receiver or the like. In this case, the positioning sensor can generate sensing data indicating the latitude and longitude of the current location of the user 800 based on signals from GNSS satellites. Further, since the above positioning sensor can detect the relative positional relationship of the user 800 from, for example, RFID (Radio Frequency Identification), Wi-Fi access points, information of radio base stations, etc., it is also possible to use such a communication device as the above positioning sensor. Furthermore, the sensor unit 102 may include a biological information sensor for obtaining the biological information of the user 800 in order to obtain the parameters according to the present embodiment. The above biological information sensor can be various biological information sensors such as a heartbeat sensor, a pulse sensor, a blood flow sensor (including a blood pressure sensor), a respiration sensor (including a calorie consumption meter based on respiration volume), an electroencephalogram sensor, a skin temperature sensor, a skin conductivity sensor, a sweating sensor, and an electromyogram sensor. In addition, the sensor unit 102 may also include an environmental information sensor for obtaining environmental information such as illuminance, temperature, and humidity of the surrounding environment in order to obtain the parameters according to the present embodiment.

[0027] Also, in the present embodiment, the sensor unit 102 is not limited to being provided in the user terminal 100, and may be provided as a device separate from the user terminal 100.

[0028] (Input unit 104) The input unit 104 can receive the input of data and commands (for example, parameters according to this embodiment) to the user terminal 100, and output the received input data to a control unit 110 described later. More specifically, the input unit 104 is realized by a touch panel, a keyboard, or the like. Further, the input unit 104 may be a microphone that receives voice input from the user 800.

[0029] (Display unit 106) The display unit 106 can display virtual objects and the like transmitted from the server 300 according to the control by the control unit 110 described later. Specifically, the display unit 106 is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like. In this embodiment, the display unit 106 may be provided so as to be fixed to the user terminal 100, or may be provided as a device separate from the user terminal 100.

[0030] (Control unit 110) The control unit 110 is provided in the user terminal 100 and can control each functional unit in the user terminal 100. The control unit 110 is realized by hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), for example.

[0031] Further, the control unit 110 incorporates a clock mechanism (not shown) for grasping the accurate time, and for example, may associate the time when the sensing data is acquired with the sensing data and transmit it to the server 300.

[0032] (Communication unit 112) The communication unit 112 is provided in the user terminal 100 and can transmit and receive information to and from an external device such as the server 300. In other words, the communication unit 112 can be said to be a communication interface having a function of transmitting and receiving data. For example, the communication unit 112 is realized by communication devices such as a communication antenna, a transmission and reception circuit, and a port.

[0033] Note that in this embodiment, the user terminal 100 is not limited to the configuration shown in FIG. 4, and may include, for example, a speaker or the like.

[0034] <2.3 Detailed Configuration of Server 300> Next, the configuration of the server 300 according to the embodiment of the present disclosure will be described with reference to FIG. 5. FIG. 5 is a block diagram of the server 300 according to the embodiment of the present disclosure. As shown in FIG. 5, the server 300 mainly includes a processing unit 310, a storage unit 400, and a communication unit 410. The details of each functional unit of the server 300 will be described below.

[0035] (Processing Unit 310) The processing unit 310 is provided in the server 300 and can control each block of the server 300. The processing unit 310 is realized by hardware such as a CPU, a ROM, and a RAM, for example. Further, the processing unit 310 can also function as a sensing data acquisition unit (data acquisition unit) 320, a parameter acquisition unit 330, an object generation unit 340, an output unit 350, and a management unit 360. The details of these functions of the processing unit 310 according to the present embodiment will be described below.

[0036] The sensing data acquisition unit 320 acquires, for example, sensing data (image) of a real object (user 800, article 810, etc.) transmitted from the user terminal 100, and outputs the acquired sensing data to an object generation unit 340 described later. As described above, in the present embodiment, the sensing data can be an image of the user 800 or an article 810 (real object) possessed by the user 800 by an imaging device, depth information of a real object by a ToF sensor, or the like.

[0037] The parameter acquisition unit 330 acquires, for example, a parameter (a predetermined parameter) transmitted from the user terminal 100, and outputs the acquired parameter to an object generation unit 340 described later. In the present embodiment, the parameter can be at least one of a position parameter, a time parameter, an attribute information parameter, and an environment parameter related to the real object, the virtual object, or the user 800.

[0038] More specifically, the position parameter can be position information in the real space of the user 800, the article 810 (real object) (for example, office, café, etc.), or position information in the virtual space or real space where the virtual object is displayed (for example, office, café, etc. in the virtual space). For example, the position information in the real space of the user 800 or the article 810 can be acquired by the positioning sensor or the like of the sensor unit 102 described above.

[0039] The time parameter can be time information in the real space of the user 800 or the article 810 (real object) (for example, morning, late at night, etc.), or time information in the virtual space where the virtual object is displayed (for example, morning, late at night, etc. in the virtual space). For example, the time information in the real space of the user 800 or the article 810 can be acquired from a clock mechanism (not shown) for grasping the accurate time built in the user terminal 100.

[0040] In addition, the attribute information parameter can be attribute information such as the gender, age, body type, etc. of the user 800, or attribute information such as the quality of the article 810, the purchase place (purchased at the event venue), the purchase time, the usage period, the sales quantity, the purchase price, etc. For example, such attribute information can be acquired by input from the user 800 or the like.

[0041] Furthermore, the environmental parameters can be environmental information (such as illuminance) of the actual space where the user 800 and the article 810 (actual object) exist, or environmental information (such as illuminance) of the virtual space or the actual space where the virtual object is displayed. For example, such environmental information can be obtained by a sensor such as an illuminance meter provided in the user terminal 100.

[0042] The object generation unit 340 can extract feature amounts from sensing data such as an image and generate virtual objects (avatar 900, icon 910) based on the feature amounts. For example, it can be realized by a neural network such as a VAE (Variational Autoencoder) or a conditional VAE (Conditional Variational Autoencoder). In this embodiment, the object generation unit 340 can adjust the virtual object according to the parameters acquired by the parameter acquisition unit 330. Furthermore, the object generation unit 340 can output the generated virtual object to an output unit 350 and a management unit 360, which will be described later. The detailed configuration of the object generation unit 340 will be described later.

[0043] The output unit 350 can output the virtual object to an external device such as the user terminal 100. For example, the output unit 350 can cause the user terminal 100 to display the virtual object on the virtual space. Also, the output unit can cause the user terminal 100 to display the virtual object so as to be superimposed on the real-space image, or display (project) the virtual object so as to be superimposed on the real space.

[0044] The management unit 360 can associate the generated icon 910 (virtual object) with the user 800 (for example, the user 800 who owns or borrows the article 810) or the article 810 (physical object) related to the sensing data used when generating the icon 910, and store it in the storage unit 400 described below. Further, the management unit 360 may associate the generated icon 910 with the avatar 900 (virtual object) of the user 800 and store it in the storage unit 400 described below.

[0045] (Storage unit 400) The storage unit 400 is provided in the server 300 and stores programs, information, etc. for the above-described processing unit 310 to execute various processes, and information (virtual objects) obtained by the processes. Note that the storage unit 400 is realized by, for example, a magnetic recording medium such as a hard disk (HD) or a nonvolatile memory such as a flash memory.

[0046] (Communication unit 410) The communication unit 410 is provided in the server 300 and can transmit and receive information to and from external devices such as the user terminal 100. Note that the communication unit 410 is realized by communication devices such as a communication antenna, a transmission / reception circuit, and a port.

[0047] Note that in the present embodiment, the server 300 is not limited to the configuration shown in FIG. 5.

[0048] <2.4 Detailed configuration of object generation unit 340> Next, with reference to FIGS. 6 to 8, a configuration example of the object generation unit 340 according to the embodiment of the present disclosure will be described. FIG. 6 is a block diagram of the object generation unit of FIG. 5, and FIGS. 7 and 8 are explanatory diagrams for explaining a method of generating a virtual object according to the embodiment of the present disclosure.

[0049] In this embodiment, as described above, the object generation unit 340 can be implemented by a neural network such as a VAE (Variational Autoencoder), a conditional VAE, or the like. Therefore, here, the object generation unit 340 will be described as a conditional VAE.

[0050] An autoencoder is a neural network that can extract, for example, the features of input data (an image in this embodiment), compress the extracted features into a latent variable of a low dimension, and restore the original input data using the latent variable. It is assumed that the autoencoder has been pre-trained so that the output data restored by the autoencoder is closer to the input data (that is, the difference between the input data and the output data is reduced). Furthermore, since the autoencoder can obtain a latent variable of a reduced dimension that can restore the original data, it can be said that the autoencoder can extract only the important features for restoring the original data.

[0051] Furthermore, in a VAE, the latent variable is treated as a variable that follows a predetermined probability distribution (for example, a normal distribution, a Bernoulli distribution, etc.). By doing so, the latent variable can be treated as a variable (distribution) determined by a limited number of variables, and furthermore, it becomes possible to generate data that resembles the original data but is different from the original data. In addition, in a VAE, by imposing constraints on the latent variable as a variable that follows a predetermined probability distribution, it becomes easier to adjust the latent variable, for example, to obtain output data that emphasizes a predetermined feature more strongly.

[0052] Also, in a conditional VAE, it is possible to obtain output data closer to the target by adding a condition (in this embodiment, the category of the article 810). For example, when training a conditional VAE with images of "dogs" and "cats", a condition of "dog" is added to the image of a "dog", and a condition of "cat" is added to the image of a "cat". Also, when outputting, a condition of "dog" or "cat" is added. By doing so, output data that conforms to the added condition can be obtained.

[0053] Specifically, as shown in FIG. 6, the object generation unit 340 mainly includes an encoding unit 342, a decoding unit 344, an adjustment unit 346, and a category acquisition unit 348. The details of each functional unit of the object generation unit 340 will be described below.

[0054] (Encoding Unit 342) The encoding unit 342 can extract features from the sensing data (e.g., image) of the actual object (user 800, article 810) on the condition of the category output from the category acquisition unit 348 described later, and can fold it into a latent variable with a predetermined dimension. Then, the encoding unit 342 can output the obtained latent variable to the adjustment unit 346.

[0055] (Decoding Unit 344) The decoding unit 344 can generate virtual objects (avatar 900, icon 910) based on the latent variable adjusted by the adjustment unit 346 described later on the condition of the category output from the category acquisition unit 348 described later.

[0056] (Adjustment Unit 346) The adjustment unit 346 can adjust the latent variable based on the parameter. Specifically, the adjustment unit 346 changes the latent variable by the amount of change determined based on the parameter. In other words, it acquires a new latent variable that is separated from the original latent variable by the amount determined based on the parameter. Furthermore, the adjustment unit 346 outputs the adjusted latent variable to the decoding unit 344.

[0057] (Category Acquisition Unit 348) The category acquisition unit 348 can acquire the category (bag, shirt, etc.) of the actual object (article 810), and input the acquired category to the encoding unit 342 and the decoding unit 344 as a condition. For example, the category may be acquired by the input from the user 800, or may be estimated from the feature amount by analyzing the image of the actual object.

[0058] Specifically, for example, the category acquisition unit 348 can acquire the category of the real object by using a DNN (Deep Neural Network) or the like pre-trained using an image (teacher data) of a real object labeled with a category. More specifically, for example, the category acquisition unit 348 can extract feature amounts and their distributions from the image of the real object, and acquire the category of the real object by searching for a category similar to the extracted feature amounts.

[0059] More specifically, the operation of the object generation unit 340 will be described with reference to FIG. 7. As shown in FIG. 7, the encoding unit 342 and the decoding unit 344 are pre-trained with images (teacher data) of a plurality of other real objects belonging to the same category as the real object (such as the article 810) (in FIG. 7, a bag), a plurality of feature amounts (such as shape, decoration, pattern, design, etc.) of the real object (teacher data), and other virtual objects (icons 910 etc. prepared in advance) corresponding to the plurality of other real objects (teacher data). By doing so, a generative model of latent variables indicating the relationship between the image of the real object of a predetermined category (bag) and the virtual object corresponding to the real object can be obtained.

[0060] Then, the case where the image (input data) of the real object is input to the encoding unit 342 without passing through the adjustment unit 346 and the virtual object (output data) is generated by the decoding unit 344 is shown on the left side of FIG. 8. In the present embodiment, as shown on the right side of FIG. 8, instead of generating a virtual object using the latent variable obtained by the encoding unit 342, the adjustment unit 346 adjusts the latent variable obtained by the encoding unit 342 based on the parameter (shifts the position), and generates a virtual object using the adjusted latent variable. By doing so, in the present embodiment, it is possible to obtain not simply a virtual object similar to the real object, but a virtual object or the like in which certain features are emphasized based on various parameters. As a result, according to the present embodiment, virtual objects of various variations can be obtained, and added value can be given to the generated virtual objects.

[0061] <2.5 Information Processing Method> As described above, the configuration of the information processing system 10 according to the present embodiment and the user terminal 100 and the server 300 included in the information processing system 10 have been described in detail. Next, the information processing method according to the present embodiment will be described with reference to FIG. 9. FIG. 9 is a sequence diagram of the information processing method according to the embodiment of the present disclosure.

[0062] Specifically, as shown in FIG. 9, the information processing method according to the present embodiment includes a plurality of steps from step S101 to step S105. Hereinafter, the details of each step included in the information processing method according to the present embodiment will be described.

[0063] The user terminal 100 captures an image of the user 800 or the article 810 (real object) and transmits the image to the server 300 (step S101).

[0064] The user terminal 100 receives the input from the user 800, various sensing data, etc., and transmits the received information to the server 300 as a parameter (step S102).

[0065] The server 300 performs recognition processing on the image and classifies the category of the actual object (step S103).

[0066] The server 300 generates a virtual object using the category, the image, and the parameters (step S104).

[0067] The server 300 transmits the generated virtual object to the user terminal 100 (step S105).

[0068] In the present embodiment, instead of generating a virtual object using the latent variable obtained based on the image, the latent variable is adjusted based on the parameters, and a virtual object is generated using the adjusted latent variable. By doing so, in the present embodiment, it is possible to obtain not simply a virtual object similar to the actual object but a virtual object in which certain features are emphasized based on various parameters. As a result, according to the present embodiment, it is possible to obtain virtual objects with various variations and to give added value to the generated virtual objects.

[0069] <<3. Example>> Above, the details of the information processing method in the embodiment of the present disclosure have been described. Next, an example of information processing according to the present embodiment will be described in more detail while showing specific examples. Note that the examples shown below are merely examples of the information processing according to the present embodiment, and the information processing according to the present embodiment is not limited to the following examples.

[0070] <3.1 Example 1> First, an example in which the embodiment of the present disclosure is applied to a secondary distribution system using an e-commerce platform will be described. Here, secondary distribution means reselling a product (article) 810 that has once been sold to a consumer as a used product with respect to the primary distribution that sells new products.

[0071] (Regarding the subject) First, the entities related to the secondary distribution service will be described with reference to FIG. 10. FIG. 10 is a diagram for explaining the entities related to the secondary distribution service according to Example 1. As shown in FIG. 10, the entities related to the secondary distribution service are the seller, the purchaser, and the secondary distribution platform. The secondary distribution platform provides an e-commerce platform through which transactions of product 810 are conducted via a network between the seller who wishes to sell product 810 and the purchaser who wishes to buy product 810. The seller lists product 810 through the secondary distribution platform. The purchaser purchases product 810 through the secondary distribution platform.

[0072] Thus, in the secondary distribution through the secondary distribution platform, since transactions are conducted via a network without going through a store, unlike purchasing in a store, the purchaser will confirm the state of product 810 etc. through images etc. Therefore, it is difficult for the purchaser to judge an appropriate price for product 810 and may hesitate to make a purchase. That is, such a state has been an obstacle to promoting the use of secondary distribution through the secondary distribution platform.

[0073] Therefore, in order to provide more detailed information for the purchaser to judge the price and the possibility of purchase, for example, in the above Patent Document 1 cited as a prior art document, a technique is disclosed in which information regarding product 810 (purchase price, purchase date and time, usage period, etc.) is obtained, the quality state of product 810 is estimated, and the estimation result is reflected in the price of the said product 810. And by using such a technique, since the purchaser can obtain a price that reflects the state of product 810 in detail that cannot be known from an image, the purchaser can make a purchase with confidence. As a result, the use of secondary distribution through the secondary distribution platform will be promoted.

[0074] However, the inventor considered that there is still room for improvement in the secondary distribution platform in order to promote secondary distribution through the secondary distribution platform. For example, in the technology disclosed in Patent Document 1 above, the price is determined with reference to the trial period of products such as product 810. When the price is determined in this way, it will not exceed the price at which the product 810 was sold as new, and the scarcity of the product 810 and the degree of arousing the purchasing desire of collectors are not reflected in the price. That is, in the secondary distribution through the secondary distribution platform, the product 810 can only have the value as a mere used product. Therefore, for the seller, since the product will be sold at a price far from the value of the product 810 that the seller feels, the seller may be reluctant to list the product.

[0075] Therefore, in view of the above situation, the inventor conceived of generating and using the icon 910 of the product 810 to be traded in order to create and add a value other than the value as a mere used product to the product 810 in the secondary distribution through the secondary distribution platform. For example, the user 800 can view the icon 910 of the product 810 owned by another user 800 and can rent the icon 910. By doing so, the user 800 can obtain and collect various icons 910. Therefore, according to the technology conceived by the inventor, a new value of the product 810 that cannot be created by the product 810 alone can be provided. Furthermore, since the purchaser can purchase the actual product 810 while enjoying it through the icon 910, the secondary distribution of the product 810 can be promoted. Also, make it possible to purchase the product 810 from the icon information to promote the distribution of the product 810.

[0076] At this time, the inventor of the present invention did not simply generate the icon 910 that closely resembles the product 810 based on the image of the product 810, but conceived of generating the icon 910 of the product 810 with its design changed based on information such as the sales period, sales location, and sales event of the product 810. By doing so, the user 800 can enjoy the icon 910 that varies depending on information such as the sales period, sales location, and sales event of the product 810, and more added value can be given to the product 810. And in this embodiment, when generating such an icon 910, the above-described embodiments of the present disclosure can be applied. Hereinafter, the details of this embodiment will be sequentially described.

[0077] (Configuration of Information Processing System) Next, the configuration of the information processing system 10a according to this embodiment will be described with reference to FIG. 11. FIG. 11 is a configuration diagram of the information processing system 10a according to Embodiment 1. As shown in FIG. 11, the information processing system 10 according to this embodiment includes a user terminal 100a on the seller side, a user terminal 100b on the purchaser side, and a server 300a, and these are communicably connected to each other via a communication network 500. Note that, as shown in FIG. 11, the information processing system 10a according to this embodiment is shown as including two user terminals 100a and 100b, but is not limited to this in the present embodiment. For example, the information processing system 10a according to this embodiment may include three or more user terminals 100. Hereinafter, an outline of each device included in the information processing system 10a according to this embodiment will be described.

[0078] ~User Terminals 100a, 100b~ The user terminal 100a is an information processing terminal used by a seller, and the user terminal 100b is an information processing terminal used by a purchaser. For example, the user terminals 100a and 100b can be wearable devices such as tablets, smartphones, mobile phones, laptop-type PCs, notebook-type PCs, desktop-type PCs, and HMDs. Note that the details of the user terminals 100a and 100b will be described later.

[0079] ~Server 300a~ Server 300a is a server device that provides a secondary distribution platform and is configured by, for example, a computer or the like. Server 300a provides an e-commerce transaction platform through which the sale and purchase of product 810 are conducted via a network between the seller who wishes to sell product 810 and the purchaser who wishes to buy product 810. Details of Server 300a will be described later.

[0080] (Configuration of User Terminals 100a and 100b) Next, the configuration of user terminals 100a and 100b according to this embodiment will be described. Except for the following points, user terminals 100a and 100b according to this embodiment share the same configuration as user terminal 100 according to the embodiment already described. Therefore, the description of the common points will be omitted here.

[0081] User terminals 100a and 100b according to this embodiment have a personal authentication information acquisition function. For example, the personal authentication information acquisition function is realized by, for example, an imaging device or a fingerprint authentication sensor mounted on user terminals 100a and 100b. For example, the personal authentication information acquisition function acquires fingerprint data, face image data, or iris image data of the seller or purchaser as personal authentication information. When the authentication is properly executed, other functions of user terminals 100a and 100b can be executed, or data can be transmitted to Server 300a. In this embodiment, personal authentication is not limited to fingerprint authentication, face authentication, iris authentication, etc., and may be gait authentication, pulse authentication, etc.

[0082] (Configuration of Server 300a) Next, with reference to FIGS. 12 and 13, the configuration of Server 300a according to this embodiment will be described. FIG. 12 is a block diagram of Server 300a according to Embodiment 1, and FIG. 13 is an explanatory diagram for explaining a configuration example of the database stored in storage unit 400a of FIG. 12.

[0083] As shown in FIG. 13, the server 300a mainly includes a processing unit 310a, a storage unit 400a, and a communication unit 410. Details of each functional unit of the server 300a will be described below. However, since the server 300a is common to the configuration of the server 300 according to the present embodiment already described, except for the following points, the description of the common points will be omitted here.

[0084] ~Processing Unit 310a~ The processing unit 310a is provided within the server 300a and can control each block of the server 300a. Further, the processing unit 310a can also function as a sensing data acquisition unit (data acquisition unit) 320, a parameter acquisition unit 330, an object generation unit 340, an output unit 350, a management unit 360, an authentication unit 370, a recommendation unit 380, and a settlement unit 390. Details of these functions of the processing unit 310a according to the present embodiment will be described below. However, since the processing unit 310a is common to the configuration of the processing unit 310 according to the present embodiment already described, except for the following points, the description of the common points will be omitted here.

[0085] The authentication unit 370 can perform personal authentication of the user 800. Specifically, the authentication unit 370 acquires personal authentication information of the user 800 from the user terminals 100a and 100b, and performs personal authentication by comparing it with the personal feature amount (personal authentication information) associated with the personal ID (identification) in the personal information DataBase (DB) 402 (see FIG. 13) stored in the storage unit 400a in advance. For example, the authentication unit 370 can perform personal authentication based on personal authentication information such as fingerprint information, iris information, or face information of the user 800.

[0086] The recommendation unit 380 can select a product 810 or an icon 910 to be recommended to the purchaser based on various information about the purchaser (such as the purchase history of the product 810, the use of the icon 910, etc.), and provide information on the selected product 810 or icon 910 to the purchaser.

[0087] The settlement unit 390 can present a price to the purchaser based on the product 810 selected by the purchaser, and when receiving an instruction for settlement processing from the purchaser, perform settlement processing based on the presented price.

[0088] ~Memory unit 400a~ The memory unit 400a is provided within the server 300a and stores programs, information, etc. for the above-described processing unit 310 to execute various processes, as well as information of the user 800, information of the avatar 900, and information of the icon 910, etc. Specifically, as shown in FIG. 13, the memory unit 400a mainly stores a personal information DB 402 and a product information DB 404.

[0089] The personal information DB 402 stores a personal ID, personal feature amounts, and an image of the avatar 900. The personal ID is identification information for identifying the user 800 who is the seller or purchaser. The personal feature amounts can be face information, fingerprint information, iris information, etc. of the user 800 and can be used as personal authentication information. Further, the avatar image is image information of the avatar 900 (virtual object) associated with the user 800.

[0090] The product information DB 404 stores a product ID, product feature amounts, an image of the icon 910, ownership information, and purchase site information. The product ID is identification information for identifying the product 810. The product feature amounts indicate feature amounts extracted from the image of the product. The image of the icon 910 is image information of the icon 910 (virtual object) associated with the product 810. The ownership information is information indicating whether the user 800 is the owner or borrower of the product 810. The purchase site information is information indicating the purchasable purchase destinations of the product 810 and the transaction status of the product 810 or the icon 910 (for example, on sale, on rental, etc.). Further, the product information DB 404 may store information such as brand names, product categories, sizes, years of use, quality, etc., and images of the product 810.

[0091] (Information processing method) As described above, the configuration of the information processing system 1a according to this embodiment, and the user terminals 100a and 10b and the server 300a included in the information processing system 10a have been described in detail. Next, the information processing method according to this embodiment will be described. The information processing according to this embodiment includes a plurality of stages such as the generation stage of the avatar 900, the generation stage of the icon 910, the viewing stage of the avatar 900 / icon 910, the rental stage of the icon 910, and the purchase stage of the product 810. Hereinafter, the information processing according to this embodiment will be described for each stage.

[0092] ~Avatar generation stage~ First, with reference to FIG. 14, the generation stage of the avatar 900 will be described. FIG. 14 is a sequence diagram of the information processing method according to Embodiment 1, and more specifically, a sequence diagram of the generation stage of the avatar 900. As shown in FIG. 14, the generation stage of the avatar 900 includes a plurality of steps from step S201 to step S204. Hereinafter, the details of each step will be described.

[0093] First, the user (seller) 800 uses the user terminal (user terminal A) 100a to perform his / her personal authentication and activate the camera function. Note that the user in the generation stage of the avatar 900 is not limited to the seller, and may be a purchaser.

[0094] Next, the user (seller) 800 uses the user terminal 100a to take a picture of his / her face image (step S201). At this time, although not shown in FIG. 14, the user 800 may input his / her age or the like as a parameter using the user terminal 100a, or the user terminal 100a may acquire the user 800's biological information (heart rate, body temperature, etc.) as a parameter. Then, the user terminal 100a transmits information such as the face image, the parameter, and the personal ID of the user 800 to the server 300a.

[0095] Next, the server 300a acquires feature amounts from the face image and classifies categories (for example, gender, hairstyle, hair color, facial contour, skin color, eyebrow shape, eye shape, eye color, nose shape, mouth shape, freckle position, ear shape, etc.) (step S202). Then, the server 300a generates an avatar 900 corresponding to the user 800 based on the face image, the feature amounts, and the categories (step S203). At this time, the server 300a may adjust the avatar 900 with parameters using the above-described embodiments of the present disclosure. By doing so, the user 800 can obtain an avatar 900 whose design varies in various ways by parameters instead of simply an avatar 900 that closely resembles himself / herself, and thus can obtain new enjoyment (added value).

[0096] Furthermore, the server 300a registers the generated avatar 900 in association with the identification information (personal ID) of the user 800 (step S204).

[0097] ~Generation of Icons~ Next, with reference to FIG. 15, the generation stage of the icon 910 will be described. FIG. 15 is a sequence diagram of the information processing method according to the first embodiment, and more specifically, is a sequence diagram of the generation stage of the icon 910. As shown in FIG. 15, the generation stage of the icon 910 includes a plurality of steps from step S301 to step S305. The details of each step will be described below.

[0098] First, the user (producer) 800 performs his / her own personal authentication using the user terminal (user terminal A) 100a and activates the camera function.

[0099] Next, the user (seller) 800 uses the user terminal 100a to photograph the product 810 (step S301). At this time, although not shown in FIG. 15, the user 800 uses the user terminal 100a to input information such as the number of years of use, purchase price, quality, purchase location, possible purchase destinations of the product 810, brand name, and size of the product 810 as parameters and product information. Then, the user terminal 100a transmits information such as the product image, parameters, product information, and the personal ID of the user 800 to the server 300a.

[0100] Next, the server 300a acquires feature amounts (shape, decoration, pattern, texture, etc.) from the product image, and based on the feature amounts, determines whether the product 810 is the same as the product 810 of the product information linked to the information of other users 800 (step S302). If the product in the acquired image is the product 810 linked to other users 800, the server 300a may copy and use the image of the already generated icon 910 (reuse of the icon 910).

[0101] Next, if the product 810 in the acquired image is not the product 810 associated with another user 800, the server 300a classifies the category of the product 810 (for example, the type such as handbag, shoulder bag, tote bag, material, size, etc.) based on the above feature amounts (step S303). Then, the server 300a generates an icon 910 corresponding to the product 810 based on the product image, feature amounts, and category (step S304). At this time, the server 300a may adjust the icon 910 with parameters using the above-described embodiments of the present disclosure. For example, when information on quality is obtained as a parameter (manual input by the user 800, quality recognition by image analysis, etc.), the generated icon 910 also changes, for example, gloss and color, to generate a more special icon 910. Further, when information indicating that the sales volume of the product 810 is small, the sales opportunity is a limited opportunity such as an event, or the product 810 is designed by a famous person is obtained as a parameter, a more special icon 910 is generated. By doing so, the user 800 can obtain icons 910 whose designs vary depending on parameters instead of simply icons 910 that closely resemble the product 810, and thus can obtain new enjoyment (added value).

[0102] Furthermore, the server 300a registers the generated icon 910 in association with the identification information (personal ID) of the user 800 (step S305). In this embodiment, when registering the icon 910, the server 300a preferably assigns an icon ID to be associated with the icon 910. Further, in this embodiment, the server 300a preferably registers, as product information, the number of years of use, purchase price, quality, purchase location, possible purchase destinations of the product 810, brand name, and size information of the product 810 input from the user 800 in association with the icon 910.

[0103] As described above, in this embodiment, the generated avatar 900 and icon 910 are registered in association with the identification information of the user 800.

[0104] In this embodiment, the user 800 can also take a picture of a product 810 owned by another person, generate an icon 910, and associate the generated icon 910 with his / her own identification information. In this case, it is called "rental of the icon 910". However, in this embodiment, in the case of such "rental of the icon 910", since the actual product 810 is owned by another person, it is assumed that the ownership of the "icon 910" cannot be permanently obtained, and it is preferable to end the ownership (rental) of the "icon 910" within a preset period. For example, in the case of "rental", the user 800 registers a period at the time of registering the icon 910 or the like, and when the registered period has elapsed, the server 300a cancels the association of the rented "icon 910" with the identification information of the renting user 800. Note that in this embodiment, if the above period is not set, the user 800 may permanently own the "icon 910".

[0105] ~Avatar / Icon Viewing~ Next, with reference to FIGS. 16 and 17, the viewing stage of the avatar 900 / icon 910 will be described. FIG. 16 is a sequence diagram of the information processing method according to Embodiment 1, and more specifically, a sequence diagram of the viewing stage of the avatar 900 / icon 910. FIG. 17 is a diagram showing an example of a screen according to Embodiment 1, and more specifically, shows an example of a screen at the viewing stage of the avatar 900 / icon 910.

[0106] As shown in FIG. 16, the viewing stage of the avatar 900 / icon 910 includes a plurality of steps from step S401 to step S404. The details of each step will be described below.

[0107] First, the user terminal (user terminal A) 100a performs personal authentication of the user 800 and transmits the personal ID and personal authentication information of the user 800 to the server 300a (step S401).

[0108] Next, the server 300a searches for the account of the user 800 using the obtained personal ID and the like (step S402). And when the server 300a can acquire the account of the user 800, the server 300a acquires the image of the avatar 900 linked to the user 800 (step S403). Further, when the server 300a can acquire the account of the user 800, the server 300a acquires the image of the icon 910 linked to the user 800 (step S404). Then, the server 300a transmits the acquired images of the avatar 900 and the icon 910 to the user terminal (user terminal A) 100a.

[0109] In this way, via the user terminal a, the user 800 can view the avatar 900 and the icon 910. For example, as shown in FIG. 17, in this embodiment, the avatar 900 can be viewed in a state equipped with the icon 910. Specifically, on the screen shown in FIG. 17, the user 800 can view these icons 910 by scrolling the icons 910 of the owned products 810, and by selecting one icon 910 from these icons 910, the user can view an image in which the avatar 900 is equipped with the selected icon 910.

[0110] Note that the user 800 at the viewing stage of the avatar 900 / icon 910 is not limited to the seller, and may be a purchaser.

[0111] ~Rental of Icons~ Next, with reference to FIGS. 18 and 19, the rental stage of the icon 910 will be described. FIG. 18 is a sequence diagram of the information processing method according to the first embodiment, and specifically, it is a sequence diagram of the rental stage of the icon 910. FIG. 19 is a diagram showing an example of a screen according to the first embodiment, and specifically, it shows an example of a screen at the rental stage of the icon 910.

[0112] As shown in FIG. 18, the rental stage of the icon 910 includes a plurality of steps from step S501 to step S503. The details of each step are described below.

[0113] First, the user terminal (user terminal B) 100b authenticates the user 800 and transmits the personal ID and personal authentication information of the user 800 to the server 300a.

[0114] Next, the user terminal 100b and the server 300a view the icon 910 in the same manner as the viewing stage of the avatar 900 / icon 910 described above (step S501).

[0115] The user 800 views the icons 910 owned by other users 800 via the user terminal 100b and selects the icon 910 to be rented from among them (step 502). Then, the user terminal 100b transmits the information of the selected icon 910 to the server 300a.

[0116] The server 300a registers the selected icon 910 in association with the identification information (personal ID) of the user 800 (step S503).

[0117] For example, as shown in FIG. 19, in this embodiment, when the male user 800 selects one icon 910 from among the plurality of icons 910 owned by the female user 800 shown on the left, the ownership of the selected icon 910 moves to the male user 800 shown on the right, and the "icon 910" is rented.

[0118] In this embodiment, if the user 800 only owns (rents) the "icon 910" even after a predetermined period has elapsed, a prompt to encourage the user 800 to purchase the product 810 corresponding to the "icon 910" may be made (for example, by presenting information on the purchase site of the product 810).

[0119] ~Purchase of Product~ Next, with reference to FIGS. 20 to 23, the purchase stage of product 810 will be described. FIG. 20 is a sequence diagram of the information processing method according to the first embodiment, and more specifically, it is a sequence diagram of the purchase stage of product 810. FIGS. 21 to 23 are diagrams showing an example of the screen according to the first embodiment, and more specifically, they show an example of the screen at the purchase stage of product 810. In this embodiment, the purchase of product 810 means actually purchasing product 810 corresponding to the icon 910 selected by user 800.

[0120] As shown in FIG. 20, the purchase stage of product 810 includes a plurality of steps from step S601 to step S604. The details of each step will be described below.

[0121] First, user terminal (user terminal B) 100b performs personal authentication of user (purchaser) 800 and transmits the personal ID and personal authentication information of user 800 to server 300a.

[0122] Next, user terminal 100b and server 300a view icon 910 in the same manner as in the viewing stage of avatar 900 / icon 910 described above (step S601).

[0123] User 800 views the icons 910 owned by other users 800 via user terminal 100b and selects the icon 910 that he / she wants to purchase from among them (step 602). Then, user terminal 100b transmits the information of the selected icon 910 to server 300a.

[0124] Next, user terminal 100b and server 300a connect to the purchase site corresponding to product 810 of the selected icon 910 and execute the purchase and settlement of product 810 (step S603).

[0125] Then, the server 300a registers an icon 910 corresponding to the product 810 purchased by the user 800 in association with the identification information (personal ID) of the user 800 as the owner of the product 810 (step S604). That is, the user 800 becomes the permanent owner of the icon 910. Then, the server 300a transmits a payment completion notice or the like to the user terminal (user terminal B) 100b.

[0126] For example, as shown in FIG. 21, in this embodiment, the user (purchaser) 800 browses the icon 910 that the user has already rented via the user terminal (user terminal B) 100b, and selects one icon 910 from a plurality of icons 910 rented by people. Then, in this embodiment, by selecting the icon 910, it is possible to view the image and product information of the product 810 corresponding to the icon 910, or to perform a purchase procedure. Further, in this embodiment, when the product 810 is purchased, it is preferable to notify that the user has become the permanent owner of the icon 910 corresponding to the product 810. In this way, in this embodiment, by renting the icon 910 and making the user 800 become attached to the icon 910, it is possible to stimulate the user 800's desire to purchase the actual product 810 corresponding to the icon 910.

[0127] Also, in this embodiment, the user may purchase the product 810 without renting the icon 910. For example, as shown in FIG. 22, in this embodiment, the user (purchaser) 800 browses an EC site via the user terminal (user terminal B) 100b and purchases the product 810. And in this embodiment, when the purchase process is completed, the user 800 becomes the permanent owner of the icon 910 corresponding to the purchased product 810 while being the owner of the purchased product 810. Therefore, also in the example of FIG. 22, it is preferable to notify that the user has become the permanent owner of the icon 910 corresponding to the product 810. Also, in the example of FIG. 22, the product 810 is not limited to being a used product and may be a new product. Further, in this embodiment, when the product 810 is purchased by the user (purchaser) 800, the association between the icon 910 of the product 810 and the user (seller) 800 may be canceled (that is, the user (seller) 800 can no longer confirm the icon 910), or the association between the icon 910 of the product 810 and the user (seller) 800 may be maintained (that is, the user (seller) 800 can continue to confirm the icon 910). Also, when the association between the icon 910 of the product 810 and the user (seller) 800 is maintained, the product 810 may be sold at a discounted price corresponding to the maintained association. Further, although the case of purchasing the product 810 has been described, this is not limiting in this embodiment, and only the icon 910 may be purchased without purchasing the product 810. Also in this case, the association between the icon 910 and the user (seller) 800 may be canceled (that is, the user (seller) 800 can no longer confirm the icon 910), or the association between the icon 910 and the user (seller) 800 may be maintained (that is, the user (seller) 800 can continue to confirm the icon 910).

[0128] Furthermore, in this embodiment, even for the same product 810, the icons 910 may be different. For example, as shown in FIG. 23, the icon 910 of the product 810 with good quality shall have a different design from the icons 910 of other same products 810. Therefore, due to the different designs, the user (purchaser) 800 can easily select the icon 910 of the product 810 with good quality and different designs from among the multiple icons 910 of the same product 810. As a result, in the example of FIG. 23, the product 810 with good quality can be selected without checking the product information of all the icons 910. That is, according to the example of FIG. 23, with the icon 910 according to this embodiment, the user 800 can easily recognize the quality of the product 810 corresponding to the icon 910.

[0129] (Modification example) Also, in this embodiment, the design of the avatar 900 may be changed in correspondence with the time, location, and ownership icons 910. For example, time information may be acquired from the user terminal 100a, and the expression and pose of the avatar 900 may be changed according to the acquired time information (for example, morning, noon, night). Also, for example, location information may be acquired from the user terminal 100a, and the expression and pose of the avatar 900 may be changed according to the acquired location information (for example, park, sea, shopping center, etc.). Furthermore, in this embodiment, the expression and pose of the avatar 900 may be changed according to the type of the icon 910 and the time when the icon 910 is equipped.

[0130] Furthermore, in this embodiment, coordinates of the icon 910 equipped on the avatar 900 may be recommended corresponding to the time, location, and possession icons 910. For example, as shown in FIG. 24 which is an explanatory diagram for explaining a configuration example of a database stored in the storage unit 400a, it is assumed that the storage unit 400a stores information on the time and location when the user 800 equipped the avatar 900 with the icon 910. More specifically, according to the data shown in FIG. 24, for example, the icon 910 of bag 1 is often equipped on the avatar 900 when the user 800 is in the office, and it can be seen that the icon 910 of bag 2 was equipped on the avatar 900 when the user 800 was outdoors at a café.

[0131] Therefore, when the server 300a can recognize from the location information that the user 800 is in the office, the server 300a presents information to the user 800 recommending equipping the avatar 900 with the icon 910 of bag 1. Also, for example, when the server 300a can recognize from the location information that the user 800 is at a café around noon, the server 300a presents information to the user 800 recommending equipping the icon 910 of bag 1 or bag 2 based on the following probabilities. Specifically, referring to FIG. 24, it can be seen that in the past, when the user 800 was at a café at the same time zone, the avatar 900 equipped the icon 910 of bag 1 twice and the icon 910 of bag 2 five times. Therefore, the server 300a recommends the icon 910 of bag 1 with a probability of 2 / 7 and the icon 910 of bag 2 with a probability of 5 / 7.

[0132] <3.2 Embodiment 2> Also, the embodiment of the present disclosure can be applied to a game or the like in which the avatar 900 operates in a virtual space. With reference to FIG. 25, such an embodiment 2 will be described. FIG. 25 is an explanatory diagram for explaining Embodiment 2.

[0133] As shown in FIG. 25, on the virtual space 700, an avatar 900 that operates as the user 800's alter ego and an icon 910 of an item equipped on the avatar 900 can be generated by applying the embodiments of the present disclosure. Specifically, in this embodiment, based on the parameters of the user 800's biometric information (heartbeat, body temperature, etc.), an avatar 900 with an expression corresponding to the biometric information (such as having a red face when excited) can be generated. By doing so, in this embodiment, not simply an avatar 900 similar to the user 800, but an avatar 900 with a design corresponding to the user 800's biometric information can be obtained. As a result, according to this embodiment, every time the user 800's biometric information changes, a corresponding avatar 900 can be obtained, so that the user 800 can feel the avatar 900 more as his or her own alter ego and feel that the real world and the virtual space of the game are linked, thus adding value to the game. Also, in this embodiment, since an avatar 900 with various designs can be obtained according to the user 800's biometric information, the variations and surprises of the avatar 900 can be enjoyed.

[0134] Furthermore, in this embodiment, based on the parameters of the user 800's location information, icons 910 with various design variations can be generated as the location information changes. As a result, according to this embodiment, the user 800 can obtain icons 910 with various design variations, feel the pleasure of collecting the icons 910, or enjoy changing the clothes of the avatar 900 using these multiple icons 910.

[0135] Recently, as remote work has been rapidly spreading, the number of companies conducting meetings and consultations that used to be in-person online has been increasing. For example, it is assumed that participants' "avatars 900" are gathered in a "meeting room" set up in a virtual space. At this time, by using the embodiments of the present disclosure, the avatar 900 can use the sensing data of the motion sensor worn by the user 800 who is the participant to make the corresponding avatar 900 operate following the movements of the user 800. Specifically, the above-described latent variables are adjusted based on the sensing data of the motion sensor, and the avatar 900 is generated using the adjusted latent variables. By doing so, for example, since movements such as the waving of the hand of the participant (user) 800 are expressed by the avatar 900, the intention of the participant becomes easier to convey.

[0136] <3.3 Example 3> In recent years, as an additional information in the real world, a technology of augmented reality (AR) that superimposes virtual objects on the real space and presents them to the user 800 has come to be used. Therefore, the case where the embodiments of the present disclosure are applied to AR will be described with reference to FIG. 26. FIG. 26 is an explanatory diagram for explaining Example 3.

[0137] For example, in this embodiment, the illuminance acquired by the illuminometer mounted on the user terminal 100c is used as a parameter to generate the icon 910 of the shirt. By using the embodiment of the present disclosure, the icon 910 of the shirt is not only very similar to the original actual shirt, but also generated to have a texture corresponding to the illuminance acquired by the illuminometer mounted on the user terminal 100. Then, as shown in FIG. 26, the user terminal 100c superimposes the generated icon 910 on the image of the user 800 and displays it. By doing so, by displaying the icon 910 having a texture corresponding to the illuminance of the environment where the user 800 actually exists, the user 800 can obtain an image of actually wearing the shirt even if the user 800 does not actually have the shirt corresponding to the icon 910 at hand. As a result, in this embodiment, the user 800 can perform coordination and the like without having the shirt at hand.

[0138] <<4. Summary>> As described above, according to the embodiment of the present disclosure described above, instead of generating a virtual object using the latent variable obtained based on the image, the latent variable is adjusted based on a parameter, and a virtual object is generated using the adjusted latent variable. By doing so, in this embodiment, it is possible to obtain not simply a virtual object similar to the actual object, but a virtual object in which a certain feature is emphasized based on various parameters. As a result, according to this embodiment, it is possible to obtain virtual objects of various variations and give added value to the generated virtual objects.

[0139] Also, in the above-described embodiment, the user terminal 100 according to this embodiment may be made to function as a server 300, so that the user terminal 100 may be a stand-alone device. Alternatively, in the above-described embodiment, a part of the functions of the server 300 may be made to be performed by the user terminal 100 according to this embodiment.

[0140] <<5. Hardware Configuration>> The information processing apparatus such as the server 300 according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in FIG. 27, for example. Hereinafter, the server 300 of the embodiment of the present disclosure will be described as an example. FIG. 27 is a hardware configuration diagram showing an example of a computer 1000 that realizes functions such as the server 300. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

[0141] The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each part. For example, the CPU 1100 expands a program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0142] The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started up, and a program dependent on the hardware of the computer 1000.

[0143] The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by such a program. Specifically, the HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0144] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from other devices or transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0145] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input / output device 1650 such as a keyboard, a mouse, or a microphone (mic) via the input / output interface 1600. Also, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media). The media is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory or the like.

[0146] For example, when the computer 1000 functions as the server 300 according to the embodiment of the present disclosure, the CPU 1100 of the computer 1000 realizes the functions of the processing unit 310 and the like by executing the program stored in the RAM 1200. Also, the information processing program and the like according to the present disclosure are stored in the HDD 1400. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be acquired from other devices via the external network 1550.

[0147] Further, the information processing apparatus according to the present embodiment may be applied to a system including a plurality of devices on the premise of connection to a network (or communication between devices) such as cloud computing or the like. That is, the information processing apparatus according to the present embodiment described above can also be realized as an information processing system according to the present embodiment by, for example, a plurality of devices.

[0148] An example of the hardware configuration of the server 300 and the like has been shown above. Each of the above-described components may be configured using general-purpose members, or may be configured using hardware specialized for the functions of each component. Such a configuration can be appropriately changed according to the technical level at the time of implementation.

[0149] <<6. Supplementary Explanation>> Note that the embodiments of the present disclosure described above may include, for example, a program for causing a computer to function as the information processing apparatus according to the present embodiment, and a non-transitory tangible medium on which the program is recorded. Further, the program may be distributed via a communication line such as the Internet (including wireless communication).

[0150] Also, each step in the processing of each of the above-described embodiments does not necessarily have to be processed in the order described. For example, each step may be processed with the order appropriately changed. Also, each step may be processed partially in parallel or individually instead of being processed in time series. Furthermore, the processing method of each step does not necessarily have to be processed according to the method described, and for example, it may be processed by another method by another functional unit.

[0151] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0152] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure can exhibit other effects that are obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0153] Note that the present technology can also adopt the following configuration. (1) A data acquisition unit that acquires sensing data of a real object, A parameter acquisition unit that acquires predetermined parameters, An object generation unit that extracts feature amounts from the sensing data and generates a virtual object based on the feature amounts, and is provided with, The object generation unit adjusts the virtual object according to the predetermined parameters. An information processing apparatus. (2) The parameter acquisition unit acquires at least one of position parameters, time parameters, attribute information parameters, and environmental parameters related to the real object, the virtual object, or the user as the predetermined parameters. The information processing apparatus according to (1) above. (3) The object generation unit, An encoding unit that extracts the feature amounts from the sensing data and convolves them into latent variables having a predetermined dimension, An adjustment unit that adjusts the latent variables based on the predetermined parameters, A decoding unit that generates the virtual object based on the adjusted latent variables, and has, The information processing apparatus according to (1) or (2) above. (4) The adjustment unit changes the latent variables by an amount of change determined based on the predetermined parameters. The information processing apparatus according to (3) above. (5) The object generation unit treats the latent variables as variables that follow a predetermined probability distribution. The information processing apparatus according to (4) above. (6) The object generation unit, further has a category acquisition unit that acquires the category of the real object, The category acquisition unit inputs the acquired category as a condition to the encoding unit and the decoding unit. The information processing apparatus according to (5) above. (7) The encoding unit and the decoding unit are neural networks that have previously performed machine learning on images of a plurality of other real objects belonging to the same category as the real object and other virtual objects corresponding to the plurality of other real objects, and the information processing apparatus according to (6) above. (8) The data acquisition unit acquires the sensing data from an imaging device or a depth sensor, and the information processing apparatus according to any one of (1) to (7) above. (9) The real object is a person, and the information processing apparatus according to any one of (1) to (8) above. (10) The virtual object is an avatar, and the information processing apparatus according to (9) above. (11) The real object is an article, and the information processing apparatus according to (10) above. (12) The virtual object is an icon, and the information processing apparatus according to (11) above. (13) The information processing apparatus according to (12) above further includes a management unit that associates and manages the icon with a user who owns or borrows the real object. (14) The management unit associates and manages the icon with the avatar based on the user, and the information processing apparatus according to (13) above. (15) The information processing apparatus according to any one of (1) to (14) above further includes an output unit that outputs the virtual object. (16) The output unit causes a display device to display the virtual object in a virtual space, and the information processing apparatus according to (15) above. (17) The output unit causes a display device to display the virtual object so as to be superimposed on a real space image, and the information processing apparatus according to (15) above. (18) The output unit is the information processing apparatus according to the above (15), which causes a display device to display the virtual object so as to be superimposed on the real space. (19) An information processing apparatus: acquires sensing data of a real object, acquires predetermined parameters, extracts a feature amount from the sensing data, and generates a virtual object based on the feature amount, including: when generating the virtual object, adjusting the virtual object according to the predetermined parameters. An information processing method. (20) A program for causing a computer to: have a function of acquiring sensing data of a real object, have a function of acquiring predetermined parameters, have a function of extracting a feature amount from the sensing data and generating a virtual object based on the feature amount, wherein when generating the virtual object, the virtual object is adjusted according to the predetermined parameters. A program. (21)(21) An information processing apparatus comprising: a data acquisition unit that acquires an image of a product, a parameter acquisition unit that acquires parameters of the product, an object generation unit that extracts a feature amount from the image and generates an icon based on the feature amount, wherein the object generation unit adjusts the icon according to the parameters. (22)(22) In the information processing apparatus according to the above (21), the parameters of the product are the quality of the product. (23) In the information processing apparatus according to the above (21) or (22), the product is a used product in secondary distribution.

Description of Reference Numerals

[0154] 10 Information Processing System 100, 100a, 100b, 100c User Terminals 102 Sensor Unit 104 Input Unit 106 Display Unit 110 Control Unit 112, 410 Communication Unit 300, 300a Servers 310, 310a Processing Unit 320 Sensing Data Acquisition Unit 330 Parameter Acquisition Unit 340 Object Generation Unit 342 Encoding Unit 344 Decoding Unit 346 Adjustment Unit 348 Category Acquisition Unit 350 Output Unit 360 Management Unit 370 Authentication Unit 380 Recommendation Unit 390 Settlement Unit 400, 400a Storage Unit 402 Personal Information DB 404 Product Information DB 500 Communication Network 700 Virtual Space 800 User 810 Articles (Products) 900 Avatar 910 Icon

Claims

1. A data acquisition unit that acquires sensing data of a real object, A parameter acquisition unit that acquires predetermined parameters, An object generation unit that extracts feature amounts from the sensing data and generates a virtual object based on the feature amounts, Comprising, The parameter acquisition unit, Attribute information parameters regarding the real object, and At least one of a position parameter, a time parameter, and an environment parameter regarding the user, Of these, at least one is Acquired as the predetermined parameter, The object generation unit, An encoding unit that extracts the feature amounts from the sensing data and convolves them into latent variables having a predetermined dimension, An adjustment unit that adjusts the latent variables by changing the latent variables by an amount of change determined based on the predetermined parameters, A decoding unit that generates the virtual object based on the adjusted latent variables, Having, Adjust the virtual object according to the predetermined parameters, An information processing apparatus.

2. The object generation unit treats the latent variables as variables that follow a predetermined probability distribution. The information processing apparatus according to claim 1.

3. The object generation unit, Further includes a category acquisition unit that acquires the category of the real object, The category acquisition unit inputs the acquired category as a condition to the encoding unit and the decoding unit. The information processing apparatus according to claim 2.

4. The encoding unit and the decoding unit are neural networks that have previously learned a plurality of images of other real objects belonging to the same category as the category to which the real object belongs and other virtual objects corresponding to the plurality of other real objects. The information processing apparatus according to claim 3.

5. The data acquisition unit acquires the sensing data from an imaging device or a depth sensor. The information processing apparatus according to any one of claims 1 to 4.

6. The real object is a person who is the user, The virtual object is an avatar, The adjustment unit adjusts the latent variables by changing the latent variables by an amount of change determined based on the time parameter regarding the user, The decoding unit generates the avatar based on the adjusted latent variables. The information processing apparatus according to any one of claims 1 to 5.

7. The real object includes articles, The virtual object includes icons, The adjustment unit adjusts the latent variable by changing the latent variable by an amount determined based on the position parameter, the environmental parameter, or the attribute information parameter of the article, related to the user, The decoding unit generates the icon based on the adjusted latent variable, The information processing apparatus according to claim 6.

8. The attribute information parameter of the article includes a parameter of the quality of the article, The information processing apparatus according to claim 7.

9. The information processing apparatus according to claim 8, wherein the article is a used article in secondary distribution.

10. The information processing apparatus according to any one of claims 7 to 9, further comprising a management unit that associates and manages the icon with the user who owns or borrows the article.

11. The information processing apparatus according to claim 10, wherein the management unit manages the icon by associating it with the avatar based on the user.

12. The information processing apparatus according to any one of claims 1 to 11, further comprising an output unit that outputs the virtual object.

13. The information processing apparatus according to claim 12, wherein the output unit causes a display device to display the virtual object on a virtual space.

14. The information processing apparatus according to claim 12, wherein the output unit causes a display device to display the virtual object so as to be superimposed on a real space image.

15. The information processing apparatus according to claim 12, wherein the output unit causes a display device to display the virtual object so as to be superimposed on the real space.

16. By an information processing apparatus, Obtain sensing data of a real object, Obtain predetermined parameters, Extract a feature amount from the sensing data, and generate a virtual object based on the feature amount, Including, In the acquisition of the predetermined parameters, An attribute information parameter related to the real object, and At least one of a position parameter, a time parameter, and an environmental parameter related to the user, Of these, Obtain as the predetermined parameter, In the generation of the virtual object, Extract the feature amount from the sensing data, and convolve it with a latent variable having a predetermined dimension, Adjust the latent variable by changing the latent variable by an amount determined based on the predetermined parameter, Generate the virtual object based on the adjusted latent variable, An information processing method.

17. On a computer, A function of acquiring sensing data of a real object, A function of acquiring a predetermined parameter, A function of extracting a feature amount from the sensing data and generating a virtual object based on the feature amount, A program for realizing, The function of acquiring the predetermined parameter is An attribute information parameter regarding the real object, and At least one of a position parameter, a time parameter, and an environment parameter regarding the user, Of these, Acquire as the predetermined parameter, The function of generating the virtual object is Extract the feature amount from the sensing data, convolve it with a latent variable having a predetermined dimension, Adjust the latent variable by changing the latent variable by an amount determined based on the predetermined parameter, Generate the virtual object based on the adjusted latent variable, A program.

Images