Information processing device and information processing method

Abstract

An information processing device according to one embodiment of the present invention comprises: an access means for accessing a database in which at least one of a content element and a service element is stored; an acquisition means for acquiring a variable that indicates the utilization degree of whole-body data related to the whole body of a user; a determination means for determining, in accordance with the variable, a method for personalizing at least one of the content element and the service element; and a provision means for providing, to the user, a combination of the personalized content element and service element.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present invention relates to a technology for promoting the use of whole-body data.

[0002] Technologies for using data managed by a server on a terminal are known. For example, Patent Document 1 discloses a technology for transmitting, to an information processing apparatus as a terminal, data for outputting a 3D model composed of specific elements among 3D modeling data managed by a server.

[0003] Japanese Patent No. 6799883

[0004] In the technology described in Patent Document 1, there was room for improvement from the viewpoint of promoting the use of content.

[0005] The present disclosure provides a technology for promoting the use of content.

[0006] One aspect of the present disclosure provides an information processing apparatus including: access means for accessing a database storing at least one of content elements and service elements; acquisition means for acquiring a variable indicating the degree of utilization of whole-body data regarding a user's whole body; determination means for determining a method of personalizing at least one of content elements and service elements according to the variable; and provision means for providing the user with a combination of the personalized content elements and service elements.

[0007] The whole-body data may be 3D model data indicating a 3D model of the user, and the variable may indicate the degree of coincidence between the state of the 3D model at the time of generating the 3D model data and a reference state.

[0008] The state may include at least one of the expression, pose, and motion of the 3D model.

[0009] The state may be the state of another user who generated a 3D model together with the user.

[0010] The whole-body data may be 3D model data indicating a 3D model of the user, and the variable may indicate the situation in which the user used the 3D model data by himself / herself after distributing the 3D model data.

[0011] The user's use of their own 3D model data may include the generation of new content by combining the 3D model data with other content elements.

[0012] The system may have a modification means for changing the 3D model or other content elements when generating new content, in accordance with the user's preferences determined from the aforementioned combinations.

[0013] The full-body data is 3D model data representing the user's 3D model, and the variables may indicate interactions performed by users other than the user with content using the 3D model data after the 3D model data has been distributed.

[0014] The interaction may be an expression of support for the content, and the variable may indicate the number of expressions of support for the content.

[0015] The system may include a calculation means for calculating the degree of utilization of the user's whole-body data based on the time-series changes in the user's whole-body data.

[0016] The system may include a calculation means for calculating the utilization rate based on a comparison between the full-body data of the aforementioned user and the full-body data of another user.

[0017] The system may include a presentation means for presenting requests regarding the whole-body data to the user, and a calculation means for calculating the utilization level based on the extent to which the user has responded to the requests.

[0018] The system may have assignment means for assigning personalized content elements to the user according to probability.

[0019] The aforementioned content elements may include characters from IP content.

[0020] The aforementioned probability may be determined according to the aforementioned variable.

[0021] The aforementioned probability may be determined according to the user's preferences.

[0022] The aforementioned content element may include motion or poses that the 3D model is instructed to perform.

[0023] The system may include means for granting the user the right to use a combination of a 3D model and motion or pose.

[0024] The aforementioned service element may include awarding points to the user.

[0025] One aspect of the present disclosure provides an information processing device comprising the steps of: accessing a database storing at least one of content elements and service elements; obtaining a variable indicating the degree of utilization of whole-body data relating to a user's whole body; determining a method for personalizing at least one of the content elements and service elements according to the variable; and providing the personalized combination of content elements and service elements to the user.

[0026] This disclosure will promote the use of the content.

[0027] A diagram illustrating an overview of an information processing system 1 according to one embodiment. A diagram illustrating the functional configuration of the information processing system 1. A diagram illustrating the hardware configuration of the server 10. A diagram illustrating the software configuration of the information processing system 1. A sequence chart illustrating the generation and update process of whole-body data in the information processing system 1. A diagram illustrating a 3D database 512. A flowchart illustrating the generation process of personalized content. A diagram illustrating a table 61 that records information related to content personalization.

[0028] 1. Diagram 1 illustrates an overview of an information processing system 1 according to one embodiment. The information processing system 1 includes a server 10, a user terminal 20, a server 30, and a scanner 40. The scanner 40 is a device that performs imaging or measurement to obtain whole-body data such as a 3D model. The user terminal 20 is a terminal device used by end users to access the information processing system 1. The server 10 manages the whole-body data. The server 30 provides applications or services that utilize the whole-body data. The information processing system 1 measures the degree to which each user utilizes the whole-body data and provides personalized (or customized) content (hereinafter referred to as "personalized content") to that user according to that degree of utilization. Personalized content is an example of an incentive given to a user according to the degree to which they utilize the whole-body data. These devices communicate via a network 9. The network 9 is a computer network such as the Internet.

[0029] Figure 2 is a diagram illustrating the functional configuration of the information processing system 1. The information processing system 1 includes a storage means 11, an access means 12, an acquisition means 13, a determination means 14, a providing means 15, and a control means 19. In this example, these functional elements are implemented in a server 10. The storage means 11 stores various types of data and programs. In this example, the data stored by the storage means 11 includes a database (details described later) that stores at least one of content elements and service elements. The access means 12 accesses this database. The acquisition means 13 acquires a variable indicating the degree of utilization of the user's full-body data. Hereinafter, this variable will simply be referred to as "utilization degree". The determination means 14 determines a method for personalizing at least one of the content elements and service elements according to the utilization degree. The providing means 15 provides the user with a combination of personalized content elements and service elements. The control means 19 performs various controls.

[0030] Here, whole-body data refers to data relating to the user's entire body, and is data obtained from measurements of the user's entire body. Whole-body data includes at least one of the user's 3D model data and vital data. 3D model data is data that represents a 3D model in a virtual space on a computer. The 3D model is used to generate an image of the 3D model viewed from a viewpoint specified by the user, i.e., a free viewpoint. The user's 3D model is sometimes called an avatar. Vital data is information about the subject's biological activity and includes, for example, weight, body temperature, heart rate, blood pressure, respiratory rate, oxygen saturation, and at least one of the following: body fat percentage.

[0031] The information processing system 1 further includes a presentation means 151, a calculation means 152, an assignment means 153, a granting means 154, and a modification means 155. The presentation means 151 presents the user with requests regarding full-body data. The calculation means 152 calculates the degree of utilization based on the usage history of the full-body data or the circumstances under which the full-body data is generated. The assignment means 153 assigns personalized content elements to the user according to probability. This is an example of a process that provides an incentive to the user. The granting means 154 grants the user the right to use a combination of a 3D model and motion or pose. This is also an example of a process that provides an incentive to the user. The modification means 155 modifies the 3D model or other content elements when generating new content according to the user's preferences.

[0032] Figure 3 illustrates the hardware configuration of server 10. Server 10 is a computer device having a CPU 101, memory 102, storage 103, and communication interface 104. The CPU 101 is a processing unit that performs various processes according to a program. The memory 102 is a main memory that functions as a work area when the CPU 101 executes a program. The storage 103 is a non-volatile auxiliary storage device that stores various data and programs. The communication interface 104 is a device that communicates with other information processing devices according to a predetermined communication standard (e.g., Ethernet®).

[0033] Although not shown in the diagram, the user terminal 20 is a computer device having a CPU, memory, storage, communication interface, input device (e.g., touchscreen or microphone), and output device (e.g., display or speaker), such as a smartphone, tablet terminal, or personal computer. The server 30 is a computer device having a CPU, memory, storage, and communication interface.

[0034] Although not shown in the diagram, the scanner 40 is a device that scans, or measures, the user who is the subject of whole-body data, and comprises a scanning room (or booth), a group of sensors, and a computer device. The scanning room is a space in which the user performs measurements to generate whole-body data, and in one example comprises a frame and wall materials. The group of sensors includes, for example, a camera, a depth sensor (or rangefinder), a weighing scale, a body fat analyzer, and a thermometer. These sensors are installed in the scanning room. The computer device controls the group of sensors and collects measurement data from them. This computer device transmits the collected measurement data to the server 10. The server 10 generates whole-body data from the measurement data, or records the measurement data as whole-body data.

[0035] Figure 4 illustrates the software configuration of the information processing system 1. The information processing system 1 has a data layer 51, a decision layer 52, and a content layer 53. The data layer 51 has an element database. The element database is a database in which element data is recorded. The element data is data generated in each of the multiple application services that utilize the information processing system 1. In one example, the element database includes a 3D database 512, a vital database 513, and a photo database 514. The 3D database 512, the vital database 513, and the IP database 532 are all examples of databases that store content elements.

[0036] The 3D database 512 is a database in which data related to 3D models is recorded. The data related to 3D models includes 3D model data and associated data. In one example, the 3D model data includes geometry data, material data, rigging data, animation data, and physical data. Geometry data is data related to the shape of the 3D model and includes, for example, mesh data and vertex data. Mesh data is data that defines the shape of the 3D model and is a collection of vertices, edges, and polygons. Vertex data includes the position coordinates and normal vectors of each vertex and is used for accurate representation of the shape and calculation of light reflection. Material data is data related to the appearance of the 3D model other than its shape and includes, for example, textures, shaders, and UV mapping. Textures are 2D image data that represent the appearance of the surface of the 3D model and represent the color, pattern, and detailed surface appearance of the 3D model. Textures include, for example, diffuse (i.e., base color), bump maps (i.e., bumps and depressions), and specular maps (i.e., the degree of light reflection). Shaders define how the surface of the 3D model receives light and shadow, and how texture and transparency are represented. UV mapping is a coordinate system for applying textures to 3D models, defining how 2D images are unfolded onto 3D models. Rigging data is data related to the skeleton that moves the 3D model, including, for example, the skeleton and weights. The skeleton represents the structure that gives movement to the 3D model. In the case of 3D models of living beings such as humans or animals, or robots that mimic them, joints and skeletons are defined, thereby controlling the movement of the 3D model. Weights are information that defines how much each vertex follows the movement of the skeleton. Weights define, for example, how the surrounding skin and clothing mesh moves when an arm is bent.

[0037] Animation data is data that represents the movement of a 3D model. Animation data includes, for example, keyframe animation and motion capture data. Keyframe animation is data that represents movement by saving the posture and shape of a 3D model at specific points in time and interpolating between them. Keyframe animation includes information such as the angle of each joint and the deformation of the model. Motion capture data is data that captures the movement of an actual human or animal and applies that data to a model. Motion capture data enables realistic animation based on actual movement.

[0038] Physical data is data that describes the physical properties of a 3D model in a virtual space. Physical data includes, for example, collision data, physical simulation data, and LOD data. Collision data defines how a 3D model comes into contact with other objects. Collision data defines how objects react when they collide with each other in a game or simulation. Physical simulation data is data used to reproduce realistic movement for things like cloth, hair, or clothing. Physical simulation data defines, for example, the effect of hair or clothing swaying naturally when a 3D model moves. LOD data is simplified model data used to reduce the load when the 3D model is far away or at a small display size. A model with fewer polygons is used at long distances, and a high-precision model is used at close distances.

[0039] The vital database 513 is a database in which the user's vital data is recorded. The photo database 514 is a database in which photographs, i.e., 2D images, are recorded. The photographs are, for example, photographs taken by a so-called photo sticker machine or photo booth. These photographs may include not only those in which the subject is the user (i.e., a person), but also those in which the subject is a landscape, building, or object.

[0040] The 3D database 512, the vital database 513, and the photo database 514 may each include the user ID and user attribute information of the user who is the source of the subject or vital data. User attribute information is information that indicates the user's attributes, and includes, for example, name, date of birth, gender, place of residence, or medical history. Furthermore, the 3D database 512, the vital database 513, and the photo database 514 each include at least one of so-called one-shot data at a certain point in time and time-series data acquired continuously over a certain period. In addition, the one-shot data and time-series data include at least one of individual data (or absolute values) for individual users and statistical data between users or within users.

[0041] The decision layer 52 makes various decisions by referring to data in the database in response to requests from the user terminal 20 or other software elements of the information processing system 1.

[0042] The content layer 53 has a content database. The content database includes, for example, a user database 531, an IP database 532, and an advertising database 533. The user database 531 is a database in which user-generated content is recorded. User-generated content is content generated by end users. Content refers to data generated by combining data recorded in databases in the data layer 51 or content layer 53, such as video data, still image data, or 3D model data. The IP database 532 is a database in which content elements (i.e., IP content elements) for which copyright licenses have been set in the information processing system 1 are recorded. Content elements in the IP database 532 include, for example, characters from manga, movies, novels, or animations, music, paintings (digital copies thereof), landscape photographs, photographs of celebrities, and story progressions. The advertising database 533 is a database in which data related to advertisements is recorded. Data related to advertisements includes, for example, 3D model data of the object of the advertisement, image data, music data, and data on how content elements are combined.

[0043] The content layer 53 further includes a service database 534. The service database 534 is a database in which information regarding service provision is recorded. Service information includes, for example, points awarded, user rank, and usage history in services provided in connection with the information processing system 1. The service database 534 is an example of a database that stores service elements.

[0044] In this example, these software elements are implemented in server 10. Storage 103 stores a program (hereinafter referred to as the "server program") for causing a computer to function as server 10 of information processing system 1. By executing the server program, CPU 101 causes the computer to implement software elements shown in FIG. 4 from the perspective of software and functional elements shown in FIG. 2 from the perspective of functions. Regarding the relationship between functional elements and hardware elements, in a state where CPU 101 is executing the server program, at least one of memory 102 and storage 103 is an example of storage means 11, and CPU 101 is an example of access means 12, acquisition means 13, determination means 14, provision means 15, and control means 19. Regarding the relationship between functional elements and software elements, data layer 51 and content layer 53 are examples of storage means 11, and judgment layer 52 is an example of access means 12, acquisition means 13, determination means 14, provision means 15, presentation means 151, calculation means 152, allocation means 153, assignment means 154, change means 155, and control means 19.

[0045] 2. Operations Next, the operations of information processing system 1 will be described. The operations of information processing system 1 are divided into generation or update of whole body data (hereinafter referred to as "generation / update"), utilization of whole body data, and generation of personalized content. These processes will be described in order below.

[0046] 2.1 Generation / Update of Whole Body Data FIG. 5 is a sequence chart illustrating the generation / update process of whole body data in information processing system 1. In one example, information processing system 1 requests user registration from a user. A user who intends to use the information processing system registers login information and attribute information. The login information is information used for logging in to information processing system 1 and includes, for example, a username and password. The attribute information is information indicating the attributes of the user and includes, for example, name, gender, and age. Information processing system 1, that is, server 10, requests login information from the accessed user.

[0047] Hereinafter, an example will be described in which a user who has already registered performs shooting with the scanner 40 in order to newly generate his / her whole-body data or update the whole-body data that has already been generated. The user accesses the information processing system 1, that is, the server 10 via the scanner 40 or via the user terminal 20.

[0048] In step S101, the scanner 40 performs shooting or measurement of the subject for generating whole-body data. In one example, the scanner 40 has a plurality of service menus. The user selects the service he / she desires from these menus. The plurality of service menus correspond to, for example, which data among the plurality of types of whole-body data is to be generated or updated. Specifically, one service menu is an item for generating or updating all types of whole-body data, and another service menu is an item for generating or updating only the 3D model. The scanner 40 controls the sensor group according to the user's service menu selection and acquires measurement data. The scanner 40 transmits a generation request for whole-body data to the server 10. This generation request includes the identification information of the selected menu item and the acquired measurement data.

[0049] In step S102, the server 10 generates whole-body data from the measurement data. In one example, the server 10 generates whole-body data according to the generation request received from the scanner 40. For example, regarding the 3D model, an algorithm for generating a 3D model from image data and distance data is implemented in the server 10, and the server 10 generates a 3D model according to this algorithm.

[0050] In step S103, the server 10 measures the utilization degree of the whole-body data. This step is optional and may be skipped depending on the selected menu item. That is, this step is executed only when a specific menu item is selected. Also, the measurement of the utilization degree of the whole-body data is not limited to this timing, that is, associated with the generation / updating of the whole-body data. Here, as the utilization degree of the whole-body data associated with the generation / updating of the whole-body data, the utilization degree regarding the pose or motion at the time of shooting will be described.

[0051] In this example, server 10 requests a user who wants to generate or update full-body data to assume a specific pose (hereinafter referred to as the "specified pose"). The data for the specified pose is recorded in, for example, the IP database 532 and is data for poses taken by animated characters. The user, who is the subject, is photographed while assuming the specified pose inside the scanner 40. Since the data captured in this state is used, the 3D model data generated in step S102 is a 3D model of the user in the state of assuming the specified pose. Server 10 compares the pose of the generated 3D model with the specified pose and calculates the degree of agreement between the two according to a predetermined algorithm. In one example, the degree of agreement between poses is evaluated from multiple perspectives. These multiple perspectives include, for example, the relative positional relationships of bones and the positional relationships between specific bones and specific coordinate axes. The assumed positional relationships of bones are further subdivided and include multiple sets of relative positional relationships between one bone and another. In one example, a weight is set for each set of bones. A specific set of bones is given a larger weight than other sets of bones. A set of weights is set for each pose.

[0052] Furthermore, server 10 converts the calculated degree of agreement into a utilization score. The formula for converting the degree of agreement to a utilization score is defined in server 10. In this example, the conversion formula is defined such that a higher degree of agreement results in a higher utilization score, and a lower degree of agreement results in a lower utilization score.

[0053] In step S104, the server 10 records the generated whole-body data in each database. The calculated, i.e., measured utilization rate is recorded in the database as a user attribute.

[0054] Figure 6 illustrates a 3D database 512. The 3D database 512 has multiple records. Each record corresponds to one user (i.e., subject). In the example in Figure 6, a record for one user is shown. Each record contains at least one set of 3D model data. One set of 3D model data is data of a 3D model generated using image data and distance data captured at a certain date and time. If there are two sets of 3D model data, these are 3D model data generated at two different dates and times. Each 3D model dataset is assigned a timestamp. Each record also contains information about utilization. The information about utilization contains zero or one or more datasets. Each dataset contains the type of utilization, identification information of the 3D model on which the utilization was calculated (i.e., 3D model ID), the utilization value, and a timestamp. User attributes are omitted from the illustration here.

[0055] Here, we will explain the different types of utilization of whole-body data. It should be noted that the timing for measuring the utilization of whole-body data is not limited to when the data is generated or updated; it can be measured at various times depending on the type of utilization, i.e., its content.

[0056] (1) Responding to requests from the system In this example, the information processing system 1 requests a user who is trying to generate or update full-body data to take a specified pose, i.e., a baseline state. The utilization level is defined as a function of the degree of agreement between the captured pose and the specified pose. A higher degree of agreement is measured as a higher utilization level. Requests from the information processing system 1 are not limited to poses when generating or updating full-body data. The baseline state is not limited to poses, but may be related to at least one of facial expressions, poses, and motions. For example, instead of specifying a pose, the information processing system 1 requests a user who is trying to generate or update full-body data to perform a specific motion (hereinafter referred to as the "specified motion"). The information processing system 1 converts the degree of agreement between the motion taken by the user and the specified motion into a utilization level. Specifically, this function is defined such that the utilization level is higher the higher the percentage of responses to requests from the information processing system 1. In one example, requests from the information processing system 1 are given from the server 30, i.e., from each application or service.

[0057] Furthermore, requests from the system may relate to the temporal changes in whole-body data, as described later, or to comparisons with other users.

[0058] (2) Other users appearing together In this example, the utilization rate is defined as a function of the state of other users appearing together in a service where multiple people are simultaneously the subject of photographs or videos, such as a photo sticker machine or photo booth. The state of other users is, for example, their facial expressions, poses, and motions at the time of shooting. In this example, the information processing system 1 requests other users appearing together with the target user to assume a specified pose, i.e., a baseline state. Here, the utilization rate is defined as a function of the degree of agreement between the other users' poses and the specified pose.

[0059] In another example, the status of other users may include attributes related to those users. These attributes might be, for example, the user category specified in Information Processing System 1. Specifically, for example, taking a photo or video with a user designated as an important user in Information Processing System 1 would measure a high level of utilization. In yet another example, the attributes related to other users might be the number of other users pictured together. Specifically, this function might be defined such that the more people pictured together, the higher the utilization score.

[0060] (3) Usage History In this example, the degree of utilization is defined as a function of the usage history of the whole body data. Use of the whole body data includes, for example, displaying the contents of the whole body data after its distribution (e.g., displaying a 3D model in a virtual space or displaying a graph of vital data), generating content by combining the whole body data as a content element with other content elements, using the generated content in a specific application, or other users' reactions to content using the whole body data. More specifically, use of the whole body data includes at least one of the number of times, time, and frequency of these actions. Use in a specific application may include not only displaying the content but also sharing on so-called social media or interactions with other users (e.g., interactions indicating support such as so-called "likes"). This function is defined such that, for example, the greater the number of times, time, or frequency of a given action related to the whole body data itself or its content, the higher the degree of utilization.

[0061] In one example, the use of whole-body data may involve generating new content by combining 3D models with other content elements. For instance, a user can generate video content by combining their own 3D model, another user's 3D model, furniture 3D models, and a 3D model of a stage space, and further defining the movement of each 3D model.

[0062] In one example, the utilization rate of a user's full-body data is calculated based solely on the usage history of the user who acquired the data (e.g., the subject), and does not consider the use of the full-body data by other users. In another example, the utilization rate of a user's full-body data may be a function of both the user's own usage history and the usage history of other users. In yet another example, for a user's full-body data, the utilization rate based on the user's own usage history and the utilization rate based on the usage history of other users may be calculated separately.

[0063] (4) Time-series changes (temporal changes) In this example, usability is defined as a function of the time-series changes (or temporal changes) of the whole-body data. Time-series changes of whole-body data refer to, for example, the temporal changes in the values ​​of specific items in the whole-body data. If the whole-body data is 3D data, the specific items are specific physical features in the 3D data, such as height, chest circumference, or waist circumference. Specifically, the temporal change is, for example, the rate of decrease (i.e., slope) when the temporal change of the physical features of the 3D data is linearly approximated. This function is defined such that, for example, the higher the rate of decrease of the physical features of the 3D data, the higher the usability.

[0064] (5) Comparison with other users In this example, usability is defined as a comparison of whole-body data with other users. Comparison with other users refers to, for example, the difference in the value of a specific item in the whole-body data with that of other users. If the whole-body data is 3D data, the specific item is a specific physical feature in the 3D data, such as height, chest circumference, or waist circumference. The comparison with other users is specifically, for example, the ratio or difference of the value of a specific physical feature of that user to the value of a specific physical feature of a specific other user. There may be multiple specific users, in which case the values ​​of the physical features may be statistical indicators (e.g., mean, mode, or median). This function is defined such that, for example, a higher comparison of the physical features of the 3D data results in a higher usability.

[0065] The utilization levels measured (i.e., calculated) as described above are recorded in the corresponding database as user or content attributes. For example, the utilization level measured for a particular piece of content is recorded in its user database 531 as an attribute of that content.

[0066] 2.2 Generating Personalized Content Figure 7 is a flowchart illustrating the process of generating personalized content. An event that initiates the personalized content generation process is predefined in the server 10. When this event occurs, the server 10 starts the process in step S201. A user is associated with the event, and the server 10 performs the following processing for the user corresponding to the event (hereinafter referred to as the "target user") and the corresponding content (hereinafter referred to as the "target content"). In one example, the event is that a user from a certain user terminal 20 has performed a specific process in a specific application. In this example, the user of the user terminal 20 is the target user, and the content specified in that application is the target content.

[0067] In step S201, the server 10 accesses a database that stores at least one of the content elements and service elements. In one example, the server 10 has a table (described later) that associates events with the target content. The server 10 refers to this table to determine which database to access.

[0068] In step S202, the server 10 obtains the utilization rate of the user's full-body data. The server 10 obtains the utilization rate of the target content for the target user from the database accessed in step S201. The utilization rate of the target content may be recorded in the database as values ​​measured at multiple different times. Also, there may be multiple pieces of target content. In these cases, the server 10 performs statistical processing on the multiple utilization rate values ​​and uses the calculated statistical index as the utilization rate of the target content for the target user, and performs the following processing.

[0069] In step S203, the server 10 determines a method for personalizing at least one of the content elements and service elements according to the acquired usage level. The server 10 has a table (described later) that associates usage levels with personalization methods. The server 10 determines the personalization method by referring to this table. In step S204, the server 10 personalizes the content element or service element using the determined method. In step S205, the server 10 provides the personalized content, i.e., the personalized combination of content elements and service elements, to the user terminal 20.

[0070] Figure 8 illustrates a table 61 that records information related to content personalization. Table 61 has multiple records. In this example, each record corresponds to one event. In the example in Figure 8, the top record indicates that, in response to the event "App A requested personalized content generation after playing a video," the target user is "the user who made the request," the target content is the video, and the personalization method is "provide a new character to be used in App A." This is a situation where, for example, when a user watches a video in App A, App A requests the server 10 to generate personalized content because this video viewing fulfills some condition defined in App A. In response to this request, the server 10 sends data for a new character to be used in App A to the user terminal 20. Note that personalized content is not limited to this example, and various examples can be applied. Several specific examples are given below.

[0071] (1) Providing users with the right to use content elements Consider a situation where an application uses content elements recorded in the IP database 532, but the content elements that can be used (i.e., that which users have the right to use) are limited for each user. Server 10 assigns one content element from among multiple IP content elements to the user with a certain probability (i.e., like a lottery) as a content element that the user can use. A content element to which the right to use is granted in this way is an example of personalized content. For example, one character selected from among several characters appearing in a certain animation is assigned to the user. In this example, the character provided to the user reflects the user's preferences. The user's preferences here refer to information on which of the several characters appearing in the animation the user likes. Regarding the user's preferences, for example, the information processing system 1 may ask the user in advance which character they like and store the answer. Alternatively, the information processing system 1 may determine which character the user likes from the user's content usage history, such as videos, and store the result of that determination. Reflecting user preferences means, for example, that if it is known that a user likes a certain character, the probability of that user being assigned that character will be higher than that of other users. Furthermore, this probability may vary depending on the user's usage level. In other words, the probability may be set so that users who use the game frequently are more likely to get their favorite character.

[0072] Furthermore, the probability may be adjusted independently of user preferences. For example, if content elements are classified into multiple categories (such as normal, rare, and super rare), server 10 assigns one content element from among multiple IP content elements belonging to a category selected with a certain probability to that user with a certain probability. In this case, this probability may be adjusted according to the user's level of use. That is, the probability may be set so that users with high usage rates are more likely to receive rarer characters.

[0073] The content elements to which users are granted usage rights are not limited to characters. The content to which users are granted usage rights may also be, for example, the motion of a 3D model. This motion may also be an IP content element. For example, this motion may be the motion of a special move performed by an animated character in a play. Alternatively, the content elements to which users are granted usage rights may also be music or images.

[0074] (2) Modification of Content Elements Modified content elements are an example of personalized content. 3D model deformation is an example of content element modification. The 3D model is intended to faithfully reproduce information from physical space, reflecting the images or measurements taken by the scanner 40. In this example, the server 10 deforms the 3D model of the target user in a way that emphasizes the user's characteristics according to the user's attributes. Specifically, for example, for a user whose height is taller than the average of the user group, the 3D model is deformed to make the user appear taller. In another example, the 3D model may be stylized like a manga or animation. Specifically, the ratio of head to body size is deformed. In these 3D models, the degree of deformation is determined according to the target user's level of use. For example, a user with a high level of use may use a more heavily deformed 3D model (i.e., one with more emphasized features or a greater degree of deformation). Alternatively, the 3D model deformation may be a single step, and the use of the deformed 3D model may be determined by the target user's level of use.

[0075] (3) Modification of Service Elements Personalization of content elements has been explained above, and the same applies to service elements. For example, server 10 may change the awarding of points or user rank for a target user related to a certain service according to the degree of utilization of whole-body data. Alternatively, server 10 may make arrangements to provide physical goods to the target user.

[0076] 3. Modifications The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the matters described below may be applied in combination.

[0077] (1) The content elements used in the content element information processing system 1 are not limited to those exemplified in the embodiments. 3D model data can be described as data for generating images viewed from a viewpoint specified by the user, i.e., a free viewpoint. In this regard, in addition to, or instead of, the (so-called narrow sense) 3D model data described in the embodiments, 2D image data may be used for techniques that generate 3D content from a set of 2D images, such as neural rendering (or NeRFs: Neural Radiance Fields). In this paper, "3D model" may be interpreted not only as a so-called narrow sense 3D model, but also broadly as a dataset for generating images viewed from a free viewpoint.

[0078] (2) Utilization of whole-body data The utilization of whole-body data is not limited to those exemplified in the embodiments. In the above example, an example was shown in which multiple types of utilization are measured at various timings, but the utilization of whole-body data for a given user may be a value obtained by statistically processing the utilization values ​​for that user at various points on the time axis, for example, the mean, mode, or median.

[0079] In one embodiment, an example was described in which the degree of usability is calculated based on the degree of agreement between a pose taken by a user for 3D model generation (hereinafter referred to as the "shooting pose") and a designated pose presented by the information processing system 1. The specific method for calculating the degree of pose agreement is not limited to the example in this embodiment. The evaluation elements related to the degree of agreement are divided into multiple perspectives, and the server 10 compares the shooting pose and the designated pose for each of these multiple perspectives. The multiple perspectives may include appearance, such as hairstyle and clothing. In this case, the server 10 evaluates the degree of agreement between the shooting pose and the designated pose for, for example, hairstyle and clothing. The evaluation results for these multiple perspectives are statistically processed to obtain an overall degree of agreement.

[0080] In another example, when a specified motion is presented by the information processing system 1, the criteria for matching may include the speed of the motion (the time from the start to the end of the motion). When capturing a user's motion, in one example, a video of the user, who is the subject, performing the specified motion is captured inside the shooting chamber of the scanner 40. So-called motion capture processing may be performed during this shooting. Alternatively, feature points may be automatically extracted from the image, and the subject's motion may be identified based on the extracted feature points.

[0081] (3) Feedback information processing system 1 for improving usability may provide the user with feedback to improve usability. For example, with respect to usability based on pose accuracy, the server 10 provides the user with information indicating which part of the pose should be improved to improve usability. As already explained, pose accuracy is evaluated from multiple perspectives. The server 10 shows the user, for example, which of these perspectives received the lowest evaluation. For example, if the evaluation of the positional relationship of a set of bones was low, the server 10 provides information such as, "The angle between the right fist and the waist does not match."

[0082] (4) Timing of measurement of utilization The timing of measurement of utilization is not limited to those exemplified in the embodiments. The timing of measurement of utilization may be specified by individual applications or services (i.e., by the server 30). Alternatively, the timing of measurement of utilization may be specified by the user (i.e., by the user terminal 20).

[0083] (5) Personalization Method The specific method by which the server 10 determines a method for personalizing content elements is not limited to those illustrated in the embodiments. For example, multiple methods for personalizing content elements may be defined in the server 10, and the server 10 may determine a method for personalizing content elements according to the user's selection or specification.

[0084] (6) The entity that performs personalization of the data is not limited to the examples in the embodiment. In the embodiment, an example was described in which server 10 performs personalization of service elements, but server 30 that provides individual services or applications may also perform personalization of service elements. In this case, server 30 may maintain a database of service elements. Server 30 queries server 10 for the degree of utilization of the target user's whole body data and receives information on the degree of utilization from server 10. Server 30 personalizes the service elements according to the information received from server 10. Server 30 may also perform personalization of content elements in a similar manner.

[0085] (7) The user recommendation information processing system 1 may provide the user with recommendation information in addition to, or instead of, personalized content. In one example, the recommendation information is information about apparel, i.e., clothing. In this example, the information processing system 1 refers to the user's whole body data, in particular whole body data relating to the user's health status, and generates recommendation information about clothing that is appropriate for the user's health status. More specifically, the whole body data relating to the user's health status is data on the body temperature distribution on the user's body surface. The information processing system 1 has an algorithm and reference data that, given the user's body temperature distribution, determines whether the user is prone to feeling cold and which parts of the body are cold, and uses this algorithm and reference data to determine whether the user is prone to feeling cold. For a user determined to be prone to feeling cold, the information processing system 1 generates recommendation information about clothing that has functions suitable for improving cold sensitivity. The information processing system 1 has a database in which the functions of clothing are recorded, and refers to this database to identify clothing that has functions suitable for improving cold sensitivity. According to this example, the information processing system 1 can suggest clothing with functions tailored to the user's health condition.

[0086] (8) The hardware and software configurations of the information processing system 1 are not limited to those exemplified in the embodiments. The information processing system 1 may have any hardware and software configuration as long as it can implement the required functions. Furthermore, the correspondence between hardware elements and functional elements, and between software elements and functional elements, are not limited to those exemplified in the embodiments. For example, at least one database may be stored in a server device other than the server 10. Alternatively, multiple physical devices may cooperate to function as the server 10.

[0087] Some of the functional elements illustrated in Figure 2 may be omitted. These functional elements may be selected and implemented by the administrator of Information Processing System 1.

[0088] The data and databases described in the embodiments are merely illustrative examples, and the data and databases in the present invention are not limited to these examples.

[0089] The programs executed by processors such as CPU 101 may be provided via download over a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Note that each processor may be replaced by, for example, an MPU (Micro Processing Unit) instead of a CPU.

[0090] 1... Information processing system, 9... Network, 10... Server, 11... Storage means, 12... Access means, 13... Acquisition means, 14... Decision means, 15... Provision means, 19... Control means, 20... User terminal, 30... Server, 40... Scanner, 51... Data layer, 52... Judgment layer, 53... Content layer, 61... Table, 101... CPU, 102... Memory, 103... Storage, 104... Communication IF, 151... Presentation means, 152... Calculation means, 153... Assignment means, 154... Assignment means, 155... Modification means, 512... 3D database, 513... Vital database, 514... Photo database, 531... User database, 532... IP database, 533... Advertising database, 534... Service database

Claims

1. An information processing device comprising: access means for accessing a database storing at least one of content elements and service elements; acquisition means for acquiring a variable indicating the degree of utilization of whole-body data relating to the user's whole body; determination means for determining a method for personalizing at least one of the content elements and service elements according to the variable; and provision means for providing the personalized combination of content elements and service elements to the user.

2. The information processing apparatus according to claim 1, wherein the whole-body data is 3D model data representing the user's 3D model, and the variable indicates the degree of agreement between the state of the 3D model at the time of generation of the 3D model data and a reference state.

3. The information processing apparatus according to claim 2, wherein the state includes at least one of the facial expression, pose, and motion of the 3D model.

4. The information processing apparatus according to claim 2, wherein the state is the state of another user who generated whole-body data together with the user.

5. The information processing device according to claim 2, wherein the whole-body data is 3D model data representing the user's 3D model, and the variable indicates the situation in which the user has used the 3D model data after it has been distributed.

6. The information processing apparatus according to claim 5, wherein the user's use of their own 3D model data includes the generation of new content by combining the 3D model data with other content elements.

7. The information processing apparatus according to claim 6, which has a modification means for modifying the 3D model or other content elements when generating new content according to the user's preferences determined from the combination.

8. The information processing device according to claim 1, wherein the whole-body data is 3D model data representing the user's 3D model, and the variable indicates interactions performed by other users other than the user with content using the 3D model data after the 3D model data has been distributed.

9. The information processing apparatus according to claim 1, further comprising a calculation means for calculating the degree of utilization of the whole body data based on the time-series changes of the user's whole body data.

10. The information processing apparatus according to claim 1, further comprising a calculation means for calculating the utilization rate based on a comparison between the full-body data of the user and the full-body data of a user other than the said user.

11. The information processing apparatus according to claim 1, further comprising: a presentation means for presenting a request regarding the whole-body data to the user; and a calculation means for calculating the utilization level based on the extent to which the user has responded to the request.

12. The information processing apparatus according to claim 1, further comprising assignment means for assigning personalized content elements to the user according to probability.

13. The information processing apparatus according to claim 12, wherein the probability is determined according to the variable.

14. The information processing apparatus according to claim 12, wherein the probability is determined according to the user's preferences.

15. The information processing apparatus according to claim 1, wherein the content element includes motion or pose to be performed on a 3D model.

16. The information processing apparatus according to claim 15, further comprising granting means for granting the user the right to use a combination of a 3D model and motion or pose.

17. The information processing apparatus according to claim 1, wherein the service element includes awarding points to the user.

18. An information processing method comprising the steps of: accessing a database storing at least one of content elements and service elements; obtaining a variable indicating the degree of utilization of whole-body data relating to the user's whole body; determining a method for personalizing at least one of the content elements and service elements according to the variable; and providing the personalized combination of content elements and service elements to the user.

Images