Information processing system and information processing method

Abstract

The information processing system includes a first acquisition means for acquiring time-series changes in a 3D model of a user, a second acquisition means for acquiring the results of behavioral analysis or mental analysis of the user, a selection means for selecting a menu suitable for the user from multiple menus included in multiple linked services using the time-series changes and / or the analysis results, and a presentation means for presenting the selected menu to the user.

Description

Technical Field

[0001] The present invention relates to the technology of information processing systems.

Background Art

[0002] Techniques for analyzing user information and proposing optimal services are known. For example, Patent Document 1 discloses a content providing system that generates a service list from user environment, terminal, biometric status information, attribute information, and history information and provides it to the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the invention according to Patent Document 1 only generates and provides a service list, and it cannot be said to have sufficient functions to play an appropriate role in assisting selection for the increasing needs of users in the future. Also, regarding information related to the user himself / herself, it does not contain sufficient information value to analyze the user's behavior or mental state in detail and provide appropriate services according to the state of each user. Furthermore, as the user's behavior range, it cannot be said to be sufficient as an invention that can respond to the service system and user needs that change due to technological advancements such as the use of the user's own 3D model in the virtual space as well as the real space.

[0005] In view of the above background, the present invention provides a more improved technique for assisting in selecting a service suitable for a user from various partner services.

Means for Solving the Problems

[0006] One aspect of this disclosure provides an information processing system comprising: a first acquisition means for acquiring time-series changes of a user's 3D model; a second acquisition means for acquiring analysis results of behavioral analysis or mental analysis of the user; a selection means for selecting a menu suitable for the user from among a plurality of menus included in a plurality of linked services using the time-series changes and / or the analysis results; and a presentation means for presenting the selected menu to the user.

[0007] The system may have a means for awarding rewards and penalties to a user based on their actual use of any of the multiple linked services after being presented with the aforementioned menu.

[0008] The aforementioned linked service may have means of utilizing the aforementioned rewards and punishments.

[0009] The reward / punishment granting means may grant the reward or punishment for use performed through the use means.

[0010] The reward and punishment awarding means may award rewards and punishments in the linked service selected from the plurality of linked services according to the time-series changes and / or the analysis results.

[0011] The reward and punishment awarding means may award the rewards and punishments to each of the multiple linked services using an algorithm that is independently configured.

[0012] The second acquisition means may have an analysis means for performing behavioral or mental analysis of the user, and the second acquisition means may acquire the analysis results of the behavioral or mental analysis from the analysis means.

[0013] The analysis means may perform the behavioral analysis or mental analysis using the user's behavioral history in the virtual space using the 3D model.

[0014] The analysis means may perform the behavioral analysis or mental analysis using the user's behavioral history in real space.

[0015] The analysis means may perform the behavioral analysis or mental analysis using the relationship between the user and other users who use the same linked service.

[0016] The analysis means may perform the behavioral analysis or mental analysis using evaluations of the user from a user other than the user in question.

[0017] The linked services may be divided into multiple types, and the selection means may select a menu from among the multiple types, which is included in the linked service of the type selected according to the time-series changes and / or the analysis results, as a menu suitable for the user.

[0018] Another aspect of this disclosure provides an information processing method comprising the steps of: obtaining a time-series change of a user's 3D model; obtaining the results of an analysis of the user's behavior or mental state; using the time-series change and / or the analysis results, selecting a menu suitable for the user from among a plurality of menus included in a plurality of linked services; and presenting the selected menu to the user. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide an improved technology for assisting users in selecting the most suitable service from among various partner services. [Brief explanation of the drawing]

[0020] [Figure 1] A diagram illustrating the overview of Information Processing System 1. [Figure 2] A diagram illustrating the hardware and network configuration of Information Processing System 1. [Figure 3] A diagram illustrating the functional configuration of information processing system 1. [Figure 4] A diagram illustrating the hardware configurations of the server device 20 and the user terminal 40. [Figure 5]A flowchart illustrating the operation concept of the information processing system 1. [Figure 6] A sequence chart illustrating data analysis in the information processing system 1. [Figure 7] A diagram illustrating the 3D database possessed by the server device 20. [Figure 8] A diagram illustrating the behavior database possessed by the server device 20. [Figure 9] A flowchart illustrating service proposal in the information processing system 1. [Figure 10] A sequence chart illustrating evaluation in the information processing system 1.

Embodiments for Carrying Out the Invention

[0021] 1. Configuration FIG. 1 is a diagram illustrating the overview of the information processing system 1. The information processing system 1 serves as a support system (or simply a system) for both partner businesses (or simply businesses) and end users (or simply users), and this system aims to make such services more convenient and beneficial for users in service businesses that are widely popular in general society, for example.

[0022] Next, in this example, a partner business is a business that provides services to end users. Services provided by a business include, for example, fitness instruction, food and beverage provision (e.g., restaurants or cafes), retail (e.g., bookstores or clothing stores), game provision, movie provision, and music provision. These services are generally operated by independent entities, and each entity can register information about its services in the system, such as a service menu (or simply a menu) or information about the goods or services offered. In this example, a service menu refers to the content of services that a partner business has prepared in advance to provide to end users, such as the content of a course at a fitness gym (e.g., a 1-hour boxing class with one-on-one instruction; a 50-minute dance lesson with group instruction). Information about goods includes attribute information such as the name, specifications, model number, and size of the goods. The location where services provided by a business (entity) can be in either the real world or a virtual world. Businesses may include those that provide services in the real world, those that provide services in a virtual world, and those that provide services in both the real world and a virtual world.

[0023] Furthermore, in this example, the term "end user" refers to a user who utilizes the service. The end user receives a service menu suggestion from the system and can choose and use the services they actually receive from the suggested options. When a user uses a service, their activity history within that service is provided to the system as activity data. In this example, activity data includes concepts such as activity history (activity records) recorded as data by the user via the system (e.g., user terminal). This activity data may include various related pieces of detailed information, such as when, where, with whom, what menu items were ordered, and how much the bill cost, regarding a meal at a restaurant. Users can also provide the system with activity data unrelated to partner businesses in order to receive the most suitable service for themselves, and this activity data provided by the user is recorded in the system.

[0024] In addition, users can receive more suitable services by providing physical data to the system. In this example, physical data refers to 3D model data and biometric data. 3D model data refers to data representing a 3D model in a virtual space. To generate a 3D model of the user, data (RGB and distance) captured by, for example, a 3D scanner is used. Biometric data refers to information about the body, including information such as blood pressure, weight, and pulse rate that can be obtained by sensors. Furthermore, physical data is not limited to data acquired at a specific single point in time, but its time-series changes are also recorded in the system. In this example, time-series changes may refer to the data sequence of physical data acquired and recorded along the time axis, or to information indicating trends in changes, such as the user's body shape "having gained (lost)" or their skin condition "having improved (worsened)."

[0025] Furthermore, users provide the system with their attribute information. This attribute information includes details about the user, such as age, gender, place of residence, occupation, and income.

[0026] Figure 2 illustrates the hardware and network configuration of Information Processing System 1. Information Processing System 1 includes a scanner device 10, a server device 20, a server device 30, a user terminal 40, and a business terminal 50. Each device is integrated and connected via network 9 as shown in Figure 2. In this example, the network is a computer network such as the Internet.

[0027] In this example, the scanner device 10 is an information processing device for acquiring the user's physical data, such as a 3D scanner. Specifically, to generate a 3D model of the human body, the scanner device 10 uses a booth-type 3D scanner to acquire (capture) photographic data (RGB and distance) from a camera installed inside the booth. This allows the scanner device 10 to acquire information such as 3D geometry, skin texture, and skin condition (color). Furthermore, by utilizing various cooperating sensors, the scanner device 10 can acquire not only photographs of the subject but also biometric data (examples of physical data) such as body temperature maps, blood pressure, weight, body fat, and pulse rate. The user can provide this physical data to the information processing system 1. The information processing system 1 can also acquire changes in the user's body (e.g., body shape and / or skin condition) by recording and managing this physical data in chronological order.

[0028] In this example, the server device 20 is an information processing device that performs analysis based on various data provided by the user and executes processing to propose the most suitable service for that user from among the services operated by collaborating partner businesses. The server device 20 functions, for example, as a management server. More specifically, for example, the server device 20 can acquire the user's 3D model data from the scanner device 10, generate the user's 3D model, and manage it by recording it in a database. Alternatively, it can analyze the user's own hobbies, preferences, and / or mental state based on behavioral data provided by the user.

[0029] In this example, the server device 30 refers to one or more information processing devices used by each partner business to provide services to users, and each partner business independently configures its own service server. In this example, the functions that can be realized by the server device 30 include providing services in the real world (restaurants, fitness centers, or apparel stores, etc.) and providing services in the virtual world (metaverse, etc.). Examples of real-world services include restaurant reservations and review input, or fitness gym membership and course selection. Examples of virtual world services include VR (Virtual Reality) experiences or NFT (Non-Fungible Token) games. Users can utilize various services through the server device 30. Various data related to the services are recorded in the server device 30.

[0030] In this example, the user terminal 40 is a terminal device (or simply a terminal) used by an end user, and through this terminal device, the user can use various services provided by the service provider. In this example, the user terminal 40 records the user's actions based on the history of the user's terminal operations. In this example, terminal operations refer to, for example, inputting exercise records at a sports gym in the real world, or processing payment when eating at a restaurant, and the history records details such as training time (e.g., 2 hours), bill amount (e.g., 10,000 yen), or the contents of the meal (e.g., that steak and wine were consumed). These records of actions (action records) are recorded and provided to the system as various data via applications on the user terminal 40 with the user's consent. Furthermore, the services that can be used through the user terminal 40 are not limited to the real world, but may also include services such as games in a virtual space.

[0031] In this example, the service provider terminal 50 is a terminal device (or simply a terminal) used by a partner service provider that provides the service. In this example, the partner service provider operating the service can freely access various service-related data recorded on the server device 30 via the service provider terminal 50. Furthermore, by acquiring and utilizing this data, they can contribute to further improving the quality of the service.

[0032] Figure 3 is a diagram illustrating the functional configuration of the information processing system 1. In this embodiment, it is assumed that each function of the information processing system 1 is mainly implemented in the server device 20. The information processing system 1 includes an acquisition means 21, a selection means 22, a presentation means 23, a reward / punishment awarding means 24, a utilization means 25, and an analysis means 26.

[0033] In this example, acquisition means 21 acquires various types of data. Furthermore, acquisition means 21 includes first acquisition means 211 and second acquisition means 212. In this example, first acquisition means 211 acquires the user's physical data. In particular, when acquiring 3D model data (an example of physical data), in addition to individual data, first acquisition means 211 can acquire the time-series changes of the user's 3D model. Note that this time-series change may include information indicating a trend in change, such as whether the user's body shape has "gained weight (lost weight)."

[0034] On the other hand, the second acquisition means 212 acquires the results of a user behavior analysis or mental analysis. In this example, behavior analysis and mental analysis refer to the types of analysis performed by the server device 20 based on the user behavior data provided to the information processing system 1. Therefore, in this example, the server device 20 can, for example, perform a behavior analysis based on frequently appearing items (e.g., exercise, eating, or music) in the acquired user behavior history (an example of behavior data) to determine the user's preferred behaviors, or perform a mental analysis based on the frequency of purchases of self-help books in the book purchase history (an example of behavior data) to estimate the user's mental state. In this example, the analysis results refer to the user's preferred behaviors (e.g., liking exercise) or estimated mental state (e.g., prone to depression) determined by the server device 20.

[0035] In this example, the selection means 22 uses at least one of the time-series changes of the user's 3D model and the analysis results to select a service menu suitable for the user from among the service menus provided by the partner business operator. The presentation means 23 then presents the service menu selected by the selection means 22 to the user. The reward / punishment awarding means 24 then awards or punishes to the user according to their performance if the user, having received the service menu presentation from the presentation means 23, actually uses one of the multiple linked service menus. Here, "rewards and punishments" is a concept that includes both things that have a positive effect on the user (i.e., benefits) and things that have a negative effect (i.e., punishments). Examples of things that have a positive effect on the user include the awarding of points, the issuance of coupons, the granting of the right to participate in events, and the awarding of titles. Examples of things that have a negative effect on the user include the deprivation of points, the deprivation of coupons, the deprivation of titles, and the awarding of behavioral restrictions. The utilization means 25 then uses the rewards and punishments (mainly benefits) given to the user in the linked service according to the user's instructions. Furthermore, the analysis means 26 performs behavioral analysis or mental analysis of each user based on the behavioral data.

[0036] Figure 4 illustrates the hardware configurations of the server device 20 and the user terminal 40, respectively. The server device 20 is a computer having a CPU (Central Processing Unit) 201, memory 202, storage 203, and communication IF 204, and is, for example, a personal computer or a mainframe. The CPU 201 is a processor that performs various calculations according to a program. The memory 202 is a main memory that functions as a work area when the CPU 201 executes a program, and includes, for example, RAM (Random Access Memory). The storage 203 is an auxiliary storage device that stores various data and programs, and includes, for example, an SSD (Solid State Drive) or HDD (Hard Disk Drive). The communication IF 204 is a device that communicates with other devices according to a predetermined communication standard, and includes, for example, a NIC (Network Interface Card).

[0037] In this example, the program stored in storage 203 includes a program for causing the computer to function as a server in information processing system 1 (hereinafter referred to as the "server program"). When CPU 201 is executing the server program, CPU 201 is an example of an acquisition means 21, a first acquisition means 211, a second acquisition means 212, a selection means 22, a presentation means 23, a reward / punishment granting means 24, a utilization means 25, and an analysis means 26.

[0038] The user terminal 40 is a computer having a CPU 401, memory 402, storage 403, communication IF 404, input device 405, and display device 406, and is, for example, a smartphone, tablet, or personal computer. The CPU 401 is a processor that performs various calculations according to a program. The memory 402 is a main memory that functions as a work area when the CPU 401 executes a program, and includes, for example, RAM. The storage 403 is an auxiliary memory that stores various data and programs, and includes, for example, an SSD or HDD. The communication IF 404 is a device that communicates with other devices according to a predetermined communication standard, and includes, for example, a wireless chip when communicating wirelessly. The input device 405 is a device for inputting information into the user terminal 40, and includes, for example, a touchscreen, keyboard, mouse, or pointing device. The display device 406 is a device that displays information, and includes, for example, an organic EL (Electro Luminescence) display or a liquid crystal display.

[0039] Furthermore, the user terminal 40 includes a device that the user can operate in order to use services in a virtual space or a mixed reality (MR) space, such as a VR headset or smart glasses. In this case, the input device 405 includes, for example, a VR controller. The display device 406 also includes, for example, a head-mounted display.

[0040] In this example, the program stored in storage 403 includes a program that allows the computer to function as a client in the information processing system 1 (hereinafter referred to as the "client program"). When the CPU 401 is executing the client program, the CPU 401, memory 402, storage 403, communication interface 404, input device 405, and display device 406 are examples of functions for operating the user terminal 40.

[0041] 2.Operation Figure 5 is a flowchart illustrating the operational concept of Information Processing System 1. In this example, the operational concept or operational concept diagram is a simplified representation (diagram) of the operation of Information Processing System 1 as a whole, in cooperation with individual devices such as the scanner device 10, server device 20, server device 30, user terminal 40, and business terminal 50, or the operation that functions through the intervention of a user or partner business. Therefore, the operation of the entire Information Processing System 1 is controlled based on the flow in the operational concept diagram. On the other hand, the details of each operation are controlled based on the sequence processing or operation flow of the individual devices. The main operational concepts will be explained below in accordance with the flowchart in Figure 5.

[0042] In step S1, the information processing system 1 uses a 3D scanner or the like to acquire the time-series changes of the user's 3D model. In this example, the time-series changes include, for example, the results of an analysis such as the user's body shape "has gained weight" or "skin condition has improved." The information processing system 1 also acquires the user's behavioral history in the real and virtual worlds (an example of behavioral data) and performs a behavioral analysis or mental analysis of the user from the behavioral data. Alternatively, the information processing system 1 may acquire the results of the behavioral analysis or mental analysis of the user.

[0043] In step S2, the information processing system 1 uses the time-series changes of the user's 3D model, which it has analyzed or acquired, as well as at least one of the analysis results of behavioral analysis and mental analysis, to select and propose a service menu suitable for the user from among the service menus provided by partner businesses. Specifically, for example, based on data such as "likes sports (behavioral analysis)," "has gained weight (time-series changes of 3D model)," and "accumulates stress (mental analysis)" for a certain user, the information processing system 1 can propose a "boxercise course" offered by a partner fitness gym to that user.

[0044] In step S3, the information processing system 1 executes the service menu ultimately selected by the user. Note that "execution" here includes concepts such as the user proactively going to a fitness gym and performing boxing exercises.

[0045] In step S4, the information processing system 1 performs an evaluation of the executed service menu. In this example, "evaluation" refers to an evaluation of the user's actions in performing the service. Alternatively, it includes the concept of a subjective evaluation by the user, such as "it was fun" or "it was boring," if the user actually used the proposed service. If the evaluation is completed in step S4, the information processing system 1 performs another user analysis based on that evaluation (returning to step S1). Alternatively, in addition to the evaluation, the information processing system 1 may perform a user analysis together with, for example, behavioral data from the "boxercise course."

[0046] The above describes the operational concept of Information Processing System 1. Next, we will explain the details of its operation. The details of each operation are controlled, for example, based on sequence processing or operation flow by individual devices. Therefore, the following explanation will be broadly divided into three operational items: "data analysis," "service proposal," and "evaluation."

[0047] 2-1. Data Analysis Figure 6 is a sequence chart illustrating data analysis in the information processing system 1. The following also describes the acquisition of data necessary for analysis by the information processing system 1. In step SA1, the scanner device 10 uses a shooting device to photograph the subject (user) and acquires the image data in order to generate a 3D model of the user. In this example, the acquired image data includes RGB data and distance data. The scanner device 10 may also acquire biometric data (an example of physical data) such as body temperature maps, blood pressure, weight, body fat, and pulse rate by utilizing various cooperating sensors.

[0048] In step SA2, the scanner device 10 transmits the acquired user image data to the server device 20. In this example, when the scanner device 10 transmits image data to the server device 20, it may also transmit biometric data such as blood pressure, weight, and pulse rate as described above.

[0049] In step SA3, the server device 20 generates a 3D model of the user from the acquired image data. In this example, the generation of the 3D model includes processes to reproduce the 3D geometry, skin texture, and skin condition (color) to be equivalent to the actual user. Note that "reproduction" here does not necessarily mean an equivalent to the actual user; for example, it may be the generation of a 3D model that has undergone correction processing, or the generation of a 3D model that is a stylized representation of the user using an avatar or the like.

[0050] In step SA4, the server device 20 records the generated 3D models in the database. In this example, the server device 20 distributes the generated 3D models to each user and records them in chronological order. Users periodically update their 3D models with the scanner device 10 (for example, once a year or once every six months), and each time, the data is recorded in the database in chronological order.

[0051] Here, in step SA4, the database of the server device 20 will be described. Figure 7 is an example of the 3D database of the server device 20. In this example, the 3D database 2001 contains a user ID, a timestamp T, 3D model data DD, biometric data LL, and attribute information. Furthermore, the attribute information includes the scanner identification information ID and the biometric information type ID. In this example, the user ID is unique ID information for identifying each user. The timestamp T is computer time information that represents the time when each data was recorded and modified in the 3D database 2001. In this example, as in timestamp T[1], the number in "〇" within [〇] represents the order of the data given in chronological order.

[0052] Furthermore, 3D model data DD is data representing a 3D model in virtual space, and includes shape data (geometry such as mesh, curves, surfaces, or metaballs) and texture data. Biometric data LL includes the user's biometric information (blood pressure, weight, body fat percentage, and blood glucose level, etc.). Scanner identification information ID is unique ID information to identify the scanner that captured the user, and biometric information type ID is unique ID information to identify the type of biometric information of the user that was acquired.

[0053] In addition, the 3D database 2001 may also have supplementary data attached to the 3D model data DD. This supplementary data includes various types of data, such as attribute data representing the user's personal attributes (age, gender, and place of residence, etc.) that are related to the 3D model, and / or behavioral data representing the user's behavioral history (combinations of actions and locations in real or virtual space), although these data are not directly used when generating the user's 3D model and displaying it on a display, etc.

[0054] Furthermore, the server device 20 can freely access the 3D database 2001 and acquire various types of data. In this example, the first acquisition means 211 of the server device 20 can acquire data recorded in the 3D database 2001 as a time-series change of the user's 3D model. In this example, the time-series change may include, for example, changes in the user's body shape such as "gaining weight (losing weight)" or "having good (bad) skin condition," and may be determined by the server device 20 based on the data recorded in the 3D database 2001. The content of these changes is information that the system uses to propose the optimal service to each user, and may be defined using any judgment criteria and means.

[0055] Returning to Figure 6, in step SA5, the user terminal 40 records the user's actions based on the user's terminal operation history. In this example, terminal operations refer to, for example, inputting exercise records at a gym in the real world, or payment processing when eating at a restaurant. The history may include details such as training time (e.g., 2 hours) and cost (e.g., 10,000 yen) or the contents of the meal (e.g., steak and wine). These activity records are recorded as various data via applications on the user terminal 40 with the user's consent. As an example, activity records by the user terminal 40 in the real world have been described, but activity records in a virtual world are also acceptable. For example, if the user terminal 40 is a VR headset, the content could be an adventure record in a game in the metaverse, such as defeating monsters in XX village.

[0056] In step SA6, the user terminal 40 transmits behavioral data to the server device 20. In this example, behavioral data includes the user's behavioral history (or behavioral record) recorded as data via the user terminal 40, and refers to various related pieces of detailed information, such as when, where, with whom, what menu items were ordered, and how much the bill cost, regarding the aforementioned meal at the restaurant. In this example, the user can also provide behavioral data to the system via the user terminal 40 in order to receive more optimal service.

[0057] In step SA7, the server device 20 records the received behavioral data in the database. In this example, it is preferable that detailed information is recorded for each user action (or time, etc.), and in this case, the format in which the data is recorded is defined in the database of the server device 20.

[0058] Here, in step SA7, the database of the server device 20 will be described. Figure 8 is an example of the behavior database of the server device 20. In this example, the behavior database 2002 holds data on the behavior history of a user identified by a unique user ID (1234567) in both the real and virtual worlds. In the case of behavior history in the real world, this data includes items such as timestamp, category, location, action, target / content, and remarks. In the case of behavior history in the virtual world, it also includes the name of the virtual world. Note that Figure 8 is merely an example, and any items may be set.

[0059] Returning to Figure 6, in step SA8, the server device 20 starts the analysis process in response to a trigger event. In this example, the trigger event includes concepts such as a predetermined amount of time having elapsed since the last analysis, or the amount of unanalyzed data exceeding a threshold, or a specific program process within the computer.

[0060] Regarding the processing in step SA8, the analysis means 26 of the server device 20 performs an analysis of each user based on the behavioral data recorded in the behavioral database 2002. This analysis is further divided into behavioral analysis and mental analysis, and each analysis will be explained below. When performing each analysis, the behavioral data used by the analysis means 26 is based solely on the data recorded in the behavioral database 2002. However, the analysis means 26 may also perform behavioral and mental analysis based on predetermined perspectives, such as the user's behavioral history in a virtual space using their 3D model, the user's behavioral history in the real world, the user's relationship with other users using the same collaborative service, or evaluations of a particular user from other users. Therefore, the specific examples used in the following explanation are merely examples based on these perspectives.

[0061] In this example, behavioral analysis refers to an analytical method for analyzing a user's behavioral history and estimating that user's preferences. For example, analysis means 26 refers to the behavioral database 2002 and, if there are behaviors that appear frequently in the history (such as weight training), it estimates that these behaviors are preferred by the user. Furthermore, analysis means 26 may indirectly estimate that the user prefers behaviors preferred by other people who visit the same or similar places as the user in real or virtual space (for example, painting in XX Park). In addition, analysis means 26 may also estimate that the user prefers behaviors preferred by other people who have similar attributes to the user (such as age or gender).

[0062] Furthermore, for example, after an action in the real or virtual space, a user may input data about their own emotions (such as happy, bored, pleased, or sad) via the user terminal 40. The server device 20 then links the acquired behavioral history (an example of behavioral data) with this emotion data and records it in the behavioral database 2002. In this example, the analysis means 26 may refer to the behavioral database 2002 and estimate the user's preferred behaviors based on the emotion data linked to the behavioral history, for example, based on positive emotions such as happy or pleased. The user's preferred behaviors estimated by the analysis means 26 may be divided into categories such as exercise, eating, music, movies, or reading, and may be further divided into more detailed genres such as weight training, sushi, or pop music.

[0063] In this example, mental analysis refers to an analytical method for analyzing a user's behavioral history and estimating that user's mental state. For example, after a user's actions in the real or virtual space, the user inputs data about their emotions (happy, bored, joyful, or sad, etc.) via the user terminal 40. The server device 20 then links the acquired behavioral history (an example of behavioral data) with this emotional data and records it in the behavioral database 2002. In this example, the analysis means 26 refers to the behavioral database 2002 and estimates the current or future mental state from the emotional data linked to the behavioral history, for example, based on the time-series changes in emotions.

[0064] Furthermore, for a user's actions in real or virtual space, for example, other users can input evaluations (such as "likes"), and the server device 20 links the acquired behavioral history (an example of behavioral data) with this evaluation data and records it in the behavioral database 2002. In this example, the analysis means 26 refers to the behavioral database 2002 and estimates the current or future mental state from the evaluation data linked to the behavioral history, for example, by looking at the time-series changes in evaluations from other users.

[0065] Furthermore, the analysis means 26 may estimate the user's mental state based on their relationship with other users who performed the same actions in real or virtual space (comparison of actions with other users). For example, in the case of a user who used a fitness service, if several people chose the same exercise course, and while their friends lost weight, the user's weight did not change, the analysis means 26 may estimate the user's mental state as depressed. In this example, the behavior database 2002 may add items such as friends, lovers, or family members for other users who are highly related to the user, and link each user ID to these items for management within the database. In other words, by linking these databases together, the analysis means 26 can compare the actions of the user with those of other users and estimate the user's mental state.

[0066] The behavioral and mental analysis performed by the server device 20 described above is merely an example, and the analysis may be performed by any method. For example, an analysis method that utilizes AI or machine learning to more accurately analyze the user's behavior and mental state from a vast amount of data may be adopted. In addition, the second acquisition means 212 in the server device 20 acquires the analysis results of the behavioral or mental analysis performed by the analysis means 26. This allows the server device 20 to acquire the necessary information that will serve as selection criteria for proposing the most suitable service to the user. Next, a more detailed embodiment of the service proposal operation in the information processing system 1 will be described.

[0067] 2-2. Service Proposal Figure 9 is a flowchart illustrating a service proposal in the information processing system 1. The service proposal flow is initiated, for example, in response to a request from a user. Alternatively, the service proposal may be initiated automatically by the server device 20. In step SB1, the server device 20 acquires the time-series changes of the user's 3D model. More specifically, the first acquisition means 211 acquires the time-series changes of the user's 3D model recorded in the 3D database 2001. In this example, the time-series changes may include, for example, changes in the user's 3D model such as "gaining weight (losing weight)" or "improving skin condition (bad condition)," and may be determined by the first acquisition means 211 based on the data recorded in the 3D database 2001.

[0068] In step SB2, the server device 20 acquires the results of the user's behavioral analysis and mental analysis. More specifically, the second acquisition means 212 acquires the results of the behavioral analysis and mental analysis for each user's analysis performed by the analysis means 26, based on the behavioral data recorded in the behavioral database 2002. In this regard, the second acquisition means 212 may acquire the analysis results directly from the analysis means 26. In this example, the results of the behavioral analysis include the user's hobbies and preferences, such as "User XX likes the behavior YY," so the second acquisition means 212 identifies a predetermined category for this "YY." In this example, the category includes exercise, food, or entertainment, and is used to narrow down the service content in order to propose the most suitable service to the user. Note that this category may be directly linked to the "category" included in the behavioral data in the behavioral database 2002.

[0069] In step SB3, the server device 20 selects a menu of linked services suitable for the user. More specifically, the selection means 22 uses at least one of the time-series changes in the user's 3D model and the analysis results to select a service menu suitable for the user from the service menus provided by partner businesses. In this example, regarding the method of service menu selection by the selection means 22, for example, based on data such as "likes sports (behavioral analysis)," "has gained weight (changes in 3D model)," and "accumulation of stress (mental analysis)" for a certain user, the selection means 22 selects the "boxercise course" provided by a linked fitness gym for that user. For example, the server device 20 stores the conditions of the behavioral history and the conditions of the service menu to be selected when those conditions are met in a database (not shown). The server device 20 refers to this database to select a service menu suitable for the user. Note that the selection method is not limited to this, and for example, AI or machine learning may be used. For example, a machine learning model is used in which the user's behavioral history, the services the user has received, and the user's feedback (or impressions) after receiving the services are given as training data. Other types of data may be used as training data. For example, data such as day of the week, time, weather, or location, or unique data such as the user's routine, may be taken into consideration to select the most optimal service for the user.

[0070] In step SB4, the server device 20 presents (proposes) the selected service menu to the user. More specifically, the presentation means 23 presents the selected service menu to the user. In this example, the user can receive a service menu proposal from the server device 20 via the user terminal 40 or the like.

[0071] Furthermore, if a user actually uses the proposed service, the server device 20 may record the behavioral data in a database such as the behavioral database 2002 via the user terminal 40. At that time, the user may be asked to input data about their own emotions (happy, bored, pleased, or sad, etc.) via the user terminal 40. The server device 20 links the acquired behavioral history (an example of behavioral data) with this emotion data and records it in the behavioral database 2002.

[0072] In this example, emotional data may be used as training data for AI learning by the server device 20 to select or evaluate the service itself. Furthermore, regarding this training data, the results of behavioral and mental analysis of users after using the service (e.g., they started going to the fitness center regularly, their stress levels decreased, etc.) may also be used by the server device 20 to learn the AI ​​to select or evaluate the service itself.

[0073] Furthermore, the use of these services by users, and the controls executed in conjunction with them, may be used, for example, as triggers for processes such as step SA5 in the data analysis in Figure 6. This allows the server device 20 to further re-analyze the user based on their actions. Next, a more detailed embodiment of the evaluation operation in the information processing system 1 will be described.

[0074] 2-3. Evaluation Figure 10 is a sequence chart illustrating evaluation in the information processing system 1. In this example, evaluation refers to the process of awarding rewards and punishments to a user based on their behavioral history. This process is primarily performed by the server device 20. Specifically, when a user who has received a service menu from the presentation means 23 actually uses one of the multiple linked service menus, the reward / punishment awarding means 24 awards a reward or punishment to the user according to their performance. Here, we will explain an example in which a special benefit is awarded among the rewards and punishments.

[0075] In this example, the points awarded as a reward include those that have value that users can exchange for other services or goods within the system or a specific service. Furthermore, in this example, the user grade includes ranks or hierarchies used to rank users based on their usage history within the system or a specific service. The specific processing steps are described below.

[0076] In step SC1, the user terminal 40 transmits the user's service usage record to the server device 20. In this example, the service usage record includes the user's activity history (activity record) provided by the user via the user terminal 40. In other words, the service usage record is an example of activity data and, in reality, refers to the user's activity history. For example, if a user trains at a fitness gym, the service usage record (activity history) includes various related pieces of detailed information such as when, where, with whom, what kind of training course was chosen, and how satisfied the user was.

[0077] In step SC2, the server device 20 records the service usage data received from the user terminal 40 in the database. In this example, the server device 20 records the service usage data as behavioral data in the behavioral database 2002.

[0078] In step SC3, the service provider terminal 50 transmits the service usage record to the server device 20. In this example, the service usage record transmitted from the service provider terminal 50 is data corresponding to the service usage record (behavioral data) provided by the user via the user terminal 40 in step SC1. More specifically, the service menu proposed by the server device 20 is mainly a service operated by a partner company, and for example, if the user actually uses the proposed service menu, the server device 30, which is the service server, also records the user's service usage record as data, similar to the user terminal 40.

[0079] In this example, if we again illustrate the user's training at the fitness gym as described above, the server device 30 will record information such as which user booked what training course, when, what their satisfaction level was after completion, and the estimated number of calories burned. This information is not necessarily limited to or identical to the behavioral data recorded on the user terminal 40, and may be unique data specific to each service.

[0080] The data recorded on the server device 30 can be freely accessed, for example, by a partner service provider operating the service via the service provider terminal 50, and used to further improve service quality. In this example, the data acquired by the service provider terminal 50 is ultimately transmitted to the server device 20 as service usage data (step SC3).

[0081] In step SC4, the server device 20 records the service usage data received from the service provider terminal 50 in a database. In this example, the server device 20 may record the service usage data as behavioral data in the behavioral database 2002, or it may record it in a database different from the behavioral database 2002, or it may define (link) the relationship with the service usage data received from the user terminal 40 and record it in various databases.

[0082] In step SC5, the server device 20 detects a trigger event before starting the evaluation process. In this example, a trigger event includes concepts such as a predetermined amount of time having elapsed since the last evaluation, or the amount of unevaluated data exceeding a threshold, or a specific program process within the computer.

[0083] In step SC6, the server device 20 evaluates the user. More specifically, the reward / punishment awarding means 24 calculates the points to be awarded to the user. In this example, the points are divided into two types: common points that can be used in common with all partner businesses, and unique points that can be used only with specific partner businesses. Depending on the intended use, users can choose which points to use or exchange common points for unique points. In this example, common points could be points that can be used to pay for goods or services at multiple physical stores or online stores operated by different businesses, while unique points could be points that can be used to pay for goods or services operated by a single specific business (an example being airline mileage).

[0084] First, regarding the calculation of common points, the reward / punishment awarding means 24 calculates common points based on the user's service usage record, according to an algorithm and calculation formula predetermined within the system. In this example, the reward / punishment awarding means 24 may refer to behavioral data in the behavioral database 2002 and employ various calculation methods, such as awarding the user 100 points for exercising at a fitness gym or dining at a restaurant, and awarding the user 3 points for listening to music via a music distribution app.

[0085] Next, regarding the calculation of proprietary points, the algorithm and formula that form the basis of the calculation method are provided by the system. Furthermore, the parameters within the algorithm and formula can be freely set by each partner business operator. As an example, consider the case where a user uses a service menu for overseas travel by airplane. In this example, the reward / penalty awarding means 24 can calculate mileage, which is proprietary points, based on the user's service usage record, using an algorithm and formula customized by the airline, and award it to the user.

[0086] The evaluation process described above applies not only to points but also to user grades. In this example, user grades may be ranked by categories such as Gold Member, Silver Member, or Bronze Member. Furthermore, user grades may have two categories: a common grade determined by the system and a unique grade determined by each partner business. In addition, the method for determining each grade is the same as the method for calculating points, and is determined by the reward / penalty awarding means 24 based on the user's service usage record, according to a predetermined algorithm and calculation formula within the system.

[0087] Furthermore, with regard to these points and user grades, the usage means 25 can utilize these rewards and penalties in the linked service at the user's instruction. In this example, the use of rewards and penalties by the usage means 25 includes, for example, in the case of points, processing such as exchanging them for other services or goods, and in the case of user grades, processing such as exercising rights granted according to the grade (e.g., a 30% discount for gold members). In addition, if these rewards and penalties are utilized by a user via the usage means 25, the reward / penalty granting means 24 may evaluate the user's use and further grant rewards or penalties to that user.

[0088] 3. Variant The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the items described below may be applied in combination.

[0089] (1) Information processing system 1 The hardware configuration in the information processing system 1 is not limited to those exemplified in the embodiment. The information processing system 1 may have any hardware configuration as long as it can realize the required functions. For example, multiple physical devices (e.g., scanner devices 10) may cooperate to function as a server device 20.

[0090] Furthermore, the network configuration in the information processing system 1 is not limited to those exemplified in the embodiment. For example, if a scanner device 10 is included as one of the services provided by a partner business to a user, the scanner device 10 and the server device 30 may be connected via the network to realize the provision of the service.

[0091] The correspondence between functions and hardware is not limited to those illustrated in the embodiments. For example, the function of generating 3D model data from image data may be implemented in the scanner device 10. In this case, the server device 20 acquires the 3D model data from the scanner device 10.

[0092] (2) Server device 20 The server device 20 may be a physical server or a virtual server (including so-called cloud servers). Furthermore, the correspondence between functional elements and hardware is not limited to those exemplified in the embodiment. For example, at least some of the functions described as being implemented in the server device 20 in the embodiment may be implemented in other devices (e.g., the scanner device 10), or conversely, at least some of the functions described as being implemented in other devices (e.g., the scanner device 10) may be implemented in the server device 20. In this example, instead of a configuration in which the user's 3D model is generated by the server device 20, a configuration in which the user's 3D model is generated by the scanner device 10 may also be used.

[0093] (3) Action steps (action concept) The flowchart shown in Figure 5 is merely one example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be omitted, the order may be changed, or new operations may be added. In this example, in step S1, the information processing system 1 performs analysis processing related to the time-series changes, behavioral analysis, or mental analysis of the user's 3D model, but this may be replaced with an operation that, for example, cooperates with an external user analysis system (not shown) to acquire only the data necessary for service selection (proposal), i.e., the analysis results.

[0094] (4) Data analysis 2-1. In data analysis, the operation of Information Processing System 1 is merely an example and is not limited to what is described above. In this example, regarding the analysis method for mental analysis, for example, voice analysis may be employed to estimate the user's psychological state from the tone, manner of speaking, or content of speech recorded via a voice interface such as Siri® and Alexa®.

[0095] Alternatively, an analysis method that includes feedback from others related to the user may be employed. In this example, feedback refers to, if the user is a student, feedback from teachers regarding their attitude in class or their lifestyle; if the user is an office worker, feedback from superiors or colleagues regarding their work attitude or behavior at work; or, if the user is a patient with some kind of condition, feedback from a doctor at a hospital regarding their symptoms or physical condition. Mental analysis may be performed in conjunction with a system that records this feedback.

[0096] Furthermore, the types of analysis performed by the server device 20 are not limited to the behavioral analysis and mental analysis described above. In this example, the analysis means 26 may perform a personality analysis of the user from the behavioral history (an example of behavioral data). For example, if a long-term activity (service) is detected from the user's behavioral history, the user may be classified by applying a predetermined personality, such as "a personality that diligently and continuously works on things" (personality classification A). In addition, the server device 20 may analyze the user's lifestyle habits based on health-related behavioral history such as eating, exercising, drinking, or smoking, and may manage the user by setting up stages of health risk (e.g., risk level 2).

[0097] Furthermore, using these analysis results, the selection means 22 may select a menu that is suitable for the user from among the multiple menus included in the multiple linked services. For example, based on data such as "likes music (behavioral analysis)," "likes new things (personality analysis)," and "high motivation to act (mental analysis)" for a particular user, the selection means 22 may select and suggest the menu "instrument lessons at a music school" to that user.

[0098] (5) Service proposal In service proposals, the operation of the information processing system 1 is merely an example and is not limited to what is described above. For example, the server device 20 does not have to present the user with a specific service menu offered by a particular partner service provider. For example, the server device 20 may refer to data showing the target user's schedule (or planned activities) and propose a service menu that corresponds to those activities. Specifically, if the target user is scheduled to attend a party, the server device 20 may suggest to the user the purpose of the party and appropriate attire for the attendees (for example, "We recommend wearing a gray hoodie"). In this case, the server device 20 will make this suggestion even if the sale of gray hoodies is not predefined as a service menu item of a particular service provider. In other words, in this case, the server device 20 can be said to be generating a new service menu by referring to service menu data obtained from a service provider or other data, and then presenting that service menu to the user.

[0099] The server device 20 may acquire data indicating public trends (or fads) in addition to the user's 3D model data and behavioral data, and may refer to this data to suggest service menus. Furthermore, although the embodiment described an example in which the server device 20 proposes a service menu by considering the time-series changes of the user's 3D model and the results of behavioral analysis and mental analysis, the server device 20 may propose a service menu by considering only at least one of these. That is, the server device 20 proposes a service menu by considering (a) the time-series changes of the user's 3D model and (b) at least one of the analysis results (in other words, the time-series changes of the user's 3D model and / or the analysis results). The analysis results include the results of at least one of behavioral analysis and mental analysis (in other words, the results of the user's behavioral analysis and / or mental analysis).

[0100] (6) Evaluation 2-3. In the evaluation, the operation of Information Processing System 1 is merely an example and is not limited to what is described above. In this example, the conditions for awarding points may be freely set. The conditions for awarding points are conditions related to the user's behavior history. For example, even if multiple users use the same service, a condition may be set in which users with a higher user grade are awarded more points than users with a lower user grade. Alternatively, if a user who has used a service proposed by Information Processing System 1 continues to use that service afterward, a condition may be set in which additional points are awarded or the point award rate is increased depending on the duration of use, etc.

[0101] The benefits may be categorized into several types, such as the awarding of points, the issuance of coupons, the granting of event participation rights, and the awarding of titles. Coupons can be used for services provided by partner service providers. Event participation rights are tickets to participate in events held in the real or virtual space (e.g., parties, concerts, meetings, or workshops). Titles are honors bestowed upon users. Users may decide which types of benefits they are awarded, or the server device 20 may decide automatically. These conditions are maintained, for example, in a database stored by the server device 20.

[0102] The above example specifically illustrates the benefits of rewards and punishments, but the same principles apply to punishments as well. For example, the server device 20 stores a database containing the conditions for imposing punishments. These conditions may include relationships between multiple different types of behavior. Different types of behavior could be, for example, exercise and diet. Specifically, a condition might be that punishment is imposed if, after exercising at a fitness gym, a specified food or drink (for example, a food or drink high in sugar or carbohydrates) is consumed within a specified time (for example, within one hour).

[0103] (7) Data format The data format of the 3D model data handled by Information Processing System 1 can be any format. For example, the 3D model data may be in a data format that includes biological data or past 3D model data as a single entity.

[0104] (8) Others CPU201, CPU401, and the various programs executed by each device may be provided via download over a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Note that each processor may be replaced with, for example, an MPU (Micro Processing Unit) instead of a CPU. [Explanation of Symbols]

[0105] 1...Information processing system, 10...Scanner device, 20...Server device, 30...Server device, 40...User terminal, 50...Business operator terminal, 9...Network, 21...Acquisition means, 211...First acquisition means, 212...Second acquisition means, 22...Selection means, 23...Presentation means, 24...Reward / punishment means, 25...Utilization means, 26...Analysis means, 201...CPU, 202...Memory, 203...Storage, 204...Communication interface, 401...CPU, 402...Memory, 403...Storage, 404...Communication interface, 405...Input device, 406...Display device, 2001...3D database, 2002...Behavioral database

Claims

1. A first acquisition means for acquiring a time series change of a 3D model of a user; A second acquisition means for acquiring an analysis result of the behavioral analysis or mental analysis of the user; a selection means for selecting a menu suitable for the user from among a plurality of menus included in a plurality of linked services, using the time-series change and / or the analysis result; a presentation means for presenting the selected menu to the user; An analysis means for performing the behavioral analysis or mental analysis of the user; having The analysis means performs the behavioral analysis or the mental analysis using a history of actions taken by the user in the virtual space using the 3D model, The second acquisition means acquires an analysis result of the behavioral analysis or the mental analysis from the analysis means. Information processing system.

2. a reward / penalty giving means for giving a reward or penalty to the user who has received the menu according to the user's actual use of any one of the plurality of linked services; 2. The information processing system according to claim 1, further comprising:

3. A means for utilizing the reward and punishment in the linked service 3. The information processing system according to claim 2, further comprising:

4. The reward / penalty giving means gives the reward / penalty for the use performed via the use means. The information processing system according to claim 3 .

5. The reward / penalty applying means applies a reward or penalty to a linked service selected from the plurality of linked services in accordance with the time-series change and / or the analysis result. The information processing system according to claim 2 .

6. The reward / punishment granting means grants the reward / punishment to each of the plurality of linked services using an algorithm that is uniquely set.

6. The information processing system according to claim 5.

7. The analysis means performs the behavioral analysis or the mental analysis using a history of actions taken by the user in real space. The information processing system according to claim 1 .

8. The analysis means performs the behavioral analysis or the mental analysis using a relationship between the user and other users who use the same linked service. The information processing system according to claim 1 .

9. The analysis means performs the behavioral analysis or the mental analysis using an evaluation of the user from a user other than the user. The information processing system according to claim 1 .

10. The linked services are divided into a plurality of types, The selection means selects, as a menu suitable for the user, a menu included in a linked service of a type selected from the plurality of types in accordance with the time-series change and / or the analysis result. The information processing system according to claim 1 .

11. A computer comprising: Obtaining a time series change of a 3D model of a user; acquiring a result of a behavioral analysis or mental analysis of the user; selecting a menu suitable for the user from among a plurality of menus included in a plurality of linked services using the time-series change and / or the analysis result; presenting the selected menu to the user; A step of performing a behavioral analysis or mental analysis of the user; having In the step of performing the behavioral analysis or mental analysis, the behavioral analysis or mental analysis is performed using a behavior history of the user in the virtual space using the 3D model, In the step of acquiring the analysis result, the analysis result of the behavioral analysis or mental analysis performed in the step of performing the behavioral analysis or mental analysis is acquired. Information processing methods.

Images