Machine learning system, machine learning device, and machine learning method

The machine learning system addresses unconscious environmental influences on behavior by optimizing environmental factors like scent, lighting, and audio to encourage target actions, improving behavior modification efficiency.

JP7586174B2Active Publication Date: 2024-11-19SONY GROUP CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2022518600
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-07-06
Filing Date
2021-01-15
Publication Date
2024-11-19
Estimated Expiration
2041-01-15

AI Technical Summary

Technical Problem

Conventional methods for encouraging a person to take a target action, such as sales promotion, do not account for unconscious changes in behavior due to environmental factors, leading to inefficiencies in behavior modification.

Method used

A machine learning system that acquires and evaluates the correlation between a person's behavior and their environment, using reinforcement learning to select environmental factors that maximize a value function to encourage desired behaviors, incorporating scent, lighting, temperature, humidity, video, and audio control to influence behavior.

Benefits of technology

The system effectively promotes target behaviors by optimizing environmental conditions to align with individual preferences, enhancing the likelihood of desired actions through unconscious influence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586174000006
    Figure 0007586174000006
  • Figure 0007586174000007
    Figure 0007586174000007
  • Figure 0007586174000008
    Figure 0007586174000008
Patent Text Reader

Abstract

Provided are a machine learning system, a machine learning device, and a machine learning method for performing machine learning of a correlation between behavior of a person and the environment surrounding the person so as to promote shift from behavior of the person to a target behavior. The present technology provides a machine learning system provided with at least: a state acquisition unit that at least acquires state information concerning behavior of a person; an evaluation unit that evaluates environment information concerning the environment surrounding the person at the time of acquisition of the state information and the state information so as to obtain a value function; and a machine learning classifier that, in order to encourage shift of behavior to a target behavior, performs reinforcement learning of the value function and selects the environment information when the value function become the highest.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present technology relates to a machine learning system, a machine learning device, and a machine learning method. [Background technology]

[0002] Conventionally, a technology has been used in which a computer uses machine learning to learn information about a person's behavior, thereby encouraging that behavior to become a target behavior. For example, Patent Document 1 discloses a "sales promotion system that utilizes a computer network to provide consumers with sales promotion information that induces consumption and promotes sales." This Patent Document 1 explains how machine learning is performed based on the behavior of consumers after the sales promotion information is provided. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-028899 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, in order to encourage a person's behavior to become a target behavior, a technique for providing information to a person in a direct manner, such as the provision of sales promotion information described in Patent Document 1, has been used.

[0005] However, some behavioral changes occur unconsciously in response to changes in the environment around a person.

[0006] Therefore, the main objective of this technology is to provide a machine learning system, a machine learning device, and a machine learning method that encourage a person's behavior to become a target behavior by machine learning the correlation between a person's behavior and the environment around that person. [Means for solving the problem]

[0007] The present technology provides a machine learning system including at least a state acquisition unit that acquires at least state information regarding a person's behavior, an evaluation unit that evaluates environmental information regarding the environment around the person at the time the state information was acquired and the state information to obtain a value function, and a machine learning classifier that performs reinforcement learning of the value function and selects the environmental information when the value function is highest in order to encourage the behavior to become a target behavior. The evaluation unit may calculate a reward based on a difference between target state information regarding the target behavior and the state information, and calculate the value function based on the reward, the environmental information, and the state information. The machine learning system may hold target state related information including a plurality of target behavior information. The target state related information may include time-specific target state information and / or stage-specific target state information. The environmental information may include information regarding scent, lighting, temperature, humidity, video, or audio. The device may further include a scent control unit, which controls the scent to be generated based on the environmental information selected by the machine learning classifier. The device may further include a fragrance attachment unit, which attaches a fragrance to an object based on the environmental information selected by the machine learning classifier, and which of the fragrance control unit and the fragrance attachment unit will emit a fragrance based on the environmental information. The device may further include a lighting control unit, and the lighting control unit may control the light to be emitted based on the environmental information selected by the machine learning classifier. The device may further include an air conditioning unit, and the air conditioning unit may control temperature and / or humidity based on the environmental information selected by the machine learning classifier. The device may further include an image control unit, and the image control unit may control the image to be displayed based on the environmental information selected by the machine learning classifier. The device may further include an audio control unit, and the audio control unit may control the audio to be played based on the environmental information selected by the machine learning classifier. The value functions are divided into a plurality of value groups, and the machine learning classifier can use the value functions possessed by each of the plurality of value groups. The system may further include a plurality of state acquisition units and an achievement difficulty calculation unit, and the achievement difficulty calculation unit may calculate an achievement difficulty for the target behavior based on state information acquired by each of the plurality of state acquisition units. The achievement difficulty may include an achievement rate indicating the degree to which the target behavior has been encouraged. The achievement difficulty may include a standard achievement time indicating a standard time required to prompt the target behavior. The achievement difficulty may include an average number of environmental items indicating the average number of items of the environmental information when the target behavior is encouraged. The present technology also provides a machine learning device including at least a state acquisition unit that acquires at least state information regarding human behavior, an evaluation unit that evaluates environmental information regarding the environment around the person at the time the state information is acquired and the state information to obtain a value function, and a machine learning classifier that performs reinforcement learning of the value function and selects the environmental information when the value function is highest in order to encourage the behavior to become a target behavior. The present technology also provides a machine learning method that includes at least acquiring state information regarding a person's behavior, evaluating the state information and environmental information regarding the environment around the person at the time the state information is acquired to obtain a value function, and performing reinforcement learning of the value function to encourage the behavior to become a target behavior, and selecting the environmental information when the value function is highest. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of a machine learning system 1 according to an embodiment of the present technology. [Diagram 2]FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Diagram 3] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Figure 4] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Diagram 5] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Figure 6] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Figure 7] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Figure 8] FIG. 1 is a conceptual diagram showing an example of behavioral modification according to an embodiment of the present technology. [Figure 9] 1 is a database showing an example of target state related information according to an embodiment of the present technology; [Figure 10] FIG. 1 is a block diagram showing a hardware configuration of a machine learning device 10 according to an embodiment of the present technology. [Figure 11] FIG. 1 is a block diagram showing a configuration of a machine learning system 1 according to an embodiment of the present technology. [Figure 12] 2 is a block diagram showing a configuration of a scent control unit 23 according to an embodiment of the present technology. FIG. [Figure 13] FIG. 1 is a block diagram showing a configuration of a machine learning system 1 according to an embodiment of the present technology. [Figure 14] 1 is a flowchart showing a procedure of a machine learning system 1 according to an embodiment of the present technology. [Figure 15] 2 is a block diagram showing a configuration of an illumination control unit 24 according to an embodiment of the present technology. FIG. [Figure 16] 2 is a block diagram showing a configuration of an air conditioning unit 25 according to an embodiment of the present technology. FIG. [Figure 17] 2 is a block diagram showing a configuration of a video control unit 26 according to an embodiment of the present technology. [Figure 18] 2 is a block diagram showing a configuration of a sound control unit 27 according to an embodiment of the present technology. FIG. [Figure 19] 2 illustrates an example of a database used by the machine learning device 10 according to an embodiment of the present technology. [Figure 20] 1 is a flowchart showing an example of a procedure of a machine learning device 10 according to an embodiment of the present technology. [Figure 21] FIG. 1 is a block diagram showing a configuration of a machine learning system 1 according to an embodiment of the present technology. [Figure 22] 5 is a diagram illustrating an achievement difficulty level calculated by an achievement difficulty level calculation unit 54 according to an embodiment of the present technology. FIG. [Figure 23] 5 is a diagram illustrating an achievement difficulty level calculated by an achievement difficulty level calculation unit 54 according to an embodiment of the present technology. FIG. [Figure 24] 1 is a flowchart showing the steps of a machine learning method according to an embodiment of the present technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A preferred embodiment for carrying out the present technology will be described below. The embodiment described below shows an example of a typical embodiment of the present technology, and the scope of the present technology is not narrowly interpreted by this. In addition, each figure is a schematic diagram, and is not necessarily illustrated strictly.

[0010] The present technology will be described in the following order. 1. First embodiment of the present technology (machine learning system example 1) (1) Overview (2) Evaluation section (3) Machine learning classifier (4) Flow of behavior change (5) Multiple target status information (6) Hardware Configuration 2. Second embodiment of the present technology (machine learning system example 2) (1) Overview (2) Scent control unit (3) Fragrance attachment part (4) Lighting control unit (5) Air conditioning unit (6) Video control unit (7) Audio control section 3. Third embodiment of the present technology (machine learning system example 3) 4. Fourth embodiment of the present technology (machine learning system example 4) 5. Fifth embodiment of the present technology (machine learning system example 5) (1) Overview (2) Difficulty of achievement 6. Sixth embodiment of the present technology (machine learning method)

[0011] [1. First embodiment of the present technology (example 1 of machine learning system)] [(1) Overview] A machine learning system according to an embodiment of the present technology can obtain a correlation between a behavior and an environment by evaluating information about a person's behavior and information about the environment around the person and performing machine learning, thereby making it possible to encourage a person's behavior to become a target behavior by controlling the environment.

[0012] A configuration of a machine learning system according to an embodiment of the present technology will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a configuration of a machine learning system 1 according to an embodiment of the present technology.

[0013] 1, the machine learning system 1 may include, for example, a machine learning device 10. The machine learning device 10 may include, for example, a state acquisition unit 11, an evaluation unit 12, a recording unit 13, and a machine learning classifier 14.

[0014] The state acquisition unit 11 acquires at least state information related to the behavior of a person, which enables the machine learning system 1 to grasp changes in the behavior of a person in response to changes in the environment.

[0015] Status information includes, for example, cookies used for browsing websites, purchase history on EC (Electronic Commerce) sites, location information obtained using GPS (Global Positioning System), conversation history such as chat, and other information obtained using sensing technologies.

[0016] Furthermore, the state information may include information about the weather or temperature of the area where the person is located, which allows the machine learning system 1 to grasp behavioral changes specific to the weather, temperature, or the like of the area where the person is located.

[0017] The evaluation unit 12 evaluates the state information and environmental information related to the environment around the person when the state information was acquired to obtain a value function. This allows the correlation between the environmental information and the state information to be obtained. Specific evaluation means will be described later.

[0018] The environmental information includes, for example, information related to scent, lighting, temperature, humidity, video, audio, etc. Specific examples of the environmental information will be described later.

[0019] The recording unit 13 records state information, environmental information, etc. Note that the machine learning system 1 acquires state information but does not acquire and uses recorded environmental information.

[0020] In order to encourage human behavior to become a target behavior, the machine learning classifier 14 performs reinforcement learning of a value function and selects the environmental information when the value function is highest, thereby obtaining a correlation between behavior and the environment.

[0021] The machine learning method is not particularly limited, but for example, reinforcement learning can be used. Reinforcement learning is a type of machine learning that deals with the problem of software observing a current state (state information in this technology) and determining an action to be taken (change in environmental information in this technology). An agent (machine learning classifier 14 in this technology) can determine the action that will be most valuable by performing reinforcement learning through trial and error of actions.

[0022] Conventionally, methods for implementing reinforcement learning include, for example, the Monte Carlo method, dynamic programming, SARSA (state-action-reward-state-action), and Q-learning. This technology will be described using Q-learning, which is an example of reinforcement learning. Note that reinforcement learning methods other than Q-learning may be used in this technology.

[0023] Although not shown in the figure, the machine learning device 10 may also have a control unit that controls each of the components, a communication interface that performs communication via a communication network, and the like.

[0024] [(2) Evaluation Department] As described above, the evaluation unit 12 evaluates the state information and environmental information relating to the environment around the person when the state information is acquired, to obtain a value function.

[0025] The means for achieving this is not particularly limited, but for example, the evaluation unit 12 can have a remuneration calculation unit (not shown) and a value calculation unit (not shown).

[0026] State information regarding human behavior may change according to changes in environmental information. The reward calculation unit calculates a reward R based on the difference between target state information regarding the target behavior and the state information when the machine learning system 1 changes the environmental information. The larger the value of reward R, the smaller the difference between the target state information and the state information. In other words, the larger the value of reward R, the closer the human behavior is to the target behavior.

[0027] The reward R can be expressed, for example, by the following formula (1) using a score Pt according to the target state information and a score Pm according to the state information.

[0028]

number

[0029] Let us explain this using a concrete example. The target behavior is set to "purchase product A using an EC site." When a person performs the target behavior due to a change in environmental information, a score of 5 points is set.

[0030] In addition, when a change in the environmental information causes a person to take an action close to the target action, such as "viewing a website where product A is listed," a score of 2 points is set.

[0031] When applied to the above formula (1), the score Pt according to the target state information is "5". When a person performs the target behavior due to a change in the environmental information, the score Pm according to the state information is also "5". In this case, the reward R is "1".

[0032] When a person behaves close to the target behavior due to a change in the environmental information, the score Pm according to the state information is "2". In this case, the reward R is "0.4".

[0033] When a person behaves in a way that is not one of these two ways due to a change in the environmental information, the score Pm according to the state information is set to "0." In this case, the reward R is also set to "0."

[0034] From this, it can be seen that the closer the person's behavior due to the change in environmental information is to the target behavior, the higher the value of the reward R. The reward calculation unit calculates the reward R when the change in environmental information is made such that the reward R is the highest.

[0035] The value calculation unit calculates the value function Q based on the reward R, the environmental information, and the state information. The value calculation unit calculates the value function Q based on the state information when the environmental information is changed so that the reward R is the highest. For example, the state information s t Changes in environment information for a t When this is done, the value function is Q(s t ,a t ) is expressed as

[0036] This value function Q may be recorded, for example, by the recording unit 13. More specifically, the recording unit 13 may record the value function Q in a table for each piece of state information or environmental information.

[0037] [(3) Machine learning classifier] As described above, the machine learning classifier 14 performs reinforcement learning on the value function Q and selects the environmental information when the value function Q is the highest.

[0038] This reinforcement learning will now be described. The machine learning classifier 14 automatically performs trial and error and learning so that the value function Q becomes the highest. The higher the value function Q, the closer the person's behavior is to the target behavior. By performing reinforcement learning so that the value function Q becomes the highest, the machine learning classifier 14 can encourage the person's behavior to become the target behavior.

[0039] When the environmental information when the value function Q is highest is determined, the machine learning classifier 14 updates the value function Q. For example, the state information s t Changes to environment information a t The state information s t+1 When the transition occurs, the value function Q(s t ,a t ) is updated using the following equation (2):

[0040]

number

[0041] Here, α is the learning coefficient. The learning coefficient α is set to a value greater than 0 and less than or equal to 1, but a value of around 0.1 is often used.

[0042] R t+1 is the reward obtained by this state information transition.

[0043] γ is the discount rate. The discount rate γ is set to a value greater than 0 and less than or equal to 1, but values ​​of around 0.9 to 0.99 are often used.

[0044] maxQ(s t+1 ,a) is the ideal future value function. maxQ(s t+1 ,a) is the state s at time t+1 t+1 This is the value function when action a is selected when the value function Q is highest. t+1 ,a) is multiplied by the discount rate γ.

[0045] The machine learning classifier 14 continues to update the value function Q using the above formula (2), and selects the environmental information when the value function Q is the highest. In this way, the machine learning classifier 14 can select environmental information that can encourage the target behavior.

[0046] [(4) Flow of behavior change] It is assumed that multiple behavioral changes are made before the target behavior is reached. This will be described with reference to FIG. 2. FIG. 2 is a conceptual diagram showing an example of behavioral change according to this embodiment. As shown in FIG. 2, the target behavior is set to "view a video on a video sharing site." Then, multiple behavioral change flows are set to prompt this target behavior. The machine learning classifier 14 can configure this behavioral change flow by repeating reinforcement learning. By following this flow, the machine learning classifier 14 can prompt a person's behavior to the target behavior.

[0047] In addition, human behaviors are divided into a plurality of levels according to their closeness to the target behavior. For example, a behavior set to the first level may be the behavior closest to the target behavior. A behavior set to the second level may be the behavior next closest to the target behavior. The value function Q associated with the behaviors in the first level is higher than the value function Q associated with the behaviors in the second level.

[0048] In this example, the actions "went to the toilet" and "sat on the sofa" are set at the first level. The actions "child went to bed", "went home", "left the table", and "drank alcohol" are set at the second level. A flow of behavioral change is formed by connecting multiple actions. For example, the characteristics of this individual's behavioral change show that when the action "child goes to bed" is performed, the action "went to the toilet" tends to be performed.

[0049] Other examples of the flow of this behavioral change are shown in Figs. 3 to 7. Figs. 3 to 7 are conceptual diagrams showing examples of behavioral change according to this embodiment. Fig. 3 shows an example of a flow of behavioral change for encouraging a target behavior of "using an EC site". Fig. 4 shows an example of a flow of behavioral change for encouraging a target behavior of "browsing an SNS (Social Networking Service)". Fig. 5 shows an example of a flow of behavioral change for encouraging a target behavior of "making an expensive purchase". Fig. 6 shows an example of a flow of behavioral change for encouraging a target behavior of "drinking beer". Fig. 7 shows an example of a flow of behavioral change for encouraging a target behavior of "going to bed". In this way, various flows of behavioral change are configured according to the target behavior.

[0050] Furthermore, even if the target behavior is the same, the flow of behavioral change to encourage the target behavior may differ from person to person. This will be described with reference to FIG. 8. FIG. 8 is a conceptual diagram showing an example of behavioral change according to this embodiment. FIG. 8A shows an example of behavioral change for a certain individual. As shown in FIG. 8A, the behavior "purchasing clothes" is set as the target behavior.

[0051] On the other hand, Figure 8B shows an example of behavioral change for an individual other than the individual shown in Figure 8A. As shown in Figure 8B, the target behavior is the same as that in Figure 8A, but the flow of behavioral change to promote the target behavior is different.

[0052] [(5) Multiple target status information] The machine learning device 10 according to an embodiment of the present technology may have target state information related to one target behavior, or may have multiple target state information related to multiple target behaviors. Some or all of the multiple target behaviors may be set by time and / or by stage, for example.

[0053] A part or all of the multiple target actions can be set by time, for example. To explain using a specific example, a part or all of the multiple target actions can be set as target actions in a first time period (e.g., 0:00-6:00 AM), a target action in a second time period (e.g., 7:00 AM-7:00 PM), and a target action in a third time period (e.g., 8:00 PM-11:00 PM).

[0054] A target behavior in a first time slot (e.g., 12:00-6:00 AM) may be, for example, "sleep". A target behavior in a second time slot (e.g., 7:00 AM-7:00 PM) may be, for example, "eat food S". A target behavior in a third time slot (e.g., 8:00 PM-11:00 PM) may be, for example, "drink beverage T".

[0055] By setting some or all of the multiple target behaviors by time, the target behaviors can be flexibly set according to, for example, a time period. For example, the machine learning device 10 can encourage the target behavior of "eating food S at 3 p.m."

[0056] Alternatively, some or all of the multiple target actions can be set, for example, by stage. To explain using a specific example, some or all of the multiple target actions can be set as target actions in a first stage and target actions in a second stage. The target action in the first stage can be, for example, "go to store U". The target action in the second stage can be, for example, "eat food S".

[0057] By setting some or all of the multiple target behaviors in stages, for example, multiple target behaviors having a series of flows can be set. For example, the machine learning device 10 can encourage the target behavior of "eat food S when you go to store U."

[0058] Alternatively, some or all of the target behaviors may be set, for example, by time and by stage. To explain using a specific example, in the third time period, target behaviors in the first and second stages may be set.

[0059] By setting some or all of the multiple target behaviors by time and by stage, multiple target behaviors having a series of flows can be flexibly set according to time periods. For example, the machine learning device 10 can encourage a target behavior such as "eat food S when you go to store U in the morning."

[0060] To achieve this, the machine learning device 10 according to an embodiment of the present technology may hold target state related information including a plurality of pieces of target state information.

[0061] The target state related information will be described with reference to FIG. 9. FIG. 9 is a database showing an example of the target state related information according to this embodiment. In FIG. 9, time periods and target actions are associated with each other. The target action in the first time period (12:00-6:00 AM) is "going to bed". The target action in the second time period (7:00 AM-7:00 PM) is "eating food S". The target action in the third time period (8:00 PM-11:00 PM) is, for example, "drinking beverage T". The same applies to the target state related information set by stage.

[0062] The target state related information can be recorded, for example, in a recording unit 13 included in the machine learning device 10. Note that the target state related information may be held by a computer device other than the machine learning device 10. For example, the target state related information may be held by a server on the cloud. In this case, the machine learning device 10 may receive the target state related information from the server via an information and communication network.

[0063] (6) Hardware configuration The hardware configuration of the machine learning device 10 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the hardware configuration of the machine learning device 10 according to this embodiment. As shown in Fig. 10, the machine learning device 10 can include, as components, a CPU 101, a storage 102, a RAM (Random Access Memory) 103, and a communication interface 104. The respective components are connected by, for example, a bus serving as a data transmission path.

[0064] The CPU 101 is realized by, for example, a microcomputer, and controls each component of the machine learning device 10. The CPU 101 can function, for example, as the evaluation unit 12 and the machine learning classifier 14. This machine learning classifier 14 can be realized by, for example, a program. The CPU 101 can function by reading this program.

[0065] The storage 102 stores control data such as programs and computation parameters used by the CPU 101. The storage 102 can be realized by using, for example, a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage 102 can function as, for example, the recording unit 13.

[0066] The RAM 103 temporarily stores, for example, the programs executed by the CPU 101.

[0067] The communication interface 104 has a function of communicating via an information and communication network using communication technologies such as Wi-Fi, Bluetooth (registered trademark), and LTE (Long Term Evolution).

[0068] The program for implementing the machine learning classifier 14 and the like may be stored in a computer device or computer system other than the machine learning system 1. In this case, the machine learning system 1 can use a cloud service that provides the functions of the program. Examples of such cloud services include SaaS (Software as a Service), IaaS (Infrastructure as a Service), and PaaS (Platform as a Service).

[0069] Furthermore, the program can be stored and provided to a computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), Compact Disc Read Only Memory (CD-ROM), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)). The program may also be provided to a computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electric signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0070] [2. Second embodiment of the present technology (machine learning system example 2)] [(1) Overview] The machine learning system 1 according to an embodiment of the present technology may include an environmental control device for controlling the environment around the person. The environmental control device controls the environment around the person based on the environmental information selected by the machine learning classifier 14. In this way, the machine learning system 1 can encourage the behavior of the person to become a target behavior.

[0071] The machine learning system 1 can promote sales, for example, by encouraging human behavior to become a target behavior. The machine learning system 1 can control the environment inside or outside a store to encourage customers to purchase products. Note that the use of the machine learning system 1 is not limited to brick-and-mortar stores, and may be, for example, an e-commerce site. Alternatively, the machine learning system 1 can be used for downloaded or streamed content and websites to promote browsing of the content or website.

[0072] Alternatively, the machine learning system 1 can improve, for example, a person's lifestyle by encouraging the person's behavior to become a target behavior. Specifically, the machine learning system 1 can control the environment to encourage smoking cessation, alcohol abstinence, etc. Furthermore, the machine learning system 1 may be used to improve, for example, shopping addiction, lack of sleep, lack of exercise, etc.

[0073] Alternatively, the machine learning system 1 can encourage a person's behavior to become a target behavior, for example, to vote in an election.

[0074] Alternatively, the machine learning system 1 can improve, for example, public health and morals by encouraging people to behave in a target manner. Specifically, the machine learning system 1 may be used for separating garbage, preventing littering and rushing to board trains, complying with cough etiquette and hand washing, lining up, preventing violence, and the like.

[0075] Alternatively, the machine learning system 1 can improve, for example, work efficiency by encouraging a person's behavior to become a target behavior. Specifically, the machine learning system 1 may be used to improve concentration, suppress distraction, switch on and off, and the like.

[0076] The configuration of a machine learning system 1 according to this embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the configuration of a machine learning system 1 according to this embodiment. Note that the same components as those in the first embodiment are given the same reference numerals, and detailed descriptions thereof will be omitted.

[0077] 11, the machine learning system 1 according to this embodiment may further include an environmental control device 20. The environmental control device 20 and the machine learning device 10 are connected via an information and communication network 40. The environmental control device 20 controls the environment, such as the scent, lighting, temperature, humidity, video, or sound, based on the environmental information selected by the machine learning classifier 14.

[0078] The environmental control device 20 may have, for example, a communication control unit 21, a memory 22, a scent control unit 23, a lighting control unit 24, an air conditioning unit 25, a video control unit 26, and an audio control unit 27.

[0079] In addition, with regard to the fragrance control unit 23, the lighting control unit 24, the air conditioning unit 25, the video control unit 26, and the audio control unit 27, the environmental control device 20 does not need to have all of these components, but rather needs to have at least one of these components.

[0080] Furthermore, the machine learning system 1 may include a plurality of environmental control devices 20. For example, when the machine learning system 1 includes two environmental control devices 20, one of the environmental control devices 20 may include a scent control unit 23, and the other environmental control device 20 may include a lighting control unit 24.

[0081] The communication control unit 21 can communicate information with the machine learning device 10 via an information and communication network 40. The communication control unit 21 may also control each of the components.

[0082] The memory 22 can record information used by the environmental control device 20, such as environmental information.

[0083] The machine learning classifier 14 and other components included in the machine learning device 10 may be included in, for example, the environmental control device 20, or may be included in another computer device.

[0084] [(2) Scent control unit] In order to control the scent around the person, the environmental control device 20 can include, for example, a scent control unit 23. The scent control unit 23 controls the scent to be generated based on the environmental information selected by the machine learning classifier 14. The environmental control device 20 including the scent control unit 23 can be realized by using, for example, an aroma diffuser.

[0085] Incidentally, aromas include not only those that humans can recognize as aromas, but also those that humans cannot recognize as aromas but have some effect on humans when inhaled. For example, aromas include inhaled medical sedatives, and odorless gases such as oxygen and carbon dioxide that have an effect on a person's physical condition when inhaled.

[0086] When a person inhales the scent optimized for that person, controlled by the scent control unit 23, the person is unconsciously prompted to perform the target behavior.

[0087] The configuration of the aroma control unit 23 will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the configuration of the aroma control unit 23 according to this embodiment. As shown in Fig. 12, the aroma control unit 23 can have, for example, an added component cartridge 231, an aroma control unit 232, and an aroma output unit 233.

[0088] The additive component cartridge 231 is a component that contains the additive component of the fragrance. The additive component cartridge 231 may be replaceable. The additive component cartridge 231 may be, for example, a container such as a cylinder, bottle, or can that contains the additive component, a material such as paper, nonwoven fabric, or stone that adsorbs the additive component, or a solid material such as wax or soap that contains the additive component.

[0089] The additive component may be a solid, liquid, or gas, including, for example, a powder or gel, or a mixture thereof. The additive component may be, for example, a natural flavor extracted from a natural source, a synthetic flavor obtained by chemical synthesis, or a blended flavor obtained by blending these. Alternatively, the additive component may not include a flavor.

[0090] The scent control unit 232 controls the additive components for generating the scent based on the environmental information. For example, the scent control unit 232 can determine the ratio of each additive component when the additive components are mixed. Alternatively, the scent control unit 232 can determine a dilution ratio. The ratio or the dilution ratio is determined according to the environmental information selected by the machine learning classifier 14.

[0091] Alternatively, the scent control unit 233 may control the scent output, such as the spray pressure or the number of sprays. The spray pressure or the number of sprays is determined according to the environmental information selected by the machine learning classifier 14.

[0092] The fragrance output unit 233 outputs a fragrance based on the information determined by the fragrance control unit 232.

[0093] The machine learning system 1 equipped with the scent control unit 23 can encourage a person to take a target action by having the person inhale a specific scent. For example, by having the person inhale a scent that physiologically makes them want to purchase a specific product, the machine learning system 1 can make the person purchase a specific product through mail order or go to a store.

[0094] Alternatively, the scent and the content may be associated with each other. In this way, the machine learning system 1 can allow a person to unconsciously learn the association between the scent and the content before performing reinforcement learning.

[0095] For example, the machine learning system 1 can have a person inhale a specific scent and watch specific video content. This video content includes, for example, an advertisement for a specific product. In this way, the machine learning system 1 can have the person unconsciously learn the association between the specific scent and the specific product. By having the person inhale this specific scent, the machine learning system 1 can have the person purchase a specific product through mail order or go to a store.

[0096] Alternatively, the scent and the environment may be associated. The environment is an environment related to a place or object that a person actually experiences. The place includes, for example, a store, public transportation, a movie theater, a theater, a theme park, etc. In this way, the machine learning system 1 can allow a person to unconsciously learn the association between the scent and the environment before performing reinforcement learning.

[0097] For example, the machine learning system 1 can make a person who visits a particular store inhale a particular scent. In this way, the machine learning system 1 can make the person unconsciously learn an association between a particular scent and a particular store. By making the person inhale this particular scent in a place other than the store, the machine learning system 1 can make the person go to the store or purchase an item displayed in the store via mail order.

[0098] For example, the machine learning system 1 can cause a person to inhale the aroma emanating from a specific product through an experience such as drinking coffee. In this way, the machine learning system 1 can cause a person to unconsciously learn the association between a specific aroma and a specific product. By having the aroma control unit 23 generate this specific aroma, the machine learning system 1 can cause the person to purchase this specific product through mail order or to go to a store where this specific product is displayed.

[0099] [(3) Fragrance attachment part] The scent may be attached to an object, such as clothing, books, miscellaneous goods, promotional materials, packaging materials, etc., that is delivered to a person to encourage the target behavior. The person is unconsciously encouraged to engage in the target behavior by inhaling the scent that is attached to the object and optimized for that person.

[0100] To achieve this, the machine learning system 1 can be provided with a scent attachment unit. This will be described with reference to FIG. 13. FIG. 13 is a block diagram showing the configuration of the machine learning system 1 according to this embodiment. Note that the same components as those in the first embodiment are given the same reference numerals and detailed descriptions will be omitted. As shown in FIG. 13, the machine learning system 1 further includes a scent attachment unit 30. The scent attachment unit 30 and the machine learning device 10 are connected via an information and communication network 40. Note that the scent attachment unit 30 may have the same configuration as the scent control unit 23.

[0101] The above-mentioned scent control unit 23 is placed around the person who is prompted to perform the target behavior. On the other hand, the scent attachment unit 30 is placed, for example, in a factory that ships objects. The scent attachment unit 30 attaches a scent to an object based on the environmental information selected by the machine learning classifier 14.

[0102] The machine learning classifier 14 determines, based on the environmental information, whether the scent control unit 23 or the scent attachment unit 30 will emit a scent.

[0103] The procedure of the machine learning system 1 at this time will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the procedure of the machine learning system 1 according to this embodiment.

[0104] 14, in the initial stage of machine learning, the machine learning classifier 14 reinforces learning of the correlation between scent and behavior (step S11), the machine learning classifier 14 selects environmental information (step S12), the scent control unit 23 determines that it will generate a scent (step S13), and the scent control unit 23 controls the scent around the person (step S14). Steps S11 to S14 are repeated until the correlation between scent and behavior is sufficiently learned (step S15: No).

[0105] When the correlation between scent and behavior has been sufficiently learned (step S15: Yes), the machine learning classifier 14 determines that the scent attachment unit 30 will emit a scent (step S16), and the scent attachment unit 30 attaches the scent to an object (step S17).

[0106] This allows the machine learning system 1 to more flexibly control the scent around a person. For example, in the early stages of machine learning (e.g., about 1 to 3 months after the start of learning), the scent control unit 23 placed around the person changes the scent in a short period of time, while the machine learning classifier 14 performs reinforcement learning of the correlation between scent and behavior with high efficiency. The machine learning classifier 14 determines the scent that is optimal for the target behavior.

[0107] After that, the target behavior can be continuously encouraged by delivering an item with an optimal scent for the target behavior. While the scent attachment unit 30 changes the scent over a long period of time, the machine learning classifier 14 continues to perform reinforcement learning of the correlation between scents and behaviors.

[0108] [(4) Lighting control section] Returning to the description of Fig. 11, in order to control the lighting around the person, the environmental control device 20 can include, for example, a lighting control unit 24. The lighting control unit 24 controls the light to be emitted based on the environmental information selected by the machine learning classifier 14. The environmental control device 20 including the lighting control unit 24 can be realized by using, for example, a light bulb (including a so-called smart light bulb).

[0109] When a person sees the light that is optimized for that person and is emitted by the lighting control unit 24, the person is unconsciously prompted to perform the target behavior.

[0110] The configuration of the illumination control unit 24 will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the configuration of the illumination control unit 24 according to this embodiment. As shown in Fig. 15, the illumination control unit 24 can have, for example, a light control unit 241 and a light output unit 242.

[0111] The light control unit 241 controls the expression of the light to be output. Specifically, the light control unit 241 can determine, for example, the color temperature or brightness of the light. The color temperature or brightness is determined according to the environmental information selected by the machine learning classifier 14. For example, the color temperature may be determined as 3500 to 3900K, and the brightness may be determined as 3000 to 4000lm. If the values ​​determined in this way have a range, the light control unit 241 may randomly determine a value from this range. The machine learning device 10 can narrow this range while accumulating reinforcement learning. The same applies to other components described below.

[0112] The light output unit 242 outputs light based on the information determined by the light control unit 241 .

[0113] [(5) Air conditioning unit] In order to adjust the air around the person, the environmental control device 20 may include, for example, an air conditioning unit 25. The air conditioning unit 25 controls the temperature and / or humidity based on the environmental information selected by the machine learning classifier 14. The environmental control device 20 including the air conditioning unit 25 can be realized by using, for example, an air conditioner.

[0114] A person is unconsciously encouraged to engage in the target behavior by the temperature and / or humidity optimized for that person, which is controlled by the air conditioning unit 25.

[0115] The configuration of the air conditioning unit 25 will be described with reference to Fig. 16. Fig. 16 is a block diagram showing the configuration of the air conditioning unit 25 according to this embodiment. As shown in Fig. 16, the air conditioning unit 25 can have, for example, an air control unit 251 and an air output unit 252.

[0116] The air control unit 251 can determine the temperature and / or humidity of the air. The temperature and / or the humidity are determined according to the environmental information selected by the machine learning classifier 14. For example, the temperature may be determined to be 25.5 to 27.5 degrees, and the humidity may be determined to be 45 to 50%, for example.

[0117] The air output unit 252 outputs air based on the information determined by the air control unit 251 .

[0118] [(6) Video control unit] In order to control the image displayed to the person, the environmental control device 20 may include, for example, an image control unit 26. The image control unit 26 controls the image to be displayed based on the environmental information selected by the machine learning classifier 14. The environmental control device 20 including the image control unit 26 can be realized by using, for example, a television, a portable game console, a PC, a tablet, a smartphone, an HMD (Head Mounted Display), a wearable device, a car navigation system, or the like.

[0119] Note that video includes still images as well as moving images. Video may also include audio.

[0120] When a person views the image optimized for that person displayed by the image control unit 26, the person is unconsciously prompted to perform the target behavior.

[0121] The configuration of the video control unit 26 will be described with reference to Fig. 17. Fig. 17 is a block diagram showing the configuration of the video control unit 26 according to this embodiment. As shown in Fig. 17, the video control unit 26 can have, for example, a video selection unit 261 and a video display unit 262.

[0122] The video selection unit 261 selects a video to be output. The means of this selection is not particularly limited, but for example, the video selection unit 261 can determine the video using an address where a video file is recorded, a code of an advertisement banner, or the like. The address, code, or the like is determined according to the environmental information selected by the machine learning classifier 14. The video selection unit 261 may also synthesize or edit a plurality of video files. Furthermore, the video selection unit 261 may adjust the color temperature, brightness, or the like of the video.

[0123] The video file may be recorded inside the video control unit 26 or outside the video control unit 26.

[0124] The video display unit 262 outputs a video based on the information determined by the video selection unit 261 .

[0125] [(7) Audio control section] In order to control the sound played back to a person, the environmental control device 20 may include, for example, a sound control unit 27. The sound control unit 27 controls the sound to be played back based on the environmental information selected by the machine learning classifier 14. The environmental control device 20 including the sound control unit 27 can be realized by using, for example, a speaker (including a so-called smart speaker or a speaker with a streaming function), a tablet, a smartphone, headphones, a wearable device, a car stereo, or the like.

[0126] When a person listens to the audio that is optimized for that person and is reproduced by the audio control unit 27, the person is unconsciously prompted to perform the target behavior.

[0127] The configuration of the audio control unit 27 will be described with reference to Fig. 18. Fig. 18 is a block diagram showing the configuration of the audio control unit 27 according to this embodiment. As shown in Fig. 18, the audio control unit 27 can have, for example, an audio selection unit 271 and an audio output unit 272.

[0128] The audio selection unit 271 selects an audio to be played. The means of this selection is not particularly limited, but for example, the audio selection unit 271 can determine the audio using an address where an audio file is recorded, a code of an advertisement banner, or the like. The address, code, or the like is determined according to the environmental information selected by the machine learning classifier 14. The audio selection unit 271 may also synthesize or edit a plurality of audio files. Furthermore, the audio selection unit 271 may adjust the pitch, volume, or the like.

[0129] The audio file may be recorded inside the audio control unit 27 or outside the audio control unit 27.

[0130] The audio output unit 272 outputs audio based on the information determined by the audio selection unit 271 .

[0131] [3. Third embodiment of the present technology (machine learning system example 3)] The machine learning device 10 according to an embodiment of the present technology can record the value function Q, the state information s, and the change a in the environmental information for each target behavior. Then, the machine learning device 10 can select environmental information that can encourage the target behavior by performing reinforcement learning of the correlation between a person's behavior and the environment around the person.

[0132] In this case, a plurality of people who have similar correlations between behavior and the environment can be set as one value group. For example, a plurality of people who are likely to be prompted to a specific target behavior when sensing a change in scent and temperature can be set as one group.

[0133] This will be described with reference to FIG. 19. FIG. 19 is an example of a database used by the machine learning device 10 according to this embodiment. As shown in FIG. 19, this database holds a value function Q, state information s, and change a in environmental information for each target behavior. This value function Q is divided into a plurality of value groups Q1 to Q8 according to the degree of similarity of the correlation between the state information s and the change a in environmental information. People h01 to h32 belonging to each value group are associated with each of the plurality of value groups Q1 to Q8. Attribute information A to C indicating the characteristics of each person may be associated with each person.

[0134] When the value group to which the subject of reinforcement learning belongs is known, the machine learning device 10 can use information related to this value group, such as the value function Q. By using information such as the value function Q that has already been reinforced learned, the machine learning device 10 can, for example, omit part of the reinforcement learning process and perform reinforcement learning in a shorter period of time.

[0135] Let us use a concrete example. Let us assume that the target behavior for a certain person is set as "playing video games." Let us assume that through reinforcement learning, we know that this person tends to play video games when influenced by scent and temperature changes.

[0136] Next, assume that the target behavior is changed from "playing video games" to "drinking beer." However, it is known from the reinforcement learning so far that this person is easily influenced by scent and temperature changes. From this, information such as the value function Q of the value group that is easily influenced by scent and temperature changes and the change a in the environmental information that has previously provided a high reward can be used as the initial value of the reinforcement learning to encourage a new target behavior. The machine learning device 10 can start the reinforcement learning by setting the information that has already been reinforced learned as the initial value.

[0137] Furthermore, information such as the value function Q that has already been reinforced learned may be used for reinforcement learning regarding the behavior of another person who belongs to the same value group. For example, information such as the value function Q that has been reinforced learned regarding the behavior of person h13 who belongs to value group Q4 can be used for reinforcement learning regarding the behavior of person h14 who belongs to the same value group Q4.

[0138] The procedure of the machine learning device 10 according to this embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing an example of the procedure of the machine learning device 10 according to this embodiment. As shown in FIG. 20, the state acquisition unit 11 included in the machine learning device 10 acquires state information (step S21).

[0139] Next, the evaluation unit 12 included in the machine learning device 10 calculates a reward and a value function based on the state information (step S22).

[0140] Next, the machine learning classifier 14 included in the machine learning device 10 updates the value function (step S23).

[0141] Next, the machine learning classifier 14 selects environmental information for learning further behavioral changes (step S24).

[0142] Next, the machine learning classifier 14 determines whether or not a predetermined condition is satisfied (step S25). This determination condition is not particularly limited, but may be determined, for example, based on whether or not the number of updates to the value function exceeds a predetermined threshold.

[0143] When a predetermined condition is satisfied (step S25: Yes), the machine learning classifier 14 refers to the database to acquire information such as the value function Q of a similar group and the change a in the environmental information that has previously obtained a high reward (step S26). This database may be provided in the machine learning device 10, or in a computer device other than the machine learning device 10. This allows the machine learning device 10 to perform reinforcement learning using information that has already been reinforcement learned.

[0144] On the other hand, when the predetermined condition is not satisfied (step S25: No), the value function of the similar group is not obtained.

[0145] Next, the machine learning classifier 14 determines whether or not to end the reinforcement learning (step S27). The condition for this determination is not particularly limited, but may be, for example, whether or not the value function is higher than a predetermined threshold value.

[0146] When it is determined that the machine learning should not be ended (step S27: No), the procedure of steps S21 to S26 is repeated.

[0147] When it is determined that the machine learning is to be ended (step S27: Yes), the machine learning classifier 14 selects environmental information (step S28).

[0148] [4. Fourth embodiment of the present technology (machine learning system example 4)] The target behavior may be set randomly. By prompting various target behaviors without being fixed to a specific target behavior, the machine learning classifier 14 can reinforce learning the correlation between the behavior and the environment. Through this reinforcement learning, the machine learning classifier 14 can discover patterns such as signs of behavior and continuity, even in the case of environmental changes that are thought to have little correlation with the behavior.

[0149] The following description will be given with reference to FIG. 19 again. Each of the multiple people registered in this database is associated with attribute information indicating the characteristics of that person. For example, attribute A may be a person who has the characteristic "tends to drink beer when the color temperature of the lighting is 3650K, the brightness of the lighting is 3000lm, and the temperature is 26.5 degrees." For example, attribute C may be a person who has the characteristic "tends to browse EC sites when the temperature is 25 degrees, the humidity is 48%, and the fragrance contains additive component T."

[0150] To achieve this, in the flowchart shown in FIG. 20, for example, instead of obtaining a value function of a similar group (step S26), a value function of a randomly selected group may be obtained.

[0151] [5. Fifth embodiment of the present technology (machine learning system example 5)] [(1) Overview] The machine learning system 1 according to an embodiment of the present technology may include multiple machine learning devices. This will be described with reference to Fig. 21. Fig. 21 is a block diagram showing a configuration of the machine learning system 1 according to an embodiment of the present technology.

[0152] 21, the machine learning system 1 may include, for example, multiple machine learning devices 10a-10d. Each of the multiple machine learning devices 10a-10d may include, for example, state acquisition units 11a-11d, evaluation units 12a-12d, recording units 13a-13d, and machine learning classifiers 14a-14d. In addition, an environmental control device (not shown) may be connected to each of the multiple machine learning devices 10a-10d. The number of machine learning devices is not particularly limited.

[0153] Furthermore, the machine learning system 1 may include an achievement difficulty calculation device 50. The achievement difficulty calculation device 50 may have a hardware configuration as shown in FIG. 10. The achievement difficulty calculation device 50 is connected to each of the multiple machine learning devices 10a to 10d via, for example, an information and communication network 40, and can aggregate information obtained from each of the multiple machine learning devices 10a to 10d to obtain a tendency of a correlation between state information and environmental information. More specifically, the achievement difficulty calculation device 50 can calculate the achievement difficulty regarding the target behavior. When it is defined that the target behavior is achieved by being prompted, the achievement difficulty indicates the difficulty of this achievement. A detailed explanation of this achievement difficulty will be given later.

[0154] The achievement difficulty calculation device 50 can include, for example, an information acquisition unit 51, a subject information recording unit 52, a behavior information recording unit 53, and an achievement difficulty calculation unit .

[0155] The information acquiring unit 51 acquires state information acquired by each of the multiple machine learning devices 10a to 10d. The information acquiring unit 51 can be realized by using, for example, the communication interface 104 or the like.

[0156] The multiple machine learning devices 10a to 10d may each target different subjects. The subject information recording unit 52 holds information about the subjects that are the targets of each of the multiple machine learning devices 10a to 10d. This information includes, for example, the subject's identification number, gender, or age. The subject information recording unit 52 can be realized by using, for example, the storage 102.

[0157] The behavior information recording unit 53 holds information about behaviors targeted by each of the multiple machine learning devices 10a to 10d. This information includes, for example, information about the target behavior, state information, or history information about the state information. The behavior information recording unit 53 can be realized by using, for example, the storage 102.

[0158] The achievement difficulty calculation unit 54 can calculate the achievement difficulty for the target behavior based on the state information acquired by each of the multiple state acquisition units 11a to 11d. The achievement difficulty calculation unit 54 can be realized by using, for example, the CPU 101 and a program.

[0159] The achievement difficulty calculation unit 54 may be provided in the achievement difficulty calculation device 50, in each of the multiple machine learning devices 10a to 10d, or in each of the multiple environmental control devices (not shown).

[0160] Although not shown in the figures, the machine learning system 1 may include a plurality of achievement difficulty calculation devices. The plurality of achievement difficulty calculation devices may include a relay achievement difficulty calculation device that aggregates information obtained from a specific machine learning device among the plurality of machine learning devices.

[0161] [(2) Achievement difficulty] As described above, the achievement difficulty indicates the difficulty of encouraging the target behavior. By calculating the achievement difficulty, the machine learning system 1 can, for example, derive subjects who are likely or unlikely to be encouraged to the target behavior, or derive environmental information that indicates whether the subject is likely or unlikely to be encouraged to the target behavior.

[0162] A group of subjects who are easily prompted by the target behavior is defined as an adaptive group, and a group of subjects who are not easily prompted by the target behavior is defined as a challenge group. By deriving this adaptive group, the machine learning system 1 can derive a target demographic for a product, for example, in product development, advertising and promotion activities, etc. This target demographic includes, for example, age and gender. By deriving the target demographic for a product, the efficiency of product development, advertising and promotion activities, etc. can be improved.

[0163] For example, a POS (Point of Sale) system installed in a convenience store can be associated with a local event (such as an athletic meet or fireworks display). This allows the machine learning system 1 to derive products that are likely to be purchased at the time of the event, the target demographic of the products, and so on.

[0164] For example, there are products that become huge hits and products that sell well for a long time. Adaptive groups are used for the development and advertising activities of the former, while challenge groups are used for the development and advertising activities of the latter.

[0165] Furthermore, target behaviors can be classified into basic target behaviors that are fundamental and applied target behaviors that are associated with the basic target behaviors. Basic target behaviors are broad classifications of behaviors, such as going out, eating and drinking, or purchasing. Applied target behaviors are more specific examples of basic target behaviors, such as going to a specific store on a sale day, traveling to a specific place, participating in local event activities, and so on.

[0166] The machine learning system 1 first derives an adaptive group that is likely to be prompted by an applied target behavior. The machine learning system 1 derives an adaptive group for each of a plurality of applied target behaviors, and can derive an adaptive group for a basic target behavior by appealing information about the adaptive group. In other words, the machine learning system 1 can obtain a tendency common to a plurality of adaptive groups. This makes it possible to derive a new target demographic that has not been noticed before, for example, in product development.

[0167] The derivation of the adaptive group may also be utilized for improving lifestyle habits as described in the second embodiment.

[0168] The difficulty of achievement may include, for example, an achievement rate r indicating the rate at which the target behavior is encouraged. Subjects with a high achievement rate r are classified into an adaptive group.

[0169] The achievement rate r is the sum of the number of state information items n that prompted the target behavior and the number of all state information items n, including state information items that did not prompt the target behavior. all Using these, it can be expressed, for example, by the following equation (3).

[0170]

number

[0171] Alternatively, the achievement difficulty may include, for example, a standard achievement time s indicating a standard time until the target behavior is prompted. Subjects with a short standard achievement time s are classified into an adaptive group.

[0172] The standard achievement time s can be expressed, for example, by the following formula (4) using the achievement time x, which indicates the time required to prompt the target behavior, and the average achievement time p, which indicates the average time required to prompt the target behavior. The average achievement time p is calculated by multiplying the total achievement time x by the number of all status information items n. all It can be calculated by dividing by

[0173]

number

[0174] In order to avoid being influenced by subjects with extremely long achievement times, the standard achievement time s is calculated using the standard deviation, but the difficulty of achievement may also include the average achievement time p, which uses the average instead of the standard deviation.

[0175] Alternatively, the achievement difficulty may include, for example, an average number of environmental items q indicating the average number of items of environmental information when the target behavior is prompted. The items of environmental information include, for example, scent, lighting, temperature, humidity, video, or sound. Subjects with a small average number of environmental items q are classified into an adaptive group. For example, a subject who is only influenced by scent is more likely to be prompted to the target behavior than a subject who is not influenced unless both scent and temperature are present.

[0176] The average number of environmental items q can be expressed, for example, by the following formula (5) using the number of state information items n that prompted the target behavior and the number of items of environmental information e when the target behavior was prompted. Note that the achievement difficulty calculation unit 54 may calculate the standard deviation as in formula (4) instead of the average.

[0177]

number

[0178] The names of the items of environmental information may be recorded together with the calculation of the average number of environmental items q. For example, the behavioral information recording unit 53 can record the names of the items of environmental information. This allows the machine learning system 1 to derive an adaptive group whose behavior change is likely to be prompted by specific environmental information. For example, the machine learning system 1 can derive an adaptive group whose behavior change is likely to be prompted by a scent.

[0179] The achievement difficulty level may include at least one of the achievement rate r, the standard achievement time s, and the average number of environmental items q. However, for example, it is easier to derive an adaptive group when both the achievement rate r and the standard achievement time s are included in the achievement difficulty level than when only the achievement rate r is included in the achievement difficulty level.

[0180] The achievement difficulty will be described with reference to Fig. 22. Fig. 22 is a diagram for explaining the achievement difficulty calculated by the achievement difficulty calculation unit 54 according to this embodiment. As shown in Fig. 22, the achievement rate (AR), standard achievement time (SAT), and average number of key variables (NKV) included in the achievement difficulty are shown.

[0181] Figure 22A shows the difficulty of achieving the target behavior of "exercise" when it is encouraged for subjects who do not exercise on a daily basis. As shown in the major categories, the subjects who are encouraged to take this target behavior are men and women aged 20 to 59. The overall achievement rate for these subjects is 30%, the standard achievement time is 54 hours, and the average number of environmental items is 2.

[0182] The medium categories are created by subdividing the major categories. Here, as an example, they are subdivided based on gender. The achievement rate of male subjects was 31%, the standard achievement time was 55 hours, and the average number of environmental items was 1. On the other hand, the achievement rate of female subjects was 29%, the standard achievement time was 53 hours, and the average number of environmental items was 3. This shows that male subjects had a higher achievement rate than female subjects. In other words, when focusing on the achievement rate, it can be said that men fall into the adaptive group.

[0183] Subdivision of the intermediate classification results in the small classification. Here, as an example, subdivision is performed based on age. For example, the achievement rate of a group of male subjects aged 20 to 39 is 34%, the standard achievement time is 38 hours, and the average number of environmental items is 1. Of the four groups shown in this small classification, this group has the highest achievement rate, the shortest standard achievement time, and the fewest average number of environmental items. In other words, this group can be said to be the adaptive group. The machine learning system 1 can derive the adaptive group in this way. Sales promotion activities for products and services related to the target behavior of "exercising" can be carried out for this adaptive group.

[0184] Figure 22B shows the difficulty of achieving the target behavior of "voting" when encouraging subjects who have never voted in the past five years to vote. Of the four groups shown in the subcategory, the female subject group aged 50 to 79 had the highest achievement rate, the shortest standard achievement time, and the fewest average number of environmental items. In other words, this group can be said to be the adaptive group.

[0185] In this way, the machine learning system 1 can derive an adaptive group by calculating the achievement rate, the standard achievement time, or the average number of environmental items. For example, the machine learning system 1 can derive an adaptive group with an achievement rate of 80% or more and a standard achievement time of 3 hours or less for the target behavior of "purchasing beer." A beer company can carry out advertising and promotion activities for a new product for this adaptive group when the product goes on sale.

[0186] Alternatively, for example, the machine learning system 1 can derive an adaptive group with an achievement rate of 90% or more and an average number of environmental items of 2 or less for the target behavior of "watching videos such as television broadcasts and video distribution services." The operator of the video subscription service can carry out advertising and promotion activities for this adaptive group regarding subscription to the video subscription service. Furthermore, even after subscribing to the video subscription service, the operator can carry out advertising and promotion activities for this adaptive group to encourage the renewal of the contract.

[0187] The value of this achievement difficulty may change as the target behavior is repeatedly encouraged. This will be described with reference to Fig. 23. Fig. 23 is a diagram for explaining the achievement difficulty calculated by the achievement difficulty calculation unit 54 according to this embodiment.

[0188] 23A shows the difficulty of achieving the target behavior of "purchasing product S two or more times." Note that the achievement rate is 100% if product S is purchased two times, and 50% if product S is purchased once.

[0189] In FIG. 23A, subjects are classified into a plurality of groups based on the difficulty of achievement. For example, the first group G1 has 396 subjects, an achievement rate of 86%, a standard achievement time of 67 hours, and an average number of environmental items of 2. The second group G2 has 283 subjects, an achievement rate of 62%, a standard achievement time of 120 hours, and an average number of environmental items of 3. Of these four groups, the first group G1 and the second group G2 have relatively high achievement rates, short standard achievement times, and small average number of environmental items. In other words, the first group G1 and the second group G2 can be said to be adaptive groups. The sales company of the product S can encourage the adaptive groups to take target actions related to the product S.

[0190] Next, the machine learning system 1 encourages the first group G1 and the second group G2 to engage in target behavior regarding product S, while encouraging the third group G3 and the fourth group G4, which are relatively challenging groups, to engage in target behavior regarding another product, product T.

[0191] FIG. 23B shows the difficulty of achievement when encouraging the target behavior of "purchasing product T two or more times." As shown in FIG. 23B, for the third group G3, the achievement rate is 68%, the standard achievement time is 258 hours, and the average number of environmental items is 3. Of these four groups, the third group G3 has a relatively high achievement rate, a short standard achievement time, and a small average number of environmental items. In other words, the third group G3 can be said to be an adaptive group.

[0192] In other words, by setting the target demographic for product S as the first group G1 and the second group G2, and the target demographic for product T as the third group G3, it is possible to, for example, increase product sales or improve the efficiency of sales promotion activities.

[0193] [6. Sixth embodiment of the present technology (machine learning method)] A machine learning method according to an embodiment of the present technology is a method for learning the correlation between a person's behavior and the environment around the person by machine learning using a computer device. The machine learning method according to this embodiment will be described with reference to FIG. 24. FIG. 24 is a flowchart showing the procedure of the machine learning method according to this embodiment. As shown in FIG. 24, the machine learning method according to this embodiment includes at least acquiring state information on a person's behavior (step S1), evaluating environmental information on the environment around the person when the state information is acquired and the state information to obtain a value function (step S2), and performing reinforcement learning of the value function to promote the behavior to a target behavior, and selecting the environmental information when the value function is highest (step S3).

[0194] The machine learning method according to this embodiment may utilize the techniques according to the first to fourth embodiments, so a repeated description of these techniques will be omitted.

[0195] The effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0196] The present technology can also be configured as follows. [1] A state acquisition unit that acquires at least state information related to a person's behavior; an evaluation unit that evaluates environmental information related to an environment around the person when the state information is acquired and the state information to obtain a value function; A machine learning system comprising at least a machine learning classifier that performs reinforcement learning on the value function and selects the environmental information when the value function is highest in order to encourage the behavior to become a target behavior. [2] The evaluation unit, Calculating a reward based on a difference between target state information regarding the target behavior and the state information; Calculating the value function based on the reward, the environmental information, and the state information. The machine learning system described in [1]. [3] Holding target state related information including a plurality of target behavior information; The machine learning system according to claim 1 or 2. [4] The target state related information includes time-based target state information and / or stage-based target state information. The machine learning system described in [3]. [5] The environmental information includes information regarding scent, lighting, temperature, humidity, video, or audio. The machine learning system according to any one of [1] to [4]. [6] The device further includes a scent control unit, The scent control unit controls the scent to be generated based on the environmental information selected by the machine learning classifier. The machine learning system according to any one of [1] to [5]. [7] It also has a scent attachment section, The fragrance attachment unit attaches a fragrance to an object based on the environmental information selected by the machine learning classifier; The machine learning classifier determines whether the scent control unit or the scent attachment unit generates a scent based on the environmental information. The machine learning system described in [6]. [8] A lighting control unit is further provided, The lighting control unit controls the light to be emitted based on the environmental information selected by the machine learning classifier. The machine learning system according to any one of [1] to [7]. [9] It further includes an air conditioning unit, The air conditioning unit controls temperature and / or humidity based on the environmental information selected by the machine learning classifier. The machine learning system according to any one of [1] to [8].

[10] A video control unit is further provided, The image control unit controls an image to be displayed based on the environmental information selected by the machine learning classifier. The machine learning system according to any one of [1] to [9].

[11] Further comprising a voice control unit, The audio control unit controls the audio to be played based on the environmental information selected by the machine learning classifier. The machine learning system according to any one of claims 1 to 10.

[12] The value function is divided into a plurality of value groups; The machine learning classifier uses a value function possessed by each of the plurality of value groups. The machine learning system according to any one of claims 1 to 11.

[13] A plurality of status acquisition units; An achievement difficulty calculation unit, the achievement difficulty calculation unit calculates an achievement difficulty for the target behavior based on the state information acquired by each of a plurality of state acquisition units; The machine learning system according to any one of claims 1 to 12.

[14] The achievement difficulty level includes an achievement rate indicating a rate at which the target behavior is encouraged. The machine learning system described in

[13] .

[15] The achievement difficulty level includes a standard achievement time indicating a standard time until the target behavior is prompted.

[13] or

[14] .

[16] The achievement difficulty level includes an average number of environmental items indicating an average number of items of the environmental information when the target behavior is prompted. The machine learning system according to any one of

[13] to

[15] .

[17] A state acquisition unit that acquires at least state information related to a person's behavior; an evaluation unit that evaluates environmental information related to an environment around the person when the state information is acquired and the state information to obtain a value function; A machine learning device comprising at least a machine learning classifier that performs reinforcement learning on the value function and selects the environmental information when the value function is highest in order to encourage the behavior to become a target behavior.

[18] obtaining at least state information regarding a person's behavior; Evaluating the state information and environmental information relating to an environment surrounding the person at the time the state information was acquired to obtain a value function; A machine learning method comprising at least the steps of: performing reinforcement learning on the value function to encourage the behavior to become a target behavior, and selecting the environmental information when the value function is highest. [Explanation of symbols]

[0197] 1: Machine learning systems 10: Machine learning device 11: Status acquisition unit 12: Evaluation section 13: Recording section 14: Machine learning classifier 20: Environmental control device 23: Scent control unit 24: Lighting control unit 25: Air conditioning unit 26: Video control unit 27: Audio control section 30: Fragrance attachment part 40: Information and communication networks 50: Achievement difficulty calculation device 51: Information acquisition department 52: Subject information recording section 53: Behavioral information recording section 54: Achievement difficulty calculation part S1: At least obtain status information S2: Evaluate the environmental information and the state information to obtain a value function. S3: Select environmental information

Claims

1. A state acquisition unit that acquires at least state information related to a person's behavior; an evaluation unit that evaluates environmental information related to an environment around the person when the state information is acquired and the state information to obtain a value function; and a machine learning classifier that performs reinforcement learning on the value function to select the environmental information when the value function is highest in order to encourage the behavior to become a target behavior. Holding target state related information including a plurality of target behavior information; The target state related information includes time-specific target state information and / or stage-specific target state information.

2. The evaluation unit, Calculating a reward based on a difference between target state information regarding the target behavior and the state information; Calculating the value function based on the reward, the environmental information, and the state information. The machine learning system of claim 1 .

3. The environmental information includes information regarding scent, lighting, temperature, humidity, video, or audio. The machine learning system of claim 1 .

4. The device further includes a scent control unit, The scent control unit controls the scent to be generated based on the environmental information selected by the machine learning classifier. The machine learning system of claim 1 .

5. It also has a scent attachment section, The fragrance attachment unit attaches a fragrance to an object based on the environmental information selected by the machine learning classifier; The machine learning classifier determines whether the scent control unit or the scent attachment unit generates a scent based on the environmental information. The machine learning system of claim 4.

6. A lighting control unit is further provided, The lighting control unit controls the light to be emitted based on the environmental information selected by the machine learning classifier. The machine learning system of claim 1 .

7. It further includes an air conditioning unit, The air conditioning unit controls temperature and / or humidity based on the environmental information selected by the machine learning classifier. The machine learning system of claim 1 .

8. A video control unit is further provided, The image control unit controls an image to be displayed based on the environmental information selected by the machine learning classifier. The machine learning system of claim 1 .

9. Further comprising a voice control unit, The audio control unit controls the audio to be played based on the environmental information selected by the machine learning classifier. The machine learning system of claim 1 .

10. The value function is divided into a plurality of value groups; The machine learning classifier uses the value function held by each of the plurality of value groups. The machine learning system of claim 1 .

11. A plurality of status acquisition units; An achievement difficulty calculation unit, the achievement difficulty calculation unit calculates an achievement difficulty for the target behavior based on the state information acquired by each of a plurality of state acquisition units; The machine learning system of claim 1 .

12. The achievement difficulty level includes an achievement rate indicating a rate at which the target behavior is encouraged. The machine learning system of claim 11.

13. The achievement difficulty level includes a standard achievement time indicating a standard time until the target behavior is prompted. The machine learning system of claim 11.

14. The achievement difficulty level includes an average number of environmental items indicating an average number of items of the environmental information when the target behavior is prompted. The machine learning system of claim 11.

15. A state acquisition unit that acquires at least state information related to a person's behavior; an evaluation unit that evaluates environmental information related to an environment around the person when the state information is acquired and the state information to obtain a value function; and a machine learning classifier that performs reinforcement learning on the value function to select the environmental information when the value function is highest in order to encourage the behavior to become a target behavior. Holding target state related information including a plurality of target behavior information; A machine learning device, wherein the target state related information includes time-specific target state information and / or stage-specific target state information.

16. obtaining at least state information regarding a person's behavior; Evaluating the state information and environmental information relating to an environment surrounding the person at the time the state information was acquired to obtain a value function; and selecting the environmental information when the value function is highest by performing reinforcement learning of the value function in order to encourage the behavior to become a target behavior. Holding target state related information including a plurality of target behavior information; A machine learning method, wherein the target state related information includes time-specific target state information and / or stage-specific target state information.

Citation Information

Patent Citations

  • Sales promotion system, machine learning device, machine learning data providing device, and program

    JP2019028899A

  • Cleanup support system, cleanup support method, and program

    WO2019087854A1

  • Machine learning device, output device, output device control system, output system, lighting device control system, wall, and ceiling

    WO2020027174A1