Inference device, information processing system, and inference method

Abstract

An inference device according to the present invention comprises: an operation information acquisition unit (21) that acquires operation information indicating an operation carried out by an occupant of a vehicle; and an inference unit (22) that infers the preference of the occupant on the basis of the operation information acquired by the operation information acquisition unit (21).

Description

[Technical Field]

[0001] The present disclosure relates to an inference device, an information processing system, and an inference method for inferring preferences of a vehicle occupant. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for inferring the preferences of a vehicle occupant from information obtained in the vehicle. For example, Patent Document 1 discloses a technology for determining a driver's preferences for advertisements based on biometric information when the driver of a vehicle views an advertisement, in other words, physiological responses such as facial expressions, vocalizations, electrocardiograms, or sweating rate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-52518 Summary of the Invention [Problem to be solved by the invention]

[0004] A person's preferences do not necessarily manifest as a physiological response. For example, a vehicle occupant may not necessarily have a physiological response to an advertisement that matches the occupant's preferences. Conventional preference inference techniques, such as that disclosed in Patent Document 1, have the problem that they may erroneously infer the preferences of vehicle occupants because they infer preferences based on biometric information.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an inference device that prevents erroneous inference of the preferences of a vehicle occupant when inferring the preferences of the occupant. [Means for solving the problem]

[0006] An inference device according to the present disclosure includes: The image pickup device mounted on the vehicle Vehicle occupants The occupant is detected based on the captured image.an action information acquisition unit that acquires action information indicating an action performed by the user; an attribute information acquisition unit that acquires attribute information indicating any one of a plurality of attributes of the occupant based on the captured image; and a model storage unit that stores a plurality of machine learning models corresponding to each type of the plurality of attributes, wherein the plurality of machine learning models receive as input the motion information and the attribute information and output preference information indicating the preferences of the occupant; and an inference unit that selects, from the plurality of machine learning models stored in the model storage unit, a machine learning model corresponding to the attribute information acquired by the attribute information acquisition unit as a second machine learning model, The motion information acquired by the motion information acquisition unit the attribute information acquired by the attribute information acquisition unit, and the second machine learning model. the inference unit infers the preferences of the occupant based on the Further It is what is prepared. [Effects of the Invention]

[0007] According to the present disclosure, when inferring the preferences of a vehicle occupant, erroneous inference of the preferences of the occupant can be prevented. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing system according to a first embodiment. [Figure 2] 4 is a flowchart illustrating the operation of the information processing system according to the first embodiment. [Figure 3] 4 is a flowchart illustrating the operation of the information processing device according to the first embodiment. [Figure 4] 4 is a flowchart illustrating the operation of the inference device according to the first embodiment. [Figure 5] 5A and 5B are diagrams illustrating an example of the hardware configuration of the information processing device and the inference device according to the first to third embodiments. [Figure 6] FIG. 10 is a diagram illustrating an example of a configuration of an information processing system according to a second embodiment. [Figure 7] 10 is a flowchart illustrating the operation of the information processing system according to the second embodiment. [Figure 8] 10 is a flowchart illustrating the operation of the information processing device according to the second embodiment. [Figure 9] 10 is a flowchart illustrating the operation of the inference device according to the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of a configuration of an information processing system according to a third embodiment. [Figure 11] 11 is a flowchart illustrating the operation of the information processing system according to the third embodiment. [Figure 12] 11 is a flowchart illustrating the operation of the information processing device according to the third embodiment. [Figure 13] 10 is a flowchart illustrating the operation of the inference device according to the third embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of the configuration of an information processing system according to a fourth embodiment. [Figure 15] 10 is a flowchart illustrating the operation of the information processing system according to the fourth embodiment. [Figure 16] 10 is a flowchart illustrating the operation of the information processing device according to the fourth embodiment. [Figure 17] 10 is a flowchart illustrating the operation of the inference device according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Embodiment 1 FIG. 1 is a diagram illustrating an example of a configuration of an information processing system 100 according to the first embodiment. The information processing system 100 according to the first embodiment includes an information processing device 1, an inference device 2, and a server 3. The information processing device 1, the inference device 2, and the server 3 are connected via a network. The information processing system 100 infers the preferences of an occupant of a vehicle (not shown) based on information indicating actions performed by the occupant (hereinafter referred to as "action information"), and provides information regarding the inferred preferences (hereinafter referred to as "preference inference results") to various devices, etc. In the following embodiment 1, it is assumed that the information processing device 1 is mounted on a vehicle. It is also assumed that the inference device 2 is a server provided outside the vehicle. It is also assumed that the server 3 is a server provided outside the vehicle. In the following embodiment 1, it is assumed that the inference device 2 and the server 3 are separate servers. This is merely an example, and the inference device 2 and the server 3 may be the same server. In the first embodiment, the vehicle occupant is assumed to be the driver of the vehicle. Note that this is merely an example, and the vehicle occupant may be a passenger in the front seat or a passenger in the rear seat. There may be multiple occupants whose preferences are to be inferred. Hereinafter, the vehicle occupant will also be simply referred to as the "occupant."

[0010] The information processing device 1 detects the actions of an occupant based on an image of the occupant captured by an imaging device 4 mounted on a vehicle. In the first embodiment, the imaging device 4 is assumed to be an imaging device used in a so-called "Driver Monitoring System (DMS)" to detect the state of the occupant, such as the direction of the occupant's face or the direction of the occupant's line of sight. The imaging device 4 is provided, for example, on the dashboard, the center console, or near the rearview mirror. The information processing device 1 outputs, to the inference device 2, motion information indicating the detected motion. The inference device 2 infers the preferences of the occupant based on the operation information output from the information processing device 1. The inference device 2 stores information relating to the inferred preferences (hereinafter referred to as “preference inference results”), and provides the preference inference results to the server 3. The server 3 is, for example, a server provided in a system managed by a company such as a manufacturer or a real estate company. The server 3 stores the preference inference results provided by the inference device 2. The server 3 also outputs various information based on the stored preference inference results. As a specific example, the server 3 analyzes a product to be advertised based on the preference inference results and outputs information for advertising the product. A company can, for example, use the preference inference results stored in the server 3 in its sales strategy. Note that, for simplicity of explanation, only one server 3 is shown in FIG. 1, but this is merely an example. The inference device 2 can be connected to multiple servers 3. The inference device 2 can also provide the preference inference result to the information processing device 1. In this case, the information processing device 1 outputs information based on the preference inference result to an output device 5, such as a display device or audio output device, provided in the vehicle. The display device and audio output device are provided in, for example, a navigation device mounted in the vehicle.

[0011] An example of the configuration of an information processing device 1 and an inference device 2 will be described with reference to FIG.

[0012] The information processing device 1 includes an image acquisition unit 11, a motion detection unit 12, a motion information output unit 13, a provided information acquisition unit 14, and a provided information output unit 15.

[0013] The image acquisition unit 11 acquires, from the imaging device 4, an image of the occupant. The image acquisition unit 11 outputs the acquired captured image to the motion detection unit 12.

[0014] The action detection unit 12 detects the action performed by the occupant based on the captured image acquired by the image acquisition unit 11. Specifically, the action detection unit 12 detects the action performed by the occupant based on the captured image using a known image recognition technique such as pattern matching.

[0015] For example, the action detection unit 12 detects the smoking action of the passenger. Furthermore, for example, the motion detection unit 12 detects that the occupant is eating or drinking. For example, when the motion detection unit 12 detects that the occupant is eating or drinking, it can also detect the category of what the occupant has eaten or drunk. The category of what the occupant has eaten or drunk is set in advance. The category of what the occupant has drunk is, for example, coffee, juice, or tea. The category of what the occupant has eaten is, for example, a hamburger or a rice ball. For example, the motion detection unit 12 can use known image recognition technology to detect what category of food the occupant is eating or what category of drink the occupant is drinking.

[0016] Furthermore, for example, the movement detection unit 12 detects that the occupant has touched their hair.

[0017] Furthermore, for example, the movement detection unit 12 detects that the occupant has put their hand on their shoulder.

[0018] Furthermore, for example, the movement detection unit 12 detects that the occupant has wiped away sweat.

[0019] Furthermore, for example, the motion detection unit 12 detects that the occupant has moved their line of sight. For example, the image acquisition unit 11 assigns the acquisition date and time to the acquired captured images and stores the captured images in chronological order in a storage unit (not shown). The motion detection unit 12 may detect that the occupant has moved their line of sight based on the captured images stored in the storage unit. For example, if the motion detection unit 12 can detect what the occupant looked at as they moved their line of sight from the captured images, it may also detect what the occupant looked at. For example, if the ring the occupant is wearing is in the line of sight after the occupant has moved their line of sight, the motion detection unit 12 can detect that the occupant has looked at the ring.

[0020] Furthermore, for example, the movement detection unit 12 detects that an occupant has dismounted. For example, based on the captured images stored in the storage unit, the movement detection unit 12 determines that an occupant who was captured just before is no longer captured.

[0021] The motion detection unit 12 may detect the motion of the occupant using a trained model in machine learning (hereinafter referred to as a "machine learning model"). In the first embodiment, the machine learning model used by the motion detection unit 12 when detecting the motion of the occupant is also referred to as a "motion detection machine learning model." The motion detection machine learning model is created in advance by an administrator or the like, and is stored in a location that can be referenced by the motion detection unit 12, such as a storage unit. For example, the motion detection machine learning model is a machine learning model that receives an image of a person as input and outputs information related to a motion performed by the person. The motion detection machine learning model may be, for example, a machine learning model that receives an image of a person as input and outputs information indicating that the person has performed some motion. For example, the motion detection unit 12 can detect that the occupant has eaten or drunk something and the classification of what the occupant has eaten or drunk, based on the captured image acquired by the image acquisition unit 11 and the motion detection machine learning model. Also, for example, the motion detection unit 12 can detect that the occupant has wiped sweat, based on the captured image acquired by the image acquisition unit 11 and the motion detection machine learning model.

[0022] The movement detection unit 12 can also detect multiple movements of a certain occupant. For example, the movement detection unit 12 can detect the movement of tilting one's head and putting one's hand on one's shoulder.

[0023] Furthermore, when detecting a motion performed by an occupant, the motion detection unit 12 can also detect the frequency with which the occupant performed that motion. The period over which the motion detection unit 12 detects the frequency with which the occupant performed that motion is determined in advance. The motion detection unit 12 can detect the frequency with which the occupant performed a certain motion, for example, based on captured images stored in the storage unit.

[0024] The movement detection unit 12 creates movement information indicating the detected movement of the occupant, and outputs the created movement information to the movement information output unit 13. The action information includes, for example, information identifying the action performed by the occupant. In this case, the action information is provided with information that can identify the occupant who performed the action indicated by the action information. The information that can identify the occupant here may be, for example, information that can identify the seating position of the occupant in the vehicle, such as the driver's seat or the passenger seat, or information that can identify an individual. Note that when the information that can identify the occupant is information that can identify an individual, the action detection unit 12 has a personal authentication function. For example, the action detection unit 12 performs personal authentication by matching the face of the occupant captured in the captured image acquired by the image acquisition unit 11 with a pre-stored face image for personal authentication. Furthermore, for example, the action detection unit 12 acquires biometric information from a biometric sensor (not shown) mounted on the vehicle and performs personal authentication by matching the biometric information with pre-stored biometric information for personal authentication.

[0025] For example, when the action detection unit 12 detects a smoking action by the occupant, the action information includes, for example, information indicating that the occupant has performed a smoking action. The action information may further include information indicating the frequency with which the occupant has performed a smoking action.

[0026] For example, if the motion detection unit 12 detects that the occupant is eating or drinking, the motion information includes, for example, information indicating that the occupant has eaten or drunk. The motion information may further include information indicating how often the occupant has eaten or drunk, or information indicating the category of what the occupant has eaten or drunk. For example, the motion detection unit 12 may include, in the motion information, information indicating that the occupant has drunk coffee or a hamburger.

[0027] For example, if the motion detection unit 12 detects that the occupant has touched their hair, the motion information may include, for example, information indicating that the occupant has touched their hair. The motion information may further include information indicating how often the occupant has touched their hair.

[0028] For example, when the motion detection unit 12 detects that the occupant has put their hand on their shoulder, the motion information includes, for example, information indicating that the occupant has put their hand on their shoulder. The motion information may further include information indicating the time period during which the occupant has put their hand on their shoulder.

[0029] For example, if the motion detection unit 12 detects that the occupant has wiped away sweat, the motion information may include, for example, information indicating that the occupant has wiped away sweat. The motion information may further include information indicating how often the occupant has wiped away sweat.

[0030] For example, when the motion detection unit 12 detects that the occupant has moved their line of sight, the motion information includes, for example, information indicating that the occupant has moved their line of sight. The motion information may further include information indicating the line of sight direction after the movement. The motion information may also include information indicating what the occupant looked at, such as a ring, or information indicating how long the occupant looked at a certain object.

[0031] For example, the action detection unit 12 may add information (hereinafter referred to as "facial expression information") about the occupant's facial expression (such as "smile" or "relaxed") or information (hereinafter referred to as "voice information") about a preset utterance indicating a preference (such as "delicious" or "I like it") to the action information and output the information to the action information output unit 13. The action detection unit 12 can detect the occupant's facial expression based on the captured image using, for example, a known image recognition technology or a machine learning model. Furthermore, the action detection unit 12 can acquire the utterance from a microphone (not shown) mounted on the vehicle and recognize the utterance using a known voice recognition technology. The movement detection unit 12 adds facial expression information or voice information to the movement information and outputs the information to the movement information output unit 13, so that the inference unit 22 of the inference device 2, which infers the preferences of the occupant using the movement information output by the movement information output unit 13, can improve the accuracy of inferring the preferences of the occupant. Details of the inference device 2 will be described later.

[0032] The operation information may be, for example, a captured image acquired by the image acquisition unit 11. For example, when the motion detection unit 12 detects that an occupant has performed some kind of motion based on the captured image acquired by the image acquisition unit 11 and a machine learning model for motion detection, the motion detection unit 12 may output the captured image to the motion information output unit 13 as motion information.

[0033] The motion information output unit 13 outputs the motion information output from the motion detection unit 12 to the inference device 2.

[0034] The provided information acquisition unit 14 acquires information output from the inference device 2. Details of the information output from the inference device 2 will be described later. The provided information acquisition unit 14 outputs the information acquired from the inference device 2 to the provided information output unit 15.

[0035] The provided information output unit 15 outputs the information acquired by the provided information acquisition unit 14 from the inference device 2 to the output device 5. The provided information output unit 15 causes the information acquired by the provided information acquisition unit 14 from the inference device 2 to be displayed on a display device or output as sound from an audio output device, for example.

[0036] In the first embodiment, the information processing device 1 includes the provided information acquisition unit 14 and the provided information output unit 15, but this is merely an example. The information processing device 1 does not necessarily have to include the provided information acquisition unit 14 and the provided information output unit 15.

[0037] The inference device 2 includes an action information acquisition unit 21 , an inference unit 22 , a model storage unit 23 , an inference result storage unit 24 , an information providing unit 25 , a learning data acquisition unit 26 , and a learning unit 27 .

[0038] The motion information acquisition unit 21 acquires the motion information output from the information processing device 1. The motion information acquiring unit 21 outputs the acquired motion information to the inferring unit 22 and the learning data acquiring unit 26.

[0039] The inference unit 22 infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21. For example, the inference unit 22 infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21 and a machine learning model (hereinafter referred to as the "first machine learning model"). The first machine learning model is a machine learning model that receives action information as input and outputs preference information indicating preferences. The first machine learning model is created in advance by an administrator or the like and stored in the model storage unit 23. The administrator or the like, for example, prepares a plurality of subjects in advance and has the subjects test drive the vehicle to create the first machine learning model. The first machine learning model is a machine learning model created by learning using the action information obtained during the test drive and preference information indicating the preferences of the occupant obtained by interviewing the occupant as training data. Note that the preference information is a training label.

[0040] For example, if the action information indicates that the occupant has performed a smoking action, the inference unit 22 inputs the action information into the first machine learning model to obtain preference information that the occupant likes cigarettes. As a result, the inference unit 22 infers that the occupant's preference is "I like cigarettes." Furthermore, for example, if the action information includes information indicating that the occupant has performed a smoking action and information indicating the frequency of smoking, the inference unit 22 inputs the action information into the first machine learning model to obtain preference information indicating the degree to which the occupant likes cigarettes. As a result, the inference unit 22 infers the "degree to which the occupant likes cigarettes" as the occupant's preference.

[0041] Furthermore, for example, if the action information includes information indicating that the occupant has drunk coffee and information indicating the frequency of drinking coffee, the inference unit 22 inputs the action information into the first machine learning model to obtain preference information indicating the degree to which the occupant likes coffee. As a result, the inference unit 22 infers the "degree to which the occupant drinks coffee" as the occupant's preference.

[0042] Furthermore, for example, if the motion detection unit 12 detects information including information indicating that the occupant touched their hair and information indicating the frequency of touching their hair, the inference unit 22 inputs the motion information into the first machine learning model to obtain preference information indicating the degree of interest in beauty. As a result, the inference unit 22 infers the "degree of interest in beauty" as the occupant's preference.

[0043] Furthermore, for example, if the action information includes information indicating that the occupant has placed their hand on their shoulder, the inference unit 22 inputs the action information into the first machine learning model to obtain preference information indicating that the occupant likes massages. As a result, the inference unit 22 infers the occupant's preference that "I like massages."

[0044] Furthermore, for example, if the action information is information indicating the frequency with which the occupant shifted their gaze to a ring, the inference unit 22 inputs the action information into the first machine learning model to obtain preference information indicating the degree to which the occupant likes accessories. As a result, the inference unit 22 infers the "degree to which the occupant likes accessories" as the occupant's preference.

[0045] Furthermore, for example, when the operation information is a captured image, the inference unit 22 may input the captured image to the first machine learning model to infer the occupant's preference. For example, the inference unit 22 infers the occupant's preference, such as "prefers cigarettes," "prefers hamburgers," "prefers coffee," or "prefers accessories."

[0046] The inference unit 22 may infer the preferences of the occupant using a method other than the method using the first machine learning model. For example, the inference unit 22 may infer the preference of the occupant by comparing the action information acquired by the action information acquisition unit 21 with conditions that define preference tendencies (hereinafter referred to as "preference inference conditions"). The preference inference conditions are created in advance by, for example, an administrator, and are stored in the inference unit 22. The preference inference conditions define, for example, what preference is inferred when the occupant performs what action.

[0047] The inference unit 22 stores the preference inference result in the inference result storage unit 24. The preference inference result is, for example, information in which preference information indicating the preference inferred by the inference unit 22 is associated with action information. The action information associated with the preference information is the action information that was input to the first machine learning model when the inference unit 22 obtained the preference information, or the action information that the inference unit 22 compared with the preference inference conditions when the inference unit 22 obtained the preference information. Furthermore, the inference unit 22 outputs preference information indicating the inferred preferences to the learning data acquisition unit 26.

[0048] The model storage unit 23 stores the first machine learning model. In the first embodiment, the model storage unit 23 is provided in the inference device 2, but this is merely an example. The model storage unit 23 may be provided in a location outside the inference device 2 that can be referenced by the inference device 2.

[0049] The inference result storage unit 24 stores the preference inference result output by the inference unit 22. In the first embodiment, the inference result storage unit 24 is provided in the inference device 2, but this is merely an example. The inference result storage unit 24 may be provided in a location outside the inference device 2 that can be referenced by the inference device 2.

[0050] The information providing unit 25 refers to the inference result storage unit 24 and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1). The information providing unit 25 determines the server 3 or the information processing device 1 to which the preference inference result is to be provided, depending on the content of the preference inference result. It is predetermined which server 3 or information processing device 1 the preference inference result is to be output to depending on the content of the preference inference result.

[0051] For example, if the preference inference result is information that associates preference information indicating the degree to which the occupant likes coffee with information indicating that the occupant has drunk coffee and behavior information including information indicating the frequency with which the occupant has drunk coffee, the information providing unit 25 outputs the preference inference result to a server 3 provided in a system owned by the beverage manufacturer. For example, server 3 stores the preference inference results output from information provider 25, and calculates the proportion of people who prefer coffee based on the stored preference inference results. Beverage manufacturers can develop sales strategies based on the proportion of people who prefer coffee based on the proportion of people who prefer coffee calculated by server 3.

[0052] For example, if the preference inference result is information that associates preference information indicating that a passenger likes massage with action information indicating that the passenger has placed their hand on the passenger's shoulder, the information providing unit 25 outputs the preference inference result to a server 3 provided in a system owned by the massage parlor. For example, the server 3 stores the preference inference results output from the information providing unit 25. The massage parlor analyzes the demand for shoulder massages from the stored amount of preference inference results.

[0053] Furthermore, for example, if the management company of the inference device 2 has a business partnership with an advertising company and information about advertisements is stored in a location that can be referenced by the information providing unit 25, the information providing unit 25 may assign information for outputting advertisements or the like (hereinafter referred to as "advertisement output control information") to the preference inference result output by the inference unit 22, and output the result to the information processing device 1. The provided information acquiring unit 14 of the information processing device 1 acquires the preference inference result output from the information providing unit 25. The provided information output unit 15 causes the output device 5 to output advertisements or the like, based on the advertisement output control information assigned to the preference inference result acquired by the provided information acquiring unit 14. For example, if the preference inference result is information in which preference information indicating the degree of interest in beauty is associated with information indicating that the occupant has touched their hair and action information including information indicating the frequency of touching their hair, the information providing unit 25 assigns advertisement output control information for outputting a commercial for a hair catalog to the preference inference result and outputs it to the information processing device 1. The provided information acquisition unit 14 of the information processing device 1 acquires the preference inference result, and the provided information output unit 15 causes the display device to display the commercial for the hair catalog.

[0054] In the first embodiment, the inference device 2 includes the information providing unit 25, but this is merely an example. For example, the inference device 2 may not include the information providing unit 25. For example, the management company of the inference device 2 may provide the inference result storage unit 24, in which the preference inference results are stored, to various companies and the like.

[0055] The learning data acquiring unit 26 acquires, as learning data, the motion information acquired by the motion information acquiring unit 21 and the preference information indicating the preference inferred by the inference unit 22. The inference unit 22 may output the preference inference result to the learning data acquisition unit 26, and the learning data acquisition unit 26 may acquire the preference inference result. In this case, the motion information acquisition unit 21 may not output the motion information to the learning data acquisition unit 26. Furthermore, for example, when the inference unit 22 provides the preference inference result to the information processing device 1, the learning data acquisition unit 26 may acquire information indicating whether the preference inference result is correct or not (hereinafter referred to as "feedback information") from the information processing device 1 and determine whether to acquire learning data. In detail, for example, the information providing unit 25 outputs the preference inference result to the information processing device 1, and in the information processing device 1, the provided information acquisition unit 14 acquires the preference inference result and the provided information output unit 15 displays the preference inference result on the output device 5. The occupant then checks the preference inference result displayed on the output device 5. If the displayed preference inference result is incorrect as the occupant's preference, in other words, is not their preference, the occupant operates the output device 5 to input that the preference is not their preference. The output device 5 may be, for example, a touch panel display. A receiving unit (not shown) of the information processing device 1 outputs the information that the occupant inputted, indicating that the preference is not their preference, as feedback information to the learning data acquisition unit 26 of the inference device 2. When the learning data acquisition unit 26 acquires the feedback information, it determines that the preference inferred by the inference unit 22 is incorrect and does not adopt the action information and the preference information as learning data; in other words, it does not acquire learning data. This can improve the accuracy of the learning data when the learning unit 27 learns the first machine learning model. Details of the learning unit 27 will be described later. The learning data acquisition unit 26 outputs the acquired learning data to the learning unit 27.

[0056] Learning unit 27 learns the first machine learning model stored in model storage unit 23 based on the learning data acquired by learning data acquisition unit 26. Learning unit 27 updates the first machine learning model stored in model storage unit 23 to the first machine learning model after learning.

[0057] In the first embodiment, the inference device 2 includes the learning data acquisition unit 26 and the learning unit 27, but this is merely an example. For example, the inference device 2 may be configured without the learning data acquisition unit 26 and the learning unit 27. However, by configuring the inference device 2 to include the learning data acquisition unit 26 and the learning unit 27, the inference device 2 can improve the accuracy of the first machine learning model. As a result, the inference device 2 can improve the accuracy of inferring occupant preferences using the first machine learning model. Furthermore, for example, the functions of the learning data acquisition unit 26 and the learning unit 27 may be provided by a learning device (not shown) that is provided outside the inference device 2 in a location that can be referenced by the inference device 2.

[0058] The operation of the information processing system 100 according to the first embodiment will be described. FIG. 2 is a flowchart illustrating the operation of the information processing system 100 according to the first embodiment.

[0059] The information processing device 1 detects a movement made by the occupant based on an image of the occupant captured by the imaging device 4, and outputs movement information indicating the detected movement to the inference device 2 (step ST1).

[0060] The inference device 2 infers the preferences of the occupant based on the operation information output from the information processing device 1 in step ST1 (step ST2). Then, the inference device 2 stores the preference inference result and provides it to the server 3 or the information processing device 1.

[0061] The operation of the information processing device 1 according to the first embodiment will be described. FIG. 3 is a flowchart for explaining the operation of the information processing device 1 according to the first embodiment. The operation of the information processing device 1 shown in FIG. 3 corresponds to the operation of step ST1 of the operation of the information processing system 100 described using the flowchart of FIG. For example, when the power supply of the vehicle is turned on, the information processing device 1 repeats the operation shown in the flowchart of FIG. 3 until the power supply of the vehicle is turned off.

[0062] The image acquisition unit 11 acquires a captured image of the occupant from the imaging device 4 (step ST11). The image acquisition unit 11 outputs the acquired captured image to the motion detection unit 12.

[0063] The movement detection unit 12 detects a movement made by the occupant based on the captured image acquired by the image acquisition unit 11 in step ST11 (step ST12). The movement detection unit 12 creates movement information indicating the detected movement by the occupant, and outputs the created movement information to the movement information output unit 13.

[0064] The motion information output unit 13 outputs the motion information output from the motion detection unit 12 in step ST12 to the inference device 2 (step ST13).

[0065] When the provided information acquisition unit 14 acquires the information (specifically, the preference inference result) output from the inference device 2, it outputs the information to the provided information output unit 15 (if "YES" in step ST14).

[0066] The provided information output unit 15 outputs the preference inference result acquired by the provided information acquisition unit 14 from the inference device 2 in step ST14 to the output device 5 (step ST15). The provided information output unit 15 causes the information based on the preference inference result acquired by the provided information acquisition unit 14 from the inference device 2 to be displayed on a display device or output as sound from an audio output device, for example.

[0067] If the provided information acquisition unit 14 does not acquire the preference inference result output from the inference device 2 ("NO" in step ST14), the information processing device 1 ends the operation shown in the flowchart of FIG.

[0068] 3, the processing is performed in the order of steps ST11 to ST15, but this is merely an example. The processing of steps ST11 to ST13 and the processing of steps ST14 to ST15 may be performed in parallel. Furthermore, if the information processing device 1 does not include the provided information acquisition unit 14 and the provided information output unit 15, the operation of the information processing device 1 can omit the processes of steps ST14 to ST15.

[0069] The operation of the inference device 2 according to the first embodiment will be described. FIG. 4 is a flowchart for explaining the operation of the inference device 2 according to the first embodiment. The operation of the inference device 2 shown in FIG. 4 corresponds to the operation of step ST2 of the operation of the information processing system 100 described using the flowchart of FIG. For example, when the power of the inference device 2 is turned on, the inference device 2 repeats the operation shown in the flowchart of FIG. 4 until the power of the inference device 2 is turned off.

[0070] The motion information acquisition unit 21 acquires the motion information output from the information processing device 1 (step ST21). The motion information acquiring unit 21 outputs the acquired motion information to the inferring unit 22 and the learning data acquiring unit 26.

[0071] The inference unit 22 infers the preferences of the occupant based on the operation information acquired by the operation information acquisition unit 21 in step ST21 (step ST22). The inference unit 22 stores the preference inference result in the inference result storage unit 24 . Furthermore, the inference unit 22 outputs preference information indicating the inferred preferences to the learning data acquisition unit 26.

[0072] The information providing unit 25 refers to the inference result storage unit 24, and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1) (step ST23).

[0073] The learning data acquiring unit 26 acquires, as learning data, the motion information acquired by the motion information acquiring unit 21 in step ST21 and the preference information indicating the preference inferred by the inference unit 22 in step ST22 (step ST24). The learning data acquisition unit 26 may acquire the preference inference result from the inference unit in step ST22. The learning data acquisition unit 26 outputs the acquired learning data to the learning unit 27.

[0074] The learning unit 27 learns the first machine learning model stored in the model storage unit 23 based on the learning data acquired by the learning data acquisition unit 26 in step ST24 (step ST25). The learning unit 27 updates the first machine learning model stored in the model storage unit 23 to the first machine learning model after learning.

[0075] 4, the processing is performed in the order of steps ST23 to ST25, but this is merely an example. The processing of step ST23 and the processing of steps ST24 to ST25 may be performed in parallel. Furthermore, if the inference device 2 does not include the learning data acquisition unit 26 and the learning unit 27, the operation of the inference device 2 can omit the processing of steps ST24 to ST25.

[0076] In this way, in the information processing system 100, the inference device 2 acquires, from the information processing device 1, action information indicating actions performed by a vehicle occupant, and infers the preferences of the occupant based on the acquired action information. Human preferences are not always expressed as physiological responses. Therefore, when the preferences of a vehicle occupant are inferred based on biometric information, as in the conventional technology described above, there is a problem that the preferences may be inferred incorrectly. In particular, a vehicle may shake while traveling. When the vehicle shakes, information based on the vehicle shaking may be included in the biometric information as noise. The noise included in the biometric information may lead to an incorrect inference of preferences. On the other hand, human preferences are expected to be reflected in the behavior of occupants, such as looking at things they like, touching things they like, or eating and drinking things they like. As described above, the inference device 2 infers the preferences of the vehicle occupant based on the action information indicating the action performed by the vehicle occupant. Therefore, when inferring the preferences of the vehicle occupant, the inference device 2 can prevent erroneous inference of the preferences of the vehicle occupant.

[0077] Furthermore, the inference device 2 infers the preferences of the vehicle occupants using captured images of the vehicle occupants, and outputs preference inference results based on the inferred preferences to the vehicle (specifically, the information processing device 1) or the server 3. As a result, the inference device 2 can provide services based on the inferred preferences to the vehicle occupants or various companies. Furthermore, since the inference device 2 infers the preferences of the occupants using captured images obtained by a DMS generally installed in the vehicle, it can effectively utilize information obtained in the vehicle.

[0078] 5A and 5B are diagrams illustrating an example of a hardware configuration of the information processing device 1 according to the first embodiment. In the first embodiment, the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 are realized by the processing circuit 1001. That is, the information processing device 1 includes the processing circuit 1001 for detecting the motion of an occupant based on an image acquired from the imaging device 4 and for controlling the output of the motion information to the inference device 2. The processing circuit 1001 may be dedicated hardware as shown in FIG. 5A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 5B.

[0079] When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0080] When the processing circuit is the processor 1004, the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005 to execute the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15. That is, the information processing device 1 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST11 to ST15 in FIG. 3 described above. It can also be said that the program stored in memory 1005 causes the computer to execute the processing procedures or methods of image acquisition unit 11, motion detection unit 12, motion information output unit 13, provided information acquisition unit 14, and provided information output unit 15. Here, memory 1005 corresponds to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0081] It is also possible to realize some of the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 with dedicated hardware and some with software or firmware. For example, the functions of the image acquisition unit 11 and the provided information acquisition unit 14 can be realized by a processing circuit 1001 as dedicated hardware, and the functions of the motion detection unit 12, the motion information output unit 13, and the provided information output unit 15 can be realized by the processor 1004 reading and executing a program stored in the memory 1005. The information processing device 1 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the inference device 2, the imaging device 4, or the output device 5. The storage unit (not shown) is composed of a HDD (Hard Disk Drive), memory 1005, a DVD, and the like.

[0082] An example of the hardware configuration of the inference device according to the first embodiment is also as shown in FIGS. 5A and 5B. In the first embodiment, the functions of the motion information acquisition unit 21, the inference unit 22, the information provision unit 25, the learning data acquisition unit 26, and the learning unit 27 are realized by a processing circuit 1001. That is, the inference device 2 includes the processing circuit 1001 for controlling the inference of the preferences of the occupant based on the motion information acquired from the information processing device 1. The processing circuit 1001 may be dedicated hardware as shown in FIG. 5A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 5B.

[0083] When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0084] When the processing circuit is a processor 1004, the functions of the motion information acquisition unit 21, the inference unit 22, the information provision unit 25, the learning data acquisition unit 26, and the learning unit 27 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 executes the functions of the motion information acquisition unit 21, the inference unit 22, the information provision unit 25, the learning data acquisition unit 26, and the learning unit 27 by reading and executing the program stored in the memory 1005. That is, the inference device 2 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST21 to ST25 in FIG. 4 described above. The program stored in the memory 1005 can also be said to cause a computer to execute the processing procedures or methods of the motion information acquisition unit 21, the inference unit 22, the information provision unit 25, the learning data acquisition unit 26, and the learning unit 27. Here, memory 1005 refers to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0085] It is also possible to realize some of the functions of the motion information acquisition unit 21, the inference unit 22, the information provision unit 25, the learning data acquisition unit 26, and the learning unit 27 with dedicated hardware and some with software or firmware. For example, the functions of the motion information acquisition unit 21 and the learning data acquisition unit 26 can be realized by a processing circuit 1001 as dedicated hardware, and the functions of the inference unit 22, the information provision unit 25, and the learning unit 27 can be realized by the processor 1004 reading and executing a program stored in the memory 1005. The inference device 2 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the information processing device 1 or the server 3. The model storage unit 23 and the inference result storage unit 24 are configured with a HDD (Hard Disk Drive), memory 1005, a DVD, and the like.

[0086] In the above-described first embodiment, the inference device 2 is connected to one information processing device 1, but this is merely an example. The inference device 2 can be connected to multiple information processing devices 1. The inference device 2 can acquire operation information from each of the information processing devices 1 installed in multiple different vehicles, and infer the preferences of the occupants of each vehicle based on the acquired operation information. Note that the operation information output from the information processing device 1 is accompanied by information that can identify the vehicle.

[0087] Furthermore, in the above-described first embodiment, a plurality of machine learning models may be stored in the model storage unit 23 of the inference device 2 according to the type of movement. The inference unit 22 selects, from the plurality of machine learning models stored in the model storage unit 23, a machine learning model according to the movement information acquired by the movement information acquisition unit 21 as the first machine learning model.

[0088] In the first embodiment, the inference device 2 may be mounted on a vehicle.

[0089] In addition, in the above-described embodiment 1, the information processing device 1 is an in-vehicle device mounted on a vehicle, and the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 are provided in the in-vehicle device. Without being limited to this, for example, some of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 may be mounted on the vehicle's on-board device, and the rest may be provided in the inference device 2. Furthermore, for example, the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, and the provided information output unit 15 may all be provided in the inference device 2.

[0090] As described above, according to the first embodiment, the inference device 2 is configured to include the motion information acquisition unit 21 that acquires motion information indicating motions performed by a vehicle occupant, and the inference unit 22 that infers the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21. Therefore, when inferring the preferences of a vehicle occupant, the inference device 2 can prevent erroneous inference of the preferences of the occupant.

[0091] Furthermore, according to the first embodiment, the information processing system 100 is configured to include an information processing device 1 having an image acquisition unit 11 that acquires captured images of a vehicle occupant, a motion detection unit 12 that detects motions performed by the occupant based on the captured images acquired by the image acquisition unit 11, and a motion information output unit 13 that outputs motion information indicating the motion of the occupant detected by the motion detection unit 12, and an inference device having a motion information acquisition unit 21 that acquires the motion information output by the motion information output unit 13, and an inference unit 22 that infers the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21. Therefore, when inferring the preferences of a vehicle occupant, the information processing system 100 can prevent erroneous inference of the preferences of the occupant.

[0092] Embodiment 2 In the first embodiment, the information processing system infers the preferences of a vehicle occupant based on the actions performed by the occupant. In the second embodiment, an embodiment will be described in which the preferences of a vehicle occupant are inferred in an information processing system by taking into consideration the attributes of the occupant in addition to the actions performed by the occupant. In the following embodiment 2, the occupant is assumed to be the driver of the vehicle, as in embodiment 1. However, this is merely an example, and in embodiment 2, the occupant may be an occupant other than the driver of the vehicle, and the information processing system may infer the preferences of multiple occupants.

[0093] FIG. 6 is a diagram illustrating an example of a configuration of an information processing system 100a according to the second embodiment. 6, the same components as those in the information processing system 100 described in the first embodiment with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. In the information processing system 100a according to the second embodiment, exemplary configurations of the information processing device 1a and the inference device 2a are different from the exemplary configurations of the information processing device 1 and the inference device 2 in the information processing system 100 according to the first embodiment. Details of the exemplary configurations of the information processing device 1a and the inference device 2a will be described later. The information processing system 100a infers the preferences of a vehicle occupant based on action information indicating the actions performed by the vehicle occupant and information indicating the attributes of the vehicle occupant (hereinafter referred to as "attribute information"), and provides the preference inference results to various devices, etc.

[0094] The information processing device 1a detects the actions performed by the occupant based on the captured image of the occupant captured by the imaging device 4, and also detects the attributes of the occupant. The information processing device 1a outputs, to the inference device 2a, movement information indicating the detected movement of the occupant and attribute information indicating the attribute of the detected occupant. The inference device 2a infers the preferences of the occupant based on the action information and attribute information output from the information processing device 1a. The inference device 2a stores the preference inference result and provides the preference inference result to the server 3 or the information processing device 1a.

[0095] An example of the configuration of the information processing device 1a and the inference device 2a will be described with reference to FIG.

[0096] Regarding the information processing device 1a according to the second embodiment shown in FIG. 6, the same components as those of the information processing device 1 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant description will be omitted. The information processing device 1a according to the second embodiment differs from the information processing device 1 according to the first embodiment in that it includes an attribute detection unit 16 and an attribute information output unit 17.

[0097] The attribute detection unit 16 detects the attributes of the occupant based on the captured image acquired by the image acquisition unit 11. Specifically, the attribute detection unit 16 detects the attributes of the occupant based on the captured image using a known image recognition technique such as pattern matching.

[0098] For example, the attribute detection unit 16 detects that an occupant is wearing expensive accessories as an attribute of the occupant. Also, for example, the attribute detection unit 16 detects that an occupant is wearing elegant accessories as an attribute of the occupant. Various classifications of accessories, such as whether they are elegant or not, or whether they are expensive or not, are set in advance.

[0099] Furthermore, for example, the attribute detection unit 16 detects the physique as an attribute of the occupant.

[0100] Furthermore, for example, the attribute detection unit 16 detects the gender as an attribute of the occupant.

[0101] Furthermore, for example, the attribute detection unit 16 detects age or generation as an attribute of the occupant.

[0102] Furthermore, for example, the attribute detection unit 16 detects hairstyle or hair color as an attribute of the occupant.

[0103] Furthermore, for example, the attribute detection unit 16 detects family structure as an attribute of the occupant. For example, if the attribute detection unit 16 detects from the captured image that one adult female, one adult male, and one child are riding in the vehicle, it determines that the family structure of the occupant is a family of three.

[0104] The attribute detection unit 16 may detect the attributes of the occupant using a machine learning model. In the second embodiment, the machine learning model used by the attribute detection unit 16 when detecting the attributes of the occupant is also referred to as an "attribute detection machine learning model." The attribute detection machine learning model is created in advance by an administrator or the like, and is stored in a location that can be referenced by the attribute detection unit 16, such as a storage unit (not shown). The attribute detection machine learning model is, for example, a machine learning model that receives an image of a person as input and outputs information related to the person's attributes. The attribute detection machine learning model may be, for example, a machine learning model that receives an image of a person as input and outputs information indicating that the person has some predetermined attribute. For example, the attribute detection unit 16 can detect that the occupant is wearing expensive accessories or that the occupant is in their 30s based on the captured image acquired by the image acquisition unit 11 and a machine learning model for attribute detection.

[0105] The attribute detection unit 16 can also detect multiple attributes for a certain occupant. For example, the attribute detection unit 16 can detect age and gender as attributes of the occupant, such as that the occupant is in his / her 30s and that the occupant is male.

[0106] The attribute detection unit 16 creates attribute information indicating the detected attributes of the occupant, and outputs the created attribute information to the attribute information output unit 17. The attribute information includes, for example, information that identifies the attributes of the occupant. In this case, the attribute information is provided with information that can identify the occupant who has the attribute indicated by the attribute information. The information that can identify the occupant here is assumed to be information that can identify the seating position of the occupant in the vehicle, such as the driver's seat or the passenger seat.

[0107] For example, if the attribute detection unit 16 detects that the occupant is wearing expensive accessories as an attribute of the occupant, the attribute information may include, for example, information indicating that the occupant is wearing expensive accessories. The attribute information may further include information indicating the type of accessories worn by the occupant, such as a ring, a necklace, or earrings.

[0108] For example, if the attribute detection unit 16 detects the physique of the occupant as an attribute of the occupant, the attribute information includes, for example, information indicating whether the occupant is large, average, or small.

[0109] For example, if the attribute detection unit 16 detects the gender as an attribute of the occupant, the attribute information includes, for example, information indicating whether the occupant is male or female.

[0110] For example, if the attribute detection unit 16 detects age or generation as an attribute of the occupant, the attribute information includes, for example, information indicating that the occupant is 32 years old or in his 30s.

[0111] For example, if the attribute detection unit 16 detects hairstyle or hair color as an attribute of an occupant, the attribute information includes information indicating, for example, that the occupant has short hair or that the occupant's hair color is red.

[0112] For example, when the attribute detection unit 16 detects family structure as an attribute of the occupant, the attribute information includes, for example, information indicating that the occupants are a family of three. The attribute information may include information indicating that the family includes one child, and may further include information indicating the child's age.

[0113] The attribute information may be, for example, a captured image acquired by the image acquisition unit 11. For example, when the attribute detection unit 16 detects some attribute of an occupant based on the captured image acquired by the image acquisition unit 11 and a machine learning model for attribute detection, the attribute detection unit 16 outputs the captured image to the attribute information output unit 17 as attribute information.

[0114] The attribute information output unit 17 outputs the attribute information output from the attribute detection unit 16 to the inference device 2a.

[0115] Regarding the inference device 2a according to the second embodiment shown in FIG. 6, the same components as those of the inference device 2 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant description will be omitted. The inference device 2a according to the second embodiment differs from the inference device 2 according to the first embodiment in that it includes an attribute information acquisition unit . Furthermore, the specific operations of inference unit 22a, information providing unit 25a, learning data acquiring unit 26a, and learning unit 27a in inference device 2a according to embodiment 2 are different from the specific operations of inference unit 22, information providing unit 25, learning data acquiring unit 26, and learning unit 27 in inference device 2 according to embodiment 1. Furthermore, the specific contents of the information stored in model storage unit 23a and inference result storage unit 24a in inference device 2 according to embodiment 2 are different from the specific contents of the information stored in model storage unit 23 and inference result storage unit 24 in inference device 2 according to embodiment 1, respectively.

[0116] The attribute information acquisition unit 28 acquires the attribute information output from the information processing device 1a. The attribute information acquisition unit 28 outputs the acquired attribute information to the inference unit 22a and the learning data acquisition unit 26a.

[0117] In the second embodiment, the inference unit 22a infers the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21 and the attribute information acquired by the attribute information acquisition unit . For example, the inference unit 22a infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and a machine learning model (hereinafter referred to as the "second machine learning model"). The second machine learning model is a machine learning model that receives action information and attribute information as input and outputs preference information indicating preferences. The second machine learning model is created in advance by an administrator or the like and stored in the model storage unit 23a. For example, the administrator or the like prepares multiple subjects in advance and has the subjects test drive the vehicle to create the second machine learning model. The second machine learning model is a machine learning model created by learning using the action information and attribute information obtained during the test drive and preference information indicating the preferences of the occupant obtained by interviewing the occupant as training data. Note that the preference information is a training label.

[0118] For example, if the action information indicates that the occupant touched their hair and the attribute information indicates that the occupant's hair color is red, the inference unit 22a inputs the action information and the attribute information into the second machine learning model to obtain preference information indicating an interest in hair coloring. As a result, the inference unit 22a infers the occupant's preference that "I like coloring my hair."

[0119] For example, if the behavior information is information indicating that the occupant touched a piercing and information indicating the frequency of touching the piercing, and the attribute information is information indicating that the piercing worn by the occupant is elegant, the inference unit 22a inputs the behavior information and the attribute information into the second machine learning model to obtain preference information indicating an interest in luxury jewelry. From this, the inference unit 22 infers the occupant's preference that "he likes luxury jewelry."

[0120] The inference unit 22a may infer the preferences of the occupant using a method other than the method using the second machine learning model. For example, the inference unit 22a may infer the preference of the occupant by comparing the action information acquired by the action information acquisition unit 21 and the attribute information acquired by the attribute information acquisition unit 28 with the preference inference conditions. In the second embodiment, the preference inference conditions define, for example, what preference is inferred when an occupant with what attribute performs what action.

[0121] The inference unit 22a stores the preference inference result in the inference result storage unit 24a. The preference inference result is, for example, information in which preference information indicating the preference inferred by the inference unit 22a is associated with action information and attribute information. The action information and attribute information associated with the preference information are the action information and attribute information that were input to the second machine learning model when the inference unit 22a obtained the preference information, or the action information and attribute information that were compared with the preference inference conditions when the inference unit 22a obtained the preference information. Furthermore, the inference unit 22a outputs preference information indicating the inferred preference to the learning data acquisition unit 26a.

[0122] In the second embodiment, the model storage unit 23a stores a second machine learning model. In the second embodiment, the model storage unit 23a is provided in the inference device 2a, but this is merely an example. The model storage unit 23a may be provided in a location outside the inference device 2a that can be referenced by the inference device 2a.

[0123] In the second embodiment, the inference result storage unit 24a stores the preference inference result output by the inference unit 22a. In the second embodiment, the inference result storage unit 24a is provided in the inference device 2a, but this is merely an example. The inference result storage unit 24a may be provided in a location outside the inference device 2a that can be referenced by the inference device 2a.

[0124] In the second embodiment, the information providing unit 25a refers to the inference result storage unit 24a and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1a). The information providing unit 25a determines the server 3 or the information processing device 1a to which the preference inference result is to be provided, depending on the content of the preference inference result. It is predetermined which server 3 or information processing device 1a the preference inference result is to be output to depending on the content of the preference inference result.

[0125] For example, if the preference inference result is information that correlates preference information indicating a preference for hair coloring, action information indicating that the occupant has touched their hair, and attribute information indicating that the occupant's hair is red and that the occupant is in their twenties, the information providing unit 25a outputs the preference inference result to a server 3 provided in a system owned by a publishing company. For example, the server 3 stores the preference inference result output from the information providing unit 25a. Based on the preference inference result stored by the server 3, a publishing company can plan to include a special feature on hair coloring in a magazine aimed at people in their twenties.

[0126] Furthermore, for example, if the management company of the inference device 2a has a business partnership with an advertising company and information about advertisements is stored in a location that the information providing unit 25a can refer to, the information providing unit 25a may assign advertisement output control information to the preference inference result in accordance with the preference inference result output by the inference unit 22a and output the result to the information processing device 1a. The provided information acquiring unit 14 of the information processing device 1a acquires the preference inference result output from the information providing unit 25a. The provided information output unit 15 causes the output device 5 to output advertisements, etc., based on the advertisement output control information assigned to the preference inference result acquired by the provided information acquiring unit 14. For example, if the preference inference result is information that associates preference information indicating a preference for luxury jewelry, action information indicating that the occupant has touched earrings, and attribute information indicating that the earrings worn by the occupant are elegant, the information providing unit 25a assigns advertisement output control information for outputting a commercial for luxury brand jewelry to the preference inference result and outputs the result to the information processing device 1a. The provided information acquisition unit 14 of the information processing device 1a acquires the preference inference result, and the provided information output unit 15 causes the display device to display the commercial for luxury brand jewelry.

[0127] In the second embodiment, the inference device 2a includes the information providing unit 25a, but this is merely an example. For example, the inference device 2a may be configured without the information providing unit 25a. For example, the management company of the inference device 2a may provide the inference result storage unit 24a, in which the preference inference results are stored, to various companies and the like.

[0128] In embodiment 2, the learning data acquisition unit 26a acquires, as learning data, the motion information acquired by the motion information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the preference information indicating the preferences inferred by the inference unit 22a. The inference unit 22a may output the preference inference result to the learning data acquisition unit 26a, and the learning data acquisition unit 26a may acquire the preference inference result. In this case, the motion information acquisition unit 21 may not output the motion information to the learning data acquisition unit 26a. Furthermore, the attribute information acquisition unit 28 may not output the attribute information to the learning data acquisition unit 26a. The learning data acquisition unit 26a outputs the acquired learning data to the learning unit 27a.

[0129] In the second embodiment, the learning unit 27a learns the second machine learning model stored in the model storage unit 23a based on the learning data acquired by the learning data acquisition unit 26a. The learning unit 27a updates the second machine learning model stored in the model storage unit 23a to the second machine learning model after learning.

[0130] In the second embodiment, the inference device 2a includes the learning data acquisition unit 26a and the learning unit 27a, but this is merely an example. For example, the inference device 2a may be configured without the learning data acquisition unit 26a and the learning unit 27a. However, by configuring the inference device 2a to include the learning data acquisition unit 26a and the learning unit 27a, the inference device 2a can improve the accuracy of the second machine learning model. As a result, the inference device 2a can improve the accuracy of inferring occupant preferences using the second machine learning model. Furthermore, for example, the functions of the learning data acquisition unit 26a and the learning unit 27a may be provided in a learning device (not shown) that is provided outside the inference device 2a in a location that can be referenced by the inference device 2a.

[0131] The operation of the information processing system 100a according to the second embodiment will be described. FIG. 7 is a flowchart illustrating the operation of the information processing system 100a according to the second embodiment.

[0132] The information processing device 1a detects the actions performed by the occupant and the attributes of the occupant based on the captured image of the occupant captured by the imaging device 4, and outputs action information indicating the detected actions and attribute information indicating the detected attributes of the occupant to the inference device 2a (step ST1a).

[0133] The inference device 2a infers the preferences of the occupant based on the operation information and attribute information output from the information processing device 1a in step ST1a (step ST2a). Then, the inference device 2a stores the preference inference result and provides it to the server 3 or the information processing device 1a.

[0134] The operation of the information processing device 1a according to the second embodiment will be described. FIG. 8 is a flowchart for explaining the operation of the information processing device 1a according to the second embodiment. The operation of the information processing device 1a shown in FIG. 8 corresponds to the operation of step ST1a in the operation of the information processing system 100a described using the flowchart of FIG. For example, when the power supply of the vehicle is turned on, the information processing device 1a repeats the operation shown in the flowchart of FIG. 8 until the power supply of the vehicle is turned off. Of the operations of the information processing device 1a shown in Figure 8, the specific operations of step ST11a, step ST12a-1 to step ST13a-1, and step ST14a to step ST15a are similar to the specific operations of step ST11 to step ST15 by the information processing device 1 according to embodiment 1, which were explained using the flowchart of Figure 3 in embodiment 1, and therefore redundant explanations will be omitted.

[0135] The attribute detection unit 16 detects the attribute of the occupant based on the captured image acquired by the image acquisition unit 11 in step ST11a (step ST12a-2). The attribute detection unit 16 creates attribute information indicating the detected attributes of the occupant, and outputs the created attribute information to the attribute information output unit 17.

[0136] The attribute information output unit 17 outputs the attribute information output from the attribute detection unit 16 in step ST12a-2 to the inference device 2a (step ST13a-2).

[0137] In the flowchart shown in FIG. 8, the processes of steps ST11a to ST13a-1 and ST13a-2 and the processes of steps ST14a to ST15a may be performed in parallel. Furthermore, if the information processing device 1a does not include the provided information acquisition unit 14 and the provided information output unit 15, the operations of the information processing device 1a can omit the processes of steps ST14a to ST15a.

[0138] The operation of the inference device 2a according to the second embodiment will be described. FIG. 9 is a flowchart for explaining the operation of the inference device 2a according to the second embodiment. The operation of the inference device 2a shown in FIG. 9 corresponds to the operation of step ST2a of the operation of the information processing system 100a described using the flowchart of FIG. For example, when the power supply of the inference device 2a is turned on, the inference device 2a repeats the operation shown in the flowchart of FIG. 9 until the power supply of the inference device 2a is turned off. Among the operations of the inference device 2a shown in Figure 9, the specific operations of step ST21a-1 are the same as the specific operations of step ST21 by the inference device 2 according to embodiment 1, which were explained using the flowchart of Figure 4 in embodiment 1, and therefore, redundant explanations will be omitted.

[0139] The attribute information acquisition unit 28 acquires the attribute information output from the information processing device 1a (step ST21a-2). The attribute information acquisition unit 28 outputs the acquired attribute information to the inference unit 22a and the learning data acquisition unit 26a.

[0140] The inference unit 22a infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21 in step ST21a-1 and the attribute information acquired by the attribute information acquisition unit 28 in step ST21a-2 (step ST22a). The inference unit 22a stores the preference inference result in the inference result storage unit 24a. Furthermore, the inference unit 22a outputs preference information indicating the inferred preference to the learning data acquisition unit 26a.

[0141] The information providing unit 25a refers to the inference result storage unit 24a, and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1a) (step ST23a).

[0142] The learning data acquisition unit 26a acquires, as learning data, the motion information acquired by the motion information acquisition unit 21 in step ST21a-1, the attribute information acquired by the attribute information acquisition unit 28 in step ST21a-2, and the preference information indicating the preferences inferred by the inference unit 22a in step ST22a (step ST24a). The learning data acquisition unit 26a outputs the acquired learning data to the learning unit 27a.

[0143] The learning unit 27a learns the second machine learning model stored in the model storage unit 23a based on the learning data acquired by the learning data acquisition unit 26a in step ST24a (step ST25a). The learning unit 27a updates the second machine learning model stored in the model storage unit 23a to the second machine learning model after learning.

[0144] 9, the processing is performed in the order of step ST23a to step ST25a, but this is merely an example. The processing of step ST23a and the processing of steps ST24a to ST25a may be performed in parallel. Furthermore, if the inference device 2a does not include the learning data acquisition unit 26a and the learning unit 27a, the operations of the inference device 2a can omit the processing of steps ST24a to ST25a.

[0145] In this way, in the information processing system 100a, the inference device 2a acquires, from the information processing device 1a, action information indicating the actions performed by the vehicle occupant and attribute information indicating the attributes of the vehicle occupant, and infers the preferences of the occupant based on the acquired action information and attribute information. In this way, the inference device 2a can prevent erroneous inference of the preferences of a vehicle occupant when inferring the preferences of the occupant. Furthermore, since the inference device 2a infers the preferences of an occupant by taking into account attribute information in addition to action information, it can infer preferences in more detail than inferring preferences from action information alone.

[0146] Furthermore, the inference device 2a infers the preferences of the vehicle occupants using captured images of the vehicle occupants, and outputs a preference inference result based on the inferred preferences to the vehicle (specifically, the information processing device 1a) or the server 3. As a result, the inference device 2a can provide services based on the inferred preferences to the vehicle occupants or various companies. Furthermore, since the inference device 2a infers the preferences of the occupants using captured images obtained by a DMS generally provided in the vehicle, it can effectively utilize information obtained in the vehicle.

[0147] The hardware configuration of the information processing device 1a according to the second embodiment is the same as the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 5A and 5, and therefore is not shown in the drawings. In the second embodiment, the functions of the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17 are realized by a processing circuit 1001. That is, the information processing device 1a includes the processing circuit 1001 for detecting the actions and attributes of an occupant based on an image acquired from the imaging device 4, and for controlling output of the action information and attribute information to the inference device 2a. The processing circuit 1001 reads and executes a program stored in the memory 1005, thereby executing the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17. That is, the information processing device 1a includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST11a to ST15a in FIG. 8 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17. The information processing device 1a includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as an inference device 2a, an imaging device 4, or an output device 5.

[0148] The hardware configuration of the inference device 2a according to the second embodiment is similar to the hardware configuration of the inference device 2 described in the first embodiment using FIGS. 5A and 5, and therefore is not shown in the drawings. In the second embodiment, the functions of the motion information acquisition unit 21, the inference unit 22a, the information providing unit 25a, the learning data acquisition unit 26a, the learning unit 27a, and the attribute information acquisition unit 28 are realized by a processing circuit 1001. That is, the inference device 2a includes the processing circuit 1001 for controlling the inference of the preferences of the occupant based on the motion information and attribute information acquired from the information processing device 1a. The processing circuit 1001 reads out and executes a program stored in the memory 1005, thereby executing the functions of the motion information acquisition unit 21, the inference unit 22a, the information providing unit 25a, the learning data acquisition unit 26a, the learning unit 27a, and the attribute information acquisition unit 28. That is, the inference device 2a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST21a-1 to ST25a in FIG. 9 described above. The program stored in the memory 1005 can also be said to cause a computer to execute the processing procedures or methods of the motion information acquisition unit 21, the inference unit 22a, the information providing unit 25a, the learning data acquisition unit 26a, the learning unit 27a, and the attribute information acquisition unit 28. The inference device 2a includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the information processing device 1a or the server 3.

[0149] In the above-described second embodiment, the inference device 2a is connected to one information processing device 1a, but this is merely an example. The inference device 2a may be connected to multiple information processing devices 1a. The inference device 2a acquires motion information and attribute information from each of the information processing devices 1a installed in multiple different vehicles, and can infer the preferences of the occupants of each vehicle based on the acquired motion information and attribute information. Note that the motion information and attribute information output from the information processing device 1a are provided with information that can identify the vehicle.

[0150] Furthermore, in the above-described second embodiment, the model storage unit 23a of the inference device 2a may store multiple machine learning models according to the type of attribute. For example, a machine learning model when the attribute is family structure, a machine learning model when the attribute is related to accessories, a machine learning model when the attribute is related to gender, a machine learning model when the attribute is related to age, etc. may be stored in the model storage unit 23a. The inference unit 22a selects, as the second machine learning model, a machine learning model corresponding to the action information acquired by the attribute information acquisition unit 28 from the multiple machine learning models stored in the model storage unit 23a.

[0151] In the second embodiment, the inference device 2a may be mounted on a vehicle.

[0152] In addition, in the above-described second embodiment, the information processing device 1a is an in-vehicle device mounted on a vehicle, and the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17 are provided in the in-vehicle device. Without being limited to this, for example, some of the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17 may be mounted on the vehicle's on-board device, and the rest may be provided in the inference device 2a. Also, for example, the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, and the attribute information output unit 17 may all be provided in the inference device 2a.

[0153] Furthermore, in the above-described second embodiment, the model memory unit 23a of the inference device 2a may be configured to store a first machine learning model and a second machine learning model, and the inference unit 22a may be configured to infer the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21 and the first machine learning model.

[0154] As described above, according to the second embodiment, the inference device 2a is configured to include the action information acquisition unit 21 that acquires action information indicating actions performed by a vehicle occupant, the attribute information acquisition unit 28 that acquires attribute information indicating attributes of the occupant, and the inference unit 22a that infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21 and the attribute information acquired by the attribute information acquisition unit 28. Therefore, when inferring the preferences of a vehicle occupant, the inference device 2a can prevent erroneous inference of the preferences of the occupant.

[0155] Furthermore, according to the second embodiment, the information processing system 100a is configured to include an information processing device 1a having an image acquisition unit 11 that acquires an image of a vehicle occupant, a motion detection unit 12 that detects motions performed by the occupant based on the image acquired by the image acquisition unit 11, a motion information output unit 13 that outputs motion information indicating the motion of the occupant detected by the motion detection unit 12, an attribute detection unit 16 that detects attributes of the occupant based on the image acquired by the image acquisition unit 11, and an attribute information output unit 17 that outputs attribute information indicating the attributes detected by the attribute detection unit 16, and an inference device 2a having a motion information acquisition unit 21 that acquires the motion information output by the motion information output unit 13, an attribute information acquisition unit 28 that acquires the attribute information output by the attribute information output unit 17, and an inference unit 22a that infers the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21 and the attribute information acquired by the attribute information acquisition unit 28. Therefore, when inferring the preferences of a vehicle occupant, the information processing system 100a can prevent erroneous inference of the preferences of the occupant.

[0156] Embodiment 3 In the first embodiment, the information processing system infers the preferences of a vehicle occupant based on the actions performed by the occupant. In embodiment 3, an embodiment will be described in which an information processing system infers the preferences of a vehicle occupant by taking into account not only the actions performed by the vehicle occupant but also information related to the vehicle (hereinafter referred to as "vehicle-related information"). In the following embodiment 3, the occupant is assumed to be the driver of the vehicle, as in embodiment 1. However, this is merely an example, and in embodiment 2, the occupant may be an occupant other than the driver of the vehicle, and the information processing system can infer the preferences of multiple occupants.

[0157] FIG. 10 is a diagram illustrating an example of a configuration of an information processing system 100b according to the third embodiment. 10, the same components as those in the information processing system 100 described in the first embodiment with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. In an information processing system 100b according to embodiment 3, exemplary configurations of an information processing device 1b and an inference device 2b differ from the exemplary configurations of the information processing device 1 and the inference device 2 in the information processing system 100 according to embodiment 1. Details of the exemplary configurations of the information processing device 1b and the inference device 2b will be described later. The information processing system 100b infers the preferences of a vehicle occupant based on the vehicle-related information and the action information indicating the action performed by the vehicle occupant, and provides the preference inference results to various devices and the like.

[0158] The information processing device 1b detects the actions performed by the occupant based on the captured image of the occupant captured by the imaging device 4. The information processing device 1b also collects vehicle-related information. In the third embodiment, the vehicle-related information includes, for example, vehicle position information, vehicle steering angle information, vehicle speed information, accelerator position information, or navigation information. The navigation information includes map information and guide route information. The vehicle-related information may include a plurality of different types of information, such as vehicle position information, navigation information, and vehicle speed information. The information processing device 1b collects the vehicle-related information from various devices (not shown) provided in the vehicle, such as a GPS, a steering angle sensor, a vehicle speed sensor, an accelerator position sensor, or a navigation device. The information processing device 1b outputs the detected movement information indicating the movement of the occupant and the collected vehicle-related information to the inference device 2b. The inference device 2b infers the preferences of the occupant based on the operation information and vehicle-related information output from the information processing device 1b. The inference device 2b stores the preference inference result and provides the preference inference result to the server 3 or the information processing device 1b.

[0159] An example of the configuration of the information processing device 1b and the inference device 2b will be described with reference to FIG.

[0160] Regarding the information processing device 1b according to the third embodiment shown in FIG. 10, the same components as those of the information processing device 1 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant description will be omitted. The information processing device 1b according to the third embodiment differs from the information processing device 1 according to the first embodiment in that it includes a vehicle-related information collection unit 18 and a vehicle-related information output unit 19.

[0161] The vehicle-related information collection unit 18 collects vehicle-related information. The vehicle-related information collection unit 18 outputs the collected vehicle-related information to the vehicle-related information output unit 19.

[0162] The vehicle-related information output unit 19 outputs the vehicle-related information output from the vehicle-related information collection unit 18 to the inference device 2b.

[0163] Regarding the inference device 2b according to the third embodiment shown in FIG. 10, the same components as those of the inference device 2 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant explanations will be omitted. The inference device 2 b according to the third embodiment differs from the inference device 2 according to the first embodiment in that it includes a vehicle-related information acquisition unit 29 . Furthermore, the specific operations of inference unit 22b, information providing unit 25b, learning data acquiring unit 26b, and learning unit 27b in inference device 2b according to embodiment 3 differ from the specific operations of inference unit 22, information providing unit 25, learning data acquiring unit 26, and learning unit 27 in inference device 2 according to embodiment 1. Furthermore, the specific contents of the information stored in model storage unit 23b and inference result storage unit 24b in inference device 2 according to embodiment 3 differ from the specific contents of the information stored in model storage unit 23 and inference result storage unit 24 in inference device 2 according to embodiment 1, respectively.

[0164] The vehicle-related information acquisition unit 29 acquires the vehicle-related information output from the information processing device 1b. The vehicle-related information acquisition unit 29 outputs the acquired vehicle-related information to the inference unit 22b and the learning data acquisition unit 26b.

[0165] In the third embodiment, the inference unit 22b infers the preferences of the occupant based on the operation information acquired by the operation information acquisition unit 21 and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. For example, the inference unit 22b infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21, the vehicle-related information acquired by the vehicle-related information acquisition unit 29, and a machine learning model (hereinafter referred to as the "third machine learning model"). The third machine learning model is a machine learning model that receives action information and vehicle-related information as input and outputs preference information indicating preferences. The third machine learning model is created in advance by an administrator or the like and stored in the model storage unit 23b. For example, the administrator or the like prepares multiple subjects in advance and has the subjects test drive the vehicle to create the third machine learning model. The third machine learning model is a machine learning model created by learning using the action information and vehicle-related information obtained during the test drive and preference information indicating the preferences of the occupant obtained by interviewing the occupant as training data. Note that the preference information is a training label.

[0166] For example, if the action information is information indicating that the occupant has moved their gaze, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22b inputs the action information and the vehicle-related information into a third machine learning model to obtain preference information indicating that the occupant is interested in a store that the occupant saw, in other words, a store that is in the occupant's line of sight. The inference unit 22b can also infer the name or genre of the store based on the navigation information. The store genre is preset. For example, "Japanese food" or "Western food" is preset as the genre of a restaurant. Also, for example, "women's" or "men's" is preset as the genre of a clothing store. The genre may be further classified into subcategories. For example, the restaurant genre "Japanese food" may be classified into subcategories such as sushi, soba or udon, and okonomiyaki, while the restaurant genre "Western food" may be classified into subcategories such as steak, hamburger steak, pasta, and hamburger. For example, if a clothing store has a category called "ladies," it may be divided into subcategories such as casual, feminine, girly, sporty, and fashionable, while if it has a category called "men's," it may be divided into subcategories such as casual, street, and outdoor. For example, the inference unit 22b inputs behavior information indicating that the occupant has moved their gaze and vehicle-related information including vehicle position information and navigation information into a third machine learning model to infer the occupant's preference, such as "I prefer hamburgers from fast food chain store X."

[0167] Furthermore, for example, if the action information is information indicating that the occupant has dismounted, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22b inputs the action information and the vehicle-related information into the third machine learning model to obtain preference information indicating that the occupant is interested in shopping mall Y located on the premises of the point where the occupant dismounted. As a result, the inference unit 22b infers the occupant's preference that "he likes shopping mall Y."

[0168] The inference unit 22b may infer the preferences of the occupant using a method other than the method using the third machine learning model. For example, the inference unit 22b may infer the preference of the occupant by comparing the action information acquired by the action information acquisition unit 21 and the attribute information acquired by the vehicle-related information acquisition unit 29 with the preference inference conditions. In the third embodiment, the preference inference conditions define, for example, what preference is inferred when what action the occupant performs in what state of the vehicle.

[0169] The inference unit 22b stores the preference inference result in the inference result storage unit 24b. The preference inference result is, for example, information in which preference information indicating the preference inferred by the inference unit 22b, action information, and vehicle-related information are associated with each other. The action information and vehicle-related information associated with the preference information are the action information and vehicle-related information that were input to the third machine learning model when the inference unit 22b obtained the preference information, or the action information and vehicle-related information that were compared with the preference inference conditions when the inference unit 22b obtained the preference information. Furthermore, the inference unit 22b outputs preference information indicating the inferred preference to the learning data acquisition unit 26b.

[0170] In the third embodiment, the model storage unit 23b stores a third machine learning model. In the third embodiment, the model storage unit 23b is provided in the inference device 2b, but this is merely an example. The model storage unit 23b may be provided in a location outside the inference device 2b that can be referenced by the inference device 2b.

[0171] In the third embodiment, the inference result storage unit 24b stores the preference inference result output by the inference unit 22b. In the third embodiment, the inference result storage unit 24b is provided in the inference device 2b, but this is merely an example. The inference result storage unit 24b may be provided in a location outside the inference device 2b that can be referenced by the inference device 2b.

[0172] In the third embodiment, the information providing unit 25b refers to the inference result storage unit 24b and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1b). The information providing unit 25b determines the server 3 or the information processing device 1b to which the preference inference result is to be provided, depending on the content of the preference inference result. It is predetermined which server 3 or information processing device 1b the preference inference result is to be output to depending on the content of the preference inference result.

[0173] For example, if the preference inference result is information that correlates preference information indicating that shopping mall Y is preferred, operation information indicating that the occupant has disembarked, and vehicle-related information including vehicle location information and navigation information, the information providing unit 25b outputs the preference inference result to a server 3 provided in a system owned by a research company. For example, the server 3 stores the preference inference results output from the information providing unit 25b. The server 3 analyzes popular commercial facilities based on the stored preference inference results. A research company can use the analysis results performed by the server 3 to conduct research to solve problems in a company's marketing strategy.

[0174] Furthermore, for example, if the management company of the inference device 2b has a business partnership with an advertising company and information about advertisements is stored in a location accessible by the information providing unit 25b, the information providing unit 25b may assign advertisement output control information to the preference inference result in accordance with the preference inference result output by the inference unit 22b, and output the result to the information processing device 1b. The provided information acquiring unit 14 of the information processing device 1b acquires the preference inference result output from the information providing unit 25b. The provided information output unit 15 causes the output device 5 to output advertisements, etc., based on the advertisement output control information assigned to the preference inference result acquired by the provided information acquiring unit 14. For example, if the preference inference result is information that associates preference information indicating that the occupant prefers hamburgers from fast food chain store X, action information indicating that the occupant has moved their line of sight, and vehicle-related information including vehicle position information and navigation information, the information providing unit 25b assigns advertisement output control information for outputting a commercial for hamburgers from fast food chain store X to the preference inference result and outputs the result to the information processing device 1b. The provided information acquisition unit 14 of the information processing device 1b acquires the preference inference result, and the provided information output unit 15 causes the display device to display the commercial for hamburgers from fast food chain store X.

[0175] In the third embodiment, the inference device 2b includes the information providing unit 25b, but this is merely an example. For example, the inference device 2b may be configured without the information providing unit 25b. For example, the management company of the inference device 2b may provide the inference result storage unit 24b, in which the preference inference results are stored, to various companies and the like.

[0176] In embodiment 3, the learning data acquisition unit 26b acquires, as learning data, the operation information acquired by the operation information acquisition unit 21, the vehicle-related information acquired by the vehicle-related information acquisition unit 29, and the preference information indicating the preferences inferred by the inference unit 22b. The inference unit 22b may output the preference inference result to the learning data acquisition unit 26b, and the learning data acquisition unit 26b may acquire the preference inference result. In this case, the operation information acquisition unit 21 may not output the operation information to the learning data acquisition unit 26b. Furthermore, the vehicle-related information acquisition unit 29 may not output the vehicle-related information to the learning data acquisition unit 26b. The learning data acquisition unit 26b outputs the acquired learning data to the learning unit 27b.

[0177] In the third embodiment, the learning unit 27b learns the third machine learning model stored in the model storage unit 23b based on the learning data acquired by the learning data acquisition unit 26b. The learning unit 27b updates the third machine learning model stored in the model storage unit 23b to the learned third machine learning model.

[0178] In the third embodiment, the inference device 2b includes the learning data acquisition unit 26b and the learning unit 27b, but this is merely an example. For example, the inference device 2b may be configured without the learning data acquisition unit 26b and the learning unit 27b. However, by configuring the inference device 2b to include the learning data acquisition unit 26b and the learning unit 27b, the inference device 2b can improve the accuracy of the third machine learning model. As a result, the inference device 2b can improve the accuracy of inferring the occupant's preferences using the third machine learning model. Furthermore, for example, the functions of the learning data acquisition unit 26b and the learning unit 27b may be provided by a learning device (not shown) that is provided outside the inference device 2b in a location that can be referenced by the inference device 2b.

[0179] The operation of the information processing system 100b according to the third embodiment will be described. FIG. 11 is a flowchart for explaining the operation of the information processing system 100b according to the third embodiment.

[0180] The information processing device 1b detects the movement performed by the occupant based on the captured image of the occupant captured by the imaging device 4. The information processing device 1b also collects vehicle-related information. The information processing device 1b outputs the movement information indicating the detected movement and the collected vehicle-related information to the inference device 2b (step ST1b).

[0181] The inference device 2b infers the preferences of the occupant based on the operation information and vehicle-related information output from the information processing device 1b in step ST1b (step ST2b). Then, the inference device 2b stores the preference inference result and provides it to the server 3 or the information processing device 1b.

[0182] The operation of the information processing device 1b according to the third embodiment will be described. FIG. 12 is a flowchart for explaining the operation of the information processing device 1b according to the third embodiment. The operation of the information processing device 1b shown in FIG. 12 corresponds to the operation of step ST1b of the operation of the information processing system 100b described using the flowchart of FIG. For example, when the power supply of the vehicle is turned on, the information processing device 1b repeats the operation shown in the flowchart of FIG. 12 until the power supply of the vehicle is turned off. Of the operations of the information processing device 1b shown in Figure 12, the specific operations of steps ST11b-1, ST12b, ST13b-1, and steps ST14b to ST15b are similar to the specific operations of steps ST11 to ST15 by the information processing device 1 according to embodiment 1, which were described using the flowchart of Figure 3 in embodiment 1, and therefore redundant explanations will be omitted.

[0183] The vehicle-related information collection unit 18 collects vehicle-related information (step ST11b-2). The vehicle-related information collection unit 18 outputs the collected vehicle-related information to the vehicle-related information output unit 19.

[0184] The vehicle-related information output unit 19 outputs the vehicle-related information output from the vehicle-related information collection unit 18 in step ST11b-2 to the inference device 2b (step ST13b-2).

[0185] In the flowchart shown in FIG. 12, the processes of step ST11b-1, step ST11b-2 to step ST13b-1, and step ST13b-2 and the processes of step ST14b to step ST15b may be performed in parallel. Furthermore, if the information processing device 1b does not include the provided information acquisition unit 14 and the provided information output unit 15, the operations of the information processing device 1b can omit the processes of steps ST14b to ST15b.

[0186] The operation of the inference device 2b according to the third embodiment will be described. FIG. 13 is a flowchart for explaining the operation of the inference device 2b according to the third embodiment. The operation of inference device 2b shown in FIG. 13 corresponds to the operation of step ST2b of information processing system 100b described using the flowchart of FIG. For example, when the power supply of the inference device 2b is turned on, the inference device 2b repeats the operation shown in the flowchart of FIG. 13 until the power supply of the inference device 2b is turned off. Among the operations of the inference device 2b shown in Figure 13, the specific operations of step ST21b-1 are the same as the specific operations of step ST21 by the inference device 2 according to embodiment 1, which were explained using the flowchart of Figure 4 in embodiment 1, and therefore, redundant explanations will be omitted.

[0187] The vehicle-related information acquisition unit 29 acquires the vehicle-related information output from the information processing device 1b (step ST21b-2). The vehicle-related information acquisition unit 29 outputs the acquired vehicle-related information to the inference unit 22b and the learning data acquisition unit 26b.

[0188] The inference unit 22b infers the preferences of the occupant based on the operation information acquired by the operation information acquisition unit 21 in step ST21b-1 and the vehicle-related information acquired by the vehicle-related information acquisition unit 29 in step ST21b-2 (step ST22b). The inference unit 22b stores the preference inference result in the inference result storage unit 24b. Furthermore, the inference unit 22b outputs preference information indicating the inferred preference to the learning data acquisition unit 26b.

[0189] The information providing unit 25b refers to the inference result storage unit 24b, and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1b) (step ST23b).

[0190] The learning data acquisition unit 26b acquires, as learning data, the operation information acquired by the operation information acquisition unit 21 in step ST21b-1, the vehicle-related information acquired by the vehicle-related information acquisition unit 29 in step ST21b-2, and the preference information indicating the preferences inferred by the inference unit 22b in step ST22b (step ST24b). The learning data acquisition unit 26b outputs the acquired learning data to the learning unit 27b.

[0191] The learning unit 27b learns the third machine learning model stored in the model storage unit 23b based on the learning data acquired by the learning data acquisition unit 26b in step ST24b (step ST25b). The learning unit 27b updates the third machine learning model stored in the model storage unit 23b to the learned third machine learning model.

[0192] 13, the processing is performed in the order of step ST23b to step ST25b, but this is merely an example. The processing of step ST23b and the processing of steps ST24b to ST25b may be performed in parallel. Furthermore, if the inference device 2b does not include the learning data acquisition unit 26b and the learning unit 27b, the operations of the inference device 2b can omit the processing of steps ST24b to ST25b.

[0193] In this way, in the information processing system 100b, the inference device 2b acquires, from the information processing device 1b, action information indicating actions performed by the vehicle occupant and vehicle-related information, and infers the occupant's preferences based on the acquired action information and vehicle-related information. In this way, the inference device 2b can prevent erroneous inference of the preferences of the vehicle occupant when inferring the preferences of the vehicle occupant. Furthermore, since the inference device 2b infers the preferences of the occupant by taking into account vehicle-related information in addition to motion information, it can infer preferences in more detail than inferring the preferences of the occupant from motion information alone.

[0194] Furthermore, the inference device 2b infers the preferences of the vehicle occupants using captured images of the vehicle occupants, and outputs a preference inference result based on the inferred preferences to the vehicle (specifically, the information processing device 1b) or the server 3. As a result, the inference device 2b can provide services based on the inferred preferences to the vehicle occupants or various companies. Furthermore, since the inference device 2b infers the preferences of the occupants using captured images obtained by a DMS generally provided in the vehicle, it can effectively utilize information obtained in the vehicle.

[0195] The hardware configuration of the information processing device 1b according to the third embodiment is the same as the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 5A and 5, and therefore is not shown in the drawings. In the third embodiment, the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 are realized by a processing circuit 1001. That is, the information processing device 1b includes the processing circuit 1001 for detecting the motion of an occupant based on an image acquired from the imaging device 4, collecting vehicle-related information from various devices, and performing control to output the motion information and the vehicle-related information to the inference device 2b. The processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19. That is, the information processing device 1b includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of step ST11b-1 and step ST11b-2 to step ST15b in FIG. 12 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the processing procedures or methods of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19. The information processing device 1b includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as an inference device 2b, an imaging device 4, an output device 5, or various other devices not shown.

[0196] The hardware configuration of the inference device 2b according to the third embodiment is similar to the hardware configuration of the inference device 2 described in the first embodiment using FIGS. 5A and 5, and therefore is not shown in the drawings. In the third embodiment, the functions of the operation information acquisition unit 21, the inference unit 22b, the information provision unit 25b, the learning data acquisition unit 26b, the learning unit 27b, and the vehicle-related information acquisition unit 29 are realized by a processing circuit 1001. That is, the inference device 2b includes the processing circuit 1001 for controlling inference of the preferences of the occupant based on the operation information and vehicle-related information acquired from the information processing device 1b. The processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the operation information acquisition unit 21, the inference unit 22b, the information provision unit 25b, the learning data acquisition unit 26b, the learning unit 27b, and the vehicle-related information acquisition unit 29. That is, the inference device 2b includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of step ST21b-1 and step ST21b-2 to step ST25b in FIG. 13 described above. It can also be said that the programs stored in the memory 1005 cause a computer to execute the processing procedures or methods of the operation information acquisition unit 21, the inference unit 22b, the information provision unit 25b, the learning data acquisition unit 26b, the learning unit 27b, and the vehicle-related information acquisition unit 29. The inference device 2b includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the information processing device 1b or the server 3.

[0197] In the above-described third embodiment, the inference device 2b is connected to one information processing device 1b, but this is merely an example. The inference device 2b may be connected to multiple information processing devices 1b. The inference device 2b acquires operation information and vehicle-related information from the information processing devices 1b installed in multiple different vehicles, and can infer the preferences of the occupants of each vehicle based on the acquired operation information and vehicle-related information. The operation information and vehicle-related information output from the information processing device 1b are accompanied by information that can identify the vehicle.

[0198] Furthermore, in the above-described third embodiment, the model storage unit 23b of the inference device 2b may store multiple machine learning models according to the type of vehicle-related information. For example, machine learning models according to regions, machine learning models according to road types, etc. may be stored in the model storage unit 23b. The inference unit 22b selects, from the multiple machine learning models stored in the model storage unit 23b, a machine learning model according to the operation information acquired by the vehicle-related information acquisition unit 29 as the third machine learning model.

[0199] Furthermore, in the above third embodiment, the inference device 2b acquires vehicle-related information via the information processing device 1b, but this is merely an example. For example, the inference device 2b may acquire vehicle-related information directly from various devices provided in the vehicle. In this case, the information processing device 1b may be configured not to include the vehicle-related information collection unit 18 and the vehicle-related information output unit 19.

[0200] In the third embodiment, the inference device 2b may be mounted on a vehicle.

[0201] In addition, in the above-mentioned embodiment 3, the information processing device 1b is an in-vehicle device mounted on a vehicle, and the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 are provided in the in-vehicle device. Without being limited to this, for example, some of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 may be mounted on the vehicle's on-board device, and the rest may be provided in the inference device 2b. Also, for example, the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 may all be provided in the inference device 2b.

[0202] Furthermore, in the above-described third embodiment, the model storage unit 23b of the inference device 2b may store a first machine learning model and a third machine learning model, and the inference unit 22b may be capable of inferring the preferences of the occupant based on the first machine learning model and the motion information acquired by the motion information acquisition unit 21. Furthermore, the model storage unit 23b may further store a second machine learning model, and the inference unit 22b may be capable of inferring the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the second machine learning model.

[0203] As described above, according to the third embodiment, the inference device 2b is configured to include the action information acquisition unit 21 that acquires action information indicating actions performed by a vehicle occupant, the vehicle-related information acquisition unit 29 that acquires vehicle-related information, and the inference unit 22b that infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21 and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. Therefore, when inferring the preferences of a vehicle occupant, the inference device 2b can prevent erroneous inference of the preferences of the occupant.

[0204] Furthermore, according to the third embodiment, the information processing system 100b is configured to include an information processing device 1b having an image acquisition unit 11 that acquires captured images of a vehicle occupant, a motion detection unit 12 that detects motions performed by the occupant based on the captured images acquired by the image acquisition unit 11, and a motion information output unit 13 that outputs motion information indicating the motion of the occupant detected by the motion detection unit 12, and an inference device 2b having a motion information acquisition unit 21 that acquires the motion information output by the motion information output unit 13, a vehicle-related information acquisition unit 29 that acquires vehicle-related information, and an inference unit 22b that infers the preferences of the occupant based on the motion information acquired by the motion information acquisition unit 21 and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. Therefore, the information processing system 100b can prevent erroneous inference of the preferences of the vehicle occupant when inferring the preferences of the vehicle occupant.

[0205] Embodiment 4 In the fourth embodiment, an embodiment will be described in which the preferences of a vehicle occupant are inferred in an information processing system by taking into consideration the actions performed by the occupant, as well as the attributes of the occupant and vehicle-related information. In the following embodiment 4, the occupant is assumed to be the driver of the vehicle, as in embodiment 1. However, this is merely an example, and in embodiment 4, the occupant may be an occupant other than the driver of the vehicle, and the information processing system can infer the preferences of multiple occupants.

[0206] FIG. 14 is a diagram illustrating an example of a configuration of an information processing system 100c according to the fourth embodiment. 14, the same components as those in the information processing system 100 described in the first embodiment with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. In an information processing system 100c according to embodiment 4, exemplary configurations of an information processing device 1c and an inference device 2c differ from the exemplary configurations of the information processing device 1 and the inference device 2 in the information processing system 100 according to embodiment 1. Details of the exemplary configurations of the information processing device 1c and the inference device 2c will be described later. The information processing system 100c infers the preferences of a vehicle occupant based on action information indicating the actions performed by the vehicle occupant, attribute information indicating the attributes of the vehicle occupant, and vehicle-related information, and provides the preference inference results to various devices, etc.

[0207] The information processing device 1c detects the actions performed by the occupant and detects the attributes of the occupant based on the captured image of the occupant captured by the imaging device 4. The information processing device 1c also collects vehicle-related information. The information processing device 1c outputs, to the inference device 2c, movement information indicating the detected movement of the occupant, attribute information indicating the detected attribute of the occupant, and the collected vehicle-related information. The inference device 2c infers the preferences of the occupant based on the operation information, attribute information, and vehicle-related information output from the information processing device 1c. The inference device 2c stores the preference inference result and provides the preference inference result to the server 3 or the information processing device 1c.

[0208] An example of the configuration of an information processing device 1c and an inference device 2c will be described with reference to FIG.

[0209] Regarding the information processing device 1c according to the fourth embodiment shown in FIG. 14, the same components as those of the information processing device 1 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant description will be omitted. The information processing device 1c of embodiment 4 differs from the information processing device 1 of embodiment 1 in that it includes an attribute detection unit 16, an attribute information output unit 17, a vehicle-related information collection unit 18, and a vehicle-related information output unit 19. The attribute detection unit 16 and the attribute information output unit 17 are the attribute detection unit 16 and the attribute information output unit 17 that are already described and included in the information processing device 1a according to the second embodiment, so a duplicated description will be omitted. The vehicle-related information collection unit 18 and the vehicle-related information output unit 19 are the vehicle-related information collection unit 18 and the vehicle-related information output unit 19 that were already provided in the information processing device 1b of embodiment 3, and therefore redundant explanations will be omitted.

[0210] Regarding the inference device 2c according to the fourth embodiment shown in FIG. 14, the same components as those of the inference device 2 described in the first embodiment using FIG. 1 are assigned the same reference numerals and redundant explanations will be omitted. The inference device 2c according to the fourth embodiment differs from the inference device 2 according to the first embodiment in that it includes an attribute information acquisition unit 28 and a vehicle-related information acquisition unit 29. Furthermore, the specific operations of inference unit 22c, information providing unit 25c, learning data acquiring unit 26c, and learning unit 27c in inference device 2c according to embodiment 4 differ from the specific operations of inference unit 22, information providing unit 25, learning data acquiring unit 26, and learning unit 27 in inference device 2 according to embodiment 1. Furthermore, the specific contents of the information stored in model storage unit 23c and inference result storage unit 24c in inference device 2 according to embodiment 4 differ from the specific contents of the information stored in model storage unit 23 and inference result storage unit 24 in inference device 2 according to embodiment 1, respectively.

[0211] The attribute information acquisition unit 28 is the attribute information acquisition unit 28 that has already been described and that is included in the inference device 2a according to the second embodiment, so a duplicated description will be omitted. The vehicle-related information acquisition unit 29 is the vehicle-related information acquisition unit 29 that is included in the inference device 2b according to the third embodiment, which has already been described, and therefore a duplicated description will be omitted.

[0212] In embodiment 4, the inference unit 22c infers the preferences of the occupant based on the operation information acquired by the operation information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. For example, the inference unit 22c infers the occupant's preferences based on the action information acquired by the action information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, the vehicle-related information acquired by the vehicle-related information acquisition unit 29, and a machine learning model (hereinafter referred to as the "fourth machine learning model"). The fourth machine learning model is a machine learning model that receives action information, attribute information, and vehicle-related information as input and outputs preference information indicating preferences. The fourth machine learning model is created in advance by an administrator or the like and stored in the model storage unit 23c. For example, the administrator or the like prepares multiple subjects in advance and has the subjects test drive the vehicle to create the fourth machine learning model. The fourth machine learning model is a machine learning model created by learning using the action information, attribute information, and vehicle-related information obtained during the test drive, and preference information indicating the occupant's preferences obtained by interviewing the occupant, as training data. The preference information is a training label.

[0213] For example, if the action information is information indicating that the occupant has moved their line of sight, the attribute information is information indicating that the family has a one-year-old boy, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22c inputs the action information, attribute information, and vehicle-related information into the fourth machine learning model to obtain preference information indicating that the occupant is interested in clothes for little boys. As a result, the inference unit 22c infers the occupant's preference that "I'm interested in clothes for little boys."

[0214] Furthermore, for example, if the action information is information indicating that the occupant has dismounted, the attribute information is information indicating that the occupant is wearing luxury brand accessories, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22c inputs the action information, attribute information, and vehicle-related information into the fourth machine learning model to obtain preference information indicating that the occupant is interested in luxury brand accessory store W located in department store Z on the premises of the location where the occupant dismounted. As a result, the inference unit 22c infers the occupant's preference that "the occupant likes products sold in luxury brand accessory store W located in department store Z."

[0215] Furthermore, for example, if the action information is information indicating that the occupant has moved their line of sight, the attribute information is information indicating that the occupant is in their twenties, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22c inputs the action information, attribute information, and vehicle-related information into a fourth machine learning model to obtain preference information indicating that the occupant in their twenties is interested in the land (address: XXX-XXX) in the direction in which the occupant is looking. As a result, the inference unit 22c infers the occupant's preference that "the occupant in their twenties is interested in the land with address: XXX-XXX."

[0216] For example, if the behavior information is information indicating that the occupant has moved their line of sight, the attribute information is information including information indicating that the occupant is male and information indicating that the occupant is large, and the vehicle-related information is vehicle position information and navigation information, the inference unit 22c inputs the behavior information, attribute information, and vehicle-related information into a fourth machine learning model to obtain preference information indicating that large men are interested in clothing stores that specialize in plus sizes. As a result, the inference unit 22c infers the occupant's preference that "I'm interested in clothing stores that specialize in plus sizes."

[0217] The inference unit 22c may infer the preferences of the occupant using a method other than the method using the fourth machine learning model. For example, the inference unit 22c may infer the preference of the occupant by comparing the operation information acquired by the operation information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the vehicle-related information acquired by the vehicle-related information acquisition unit 29 with the preference inference conditions. In the fourth embodiment, the preference inference conditions define, for example, what preference is inferred when an occupant with what attributes performs what action in what state of the vehicle.

[0218] The inference unit 22c stores the preference inference result in the inference result storage unit 24c. The preference inference result is, for example, information in which preference information indicating the preference inferred by the inference unit 22c, action information, attribute information, and vehicle-related information are associated with each other. The action information / attribute information and vehicle-related information associated with the preference information are the action information, attribute information, and vehicle-related information that were input to the fourth machine learning model when the inference unit 22c obtained the preference information, or the action information, attribute information, and vehicle-related information that were compared with the preference inference conditions when the inference unit 22c obtained the preference information. Furthermore, the inference unit 22c outputs preference information indicating the inferred preference to the learning data acquisition unit 26c.

[0219] In the fourth embodiment, the model storage unit 23c stores a fourth machine learning model. In the fourth embodiment, the model storage unit 23c is provided in the inference device 2c, but this is merely an example. The model storage unit 23c may be provided in a location outside the inference device 2c that can be referenced by the inference device 2c.

[0220] In the fourth embodiment, the inference result storage unit 24c stores the preference inference result output by the inference unit 22c. In the fourth embodiment, the inference result storage unit 24c is provided in the inference device 2c, but this is merely an example. The inference result storage unit 24c may be provided in a location outside the inference device 2c that can be referenced by the inference device 2c.

[0221] In the fourth embodiment, the information providing unit 25c refers to the inference result storage unit 24c and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1c). The information providing unit 25c determines the server 3 or the information processing device 1c to which the preference inference result is to be provided, depending on the content of the preference inference result. It is predetermined which server 3 or information processing device 1c the preference inference result is to be output to depending on the content of the preference inference result.

[0222] For example, if the preference inference result is information that correlates preference information indicating that a passenger in his / her twenties is interested in land at address XXX-△△, action information indicating that the passenger has moved his / her gaze, attribute information indicating that the passenger is in his / her twenties, and vehicle-related information including vehicle location information and navigation information, the information providing unit 25c outputs the preference inference result to a server 3 provided in a system owned by a real estate company. For example, the server 3 stores the preference inference results output from the information providing unit 25c. Based on the preference inference results stored by the server 3, the real estate company can narrow down the target age group for each piece of land and develop a strategy for what kind of facilities should be built on that piece of land. For example, based on the preference that passengers in their 20s are interested in land with the address XXX-△△, the real estate company can develop a strategy to build an entertainment facility (karaoke, game center, etc.) on that piece of land that mainly targets young people.

[0223] Furthermore, for example, if the management company of the inference device 2c has a business partnership with an advertising company and information about advertisements is stored in a location accessible by the information providing unit 25c, the information providing unit 25c may assign advertisement output control information to the preference inference result in accordance with the preference inference result output by the inference unit 22c, and output the result to the information processing device 1c. The provided information acquiring unit 14 of the information processing device 1c acquires the preference inference result output from the information providing unit 25c. The provided information output unit 15 causes the output device 5 to output advertisements, etc., based on the advertisement output control information assigned to the preference inference result acquired by the provided information acquiring unit 14. For example, if the preference inference result is information that associates preference information indicating an interest in plus-size clothing with action information indicating that the occupant has moved their line of sight and attribute information including information indicating that the occupant is male and information indicating that the occupant is large-sized, the information providing unit 25c assigns advertisement output control information to the preference inference result for outputting a commercial for a fashion brand that sells plus-size men's clothing and outputs the result to the information processing device 1c. The provided information acquisition unit 14 of the information processing device 1c acquires the preference inference result, and the provided information output unit 15 causes the display device to display a commercial for a fashion brand that sells plus-size men's clothing.

[0224] In the fourth embodiment, the inference device 2c includes the information providing unit 25c, but this is merely an example. For example, the inference device 2c may be configured without the information providing unit 25c. For example, the management company of the inference device 2c may provide the inference result storage unit 24c, in which the preference inference results are stored, to various companies and the like.

[0225] In embodiment 4, the learning data acquisition unit 26c acquires, as learning data, the operation information acquired by the operation information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, the vehicle-related information acquired by the vehicle-related information acquisition unit 29, and the preference information indicating the preferences inferred by the inference unit 22c. The inference unit 22c may output the preference inference result to the learning data acquisition unit 26c, and the learning data acquisition unit 26c may acquire the preference inference result. In this case, the motion information acquisition unit 21 may not output the motion information to the learning data acquisition unit 26c. Furthermore, the attribute information acquisition unit 28 may not output the attribute information to the learning data acquisition unit 26c. The learning data acquisition unit 26c outputs the acquired learning data to the learning unit 27c.

[0226] In the fourth embodiment, the learning unit 27c learns the fourth machine learning model stored in the model storage unit 23c based on the learning data acquired by the learning data acquisition unit 26c. The learning unit 27c updates the fourth machine learning model stored in the model storage unit 23c to the fourth machine learning model after learning.

[0227] In the fourth embodiment, the inference device 2c includes the learning data acquisition unit 26c and the learning unit 27c, but this is merely an example. For example, the inference device 2c may be configured without the learning data acquisition unit 26c and the learning unit 27c. However, by configuring the inference device 2c to include the learning data acquisition unit 26c and the learning unit 27c, the inference device 2c can improve the accuracy of the fourth machine learning model. As a result, the inference device 2c can improve the accuracy of inferring occupant preferences using the fourth machine learning model. Furthermore, for example, the functions of the learning data acquisition unit 26c and the learning unit 27c may be provided by a learning device (not shown) that is provided outside the inference device 2c in a location that can be referenced by the inference device 2c.

[0228] The operation of the information processing system 100c according to the fourth embodiment will be described. FIG. 15 is a flowchart illustrating the operation of the information processing system 100c according to the fourth embodiment.

[0229] The information processing device 1c detects the actions performed by the occupant and the attributes of the occupant based on the captured image of the occupant captured by the imaging device 4. The information processing device 1c also collects vehicle-related information. The information processing device 1c outputs action information indicating the detected actions, attribute information indicating the detected attributes, and the collected vehicle-related information to the inference device 2c (step ST1c).

[0230] The inference device 2c infers the occupant's preferences based on the operation information, attribute information, and vehicle-related information output from the information processing device 1c in step ST1c (step ST2c).Then, the inference device 2c stores the preference inference result and provides it to the server 3 or the information processing device 1c.

[0231] The operation of the information processing device 1c according to the fourth embodiment will be described. FIG. 16 is a flowchart for explaining the operation of the information processing device 1c according to the fourth embodiment. The operation of the information processing device 1c shown in FIG. 16 corresponds to the operation of step ST1c in the operation of the information processing system 100c described using the flowchart of FIG. For example, when the power supply of the vehicle is turned on, the information processing device 1c repeats the operation shown in the flowchart of FIG. 16 until the power supply of the vehicle is turned off.

[0232] Of the operations of the information processing device 1c shown in Figure 16, the specific operations of steps ST11c-1, ST12c-1, ST13c-1, and steps ST14c to ST15c are similar to the specific operations of steps ST11 to ST15 by the information processing device 1 according to embodiment 1, which were described using the flowchart of Figure 3 in embodiment 1, and therefore redundant explanations will be omitted.

[0233] Furthermore, among the operations of the information processing device 1c shown in Figure 16, the specific operations of steps ST12c-2 and ST13c-2 are similar to the specific operations of steps ST12a-2 and ST13a-2 by the information processing device 1a according to embodiment 2, which were explained using the flowchart of Figure 8 in embodiment 2, and therefore redundant explanations will be omitted.

[0234] Furthermore, among the operations of the information processing device 1c shown in Figure 16, the specific operations of steps ST11c-2 and ST13c-3 are similar to the specific operations of steps ST11b-2 and ST13b-2 by the information processing device 1b according to embodiment 3, which were explained using the flowchart of Figure 12 in embodiment 3, and therefore redundant explanations will be omitted.

[0235] In the flowchart shown in FIG. 16, the processes of steps ST11c-1 to ST13c-1, ST13c-2, and ST11c-2 to ST13c-3 may be performed in parallel with the processes of steps ST14c to ST15c. Furthermore, if the information processing device 1c does not include the provided information acquisition unit 14 and the provided information output unit 15, the operations of the information processing device 1c can omit the processes of steps ST14c to ST15c.

[0236] The operation of the inference device 2c according to the fourth embodiment will be described. FIG. 17 is a flowchart for explaining the operation of the inference device 2c according to the fourth embodiment. The operation of the inference device 2c shown in FIG. 17 corresponds to the operation of step ST2c of the operation of the information processing system 100c described using the flowchart of FIG. For example, when the power supply of the inference device 2c is turned on, the inference device 2c repeats the operation shown in the flowchart of FIG. 17 until the power supply of the inference device 2c is turned off. Among the operations of the inference device 2c shown in Figure 17, the specific operations of step ST21c-1 are the same as the specific operations of step ST21 by the inference device 2 according to embodiment 1, which were explained using the flowchart of Figure 4 in embodiment 1, and therefore, redundant explanations will be omitted. Furthermore, among the operations of the inference device 2c shown in Figure 17, the specific operations of step ST21c-2 are similar to the specific operations of step ST21a-2 by the inference device 2a according to embodiment 2, which were explained using the flowchart of Figure 9 in embodiment 2, and therefore redundant explanations will be omitted. Furthermore, among the operations of the inference device 2c shown in Figure 17, the specific operations of step ST21c-3 are similar to the specific operations of step ST21b-2 by the inference device 2b according to embodiment 3, which were explained using the flowchart of Figure 13 in embodiment 3, and therefore redundant explanations will be omitted.

[0237] The inference unit 22c infers the occupant's preferences based on the operation information acquired by the operation information acquisition unit 21 in step ST21c-1, the attribute information acquired by the attribute information acquisition unit 28 in step ST21c-2, and the vehicle-related information acquired by the vehicle-related information acquisition unit 29 in step ST21c-3 (step ST22c). The inference unit 22c stores the preference inference result in the inference result storage unit 24c. Furthermore, the inference unit 22c outputs preference information indicating the inferred preference to the learning data acquisition unit 26c.

[0238] The information providing unit 25c refers to the inference result storage unit 24c, and outputs the preference inference result to the server 3 or a device mounted on the vehicle (specifically, the information processing device 1c) (step ST23c).

[0239] The learning data acquisition unit 26c acquires, as learning data (step ST24c), the operation information acquired by the operation information acquisition unit 21 in step ST21c-1, the attribute information acquired by the attribute information acquisition unit 28 in step ST21c-2, the vehicle-related information acquired by the vehicle-related information acquisition unit 29 in step ST21c-3, and the preference information indicating the preferences inferred by the inference unit 22c in step ST22c. The learning data acquisition unit 26c outputs the acquired learning data to the learning unit 27c.

[0240] The learning unit 27c learns the fourth machine learning model stored in the model storage unit 23c based on the learning data acquired by the learning data acquisition unit 26c in step ST24c (step ST25c). The learning unit 27c updates the fourth machine learning model stored in the model storage unit 23c to the learned fourth machine learning model.

[0241] 17, the processing is performed in the order of step ST23c to step ST25c, but this is merely an example. The processing of step ST23c and the processing of steps ST24c to ST25c may be performed in parallel. Furthermore, if the inference device 2c does not include the learning data acquisition unit 26c and the learning unit 27c, the operations of the inference device 2c can omit the processing of steps ST24c to ST25c.

[0242] In this way, in the information processing system 100c, the inference device 2c acquires, from the information processing device 1c, action information indicating the actions performed by the vehicle occupant, attribute information indicating the attributes of the vehicle occupant, and vehicle-related information, and infers the occupant's preferences based on the acquired action information, attribute information, and vehicle-related information. Even in this way, the inference device 2c can prevent erroneous inference of the preferences of the vehicle occupant when inferring the preferences of the vehicle occupant. Furthermore, since the inference device 2c infers the preferences of the occupant by taking into account the attribute information and the vehicle-related information in addition to the action information, it can infer preferences in more detail than inferring the preferences of the occupant from the action information alone.

[0243] Furthermore, the inference device 2c infers the preferences of the vehicle occupants using captured images of the vehicle occupants, and outputs a preference inference result based on the inferred preferences to the vehicle (specifically, the information processing device 1c) or the server 3. Therefore, the inference device 2c can provide services based on the inferred preferences to the vehicle occupants or various companies. Furthermore, because the inference device 2c infers the preferences of the occupants using captured images obtained by a DMS generally installed in the vehicle, it can effectively utilize information obtained in the vehicle.

[0244] The hardware configuration of the information processing device 1c according to the fourth embodiment is the same as the hardware configuration of the information processing device 1 described in the first embodiment with reference to FIGS. 5A and 5, and therefore is not shown in the drawings. In the fourth embodiment, the functions of the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 are realized by a processing circuit 1001. That is, the information processing device 1c includes the processing circuit 1001 for detecting the actions and attributes of the occupant based on the captured image acquired from the imaging device 4, collecting vehicle-related information, and controlling the output of the action information, attribute information, and vehicle-related information to the inference device 2c. The processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the image acquisition unit 11, the motion detection unit 12, the motion information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19. That is, the information processing device 1c includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of step ST11c-1 and step ST11c-2 to step ST15c in Fig. 16 described above. In addition, the program stored in memory 1005 can also be said to cause the computer to execute the processing procedures or methods of the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19. The information processing device 1c includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the inference device 2a, the imaging device 4, or the output device 5.

[0245] The hardware configuration of the inference device 2c according to the fourth embodiment is similar to the hardware configuration of the inference device 2 described in the first embodiment using FIGS. 5A and 5, and therefore is not shown in the drawings. In the fourth embodiment, the functions of the motion information acquisition unit 21, the inference unit 22c, the information provision unit 25c, the learning data acquisition unit 26c, the learning unit 27c, the attribute information acquisition unit 28, and the vehicle-related information acquisition unit 29 are realized by a processing circuit 1001. That is, the inference device 2c includes the processing circuit 1001 for controlling inference of the preferences of the occupant based on the motion information and attribute information acquired from the information processing device 1c. The processing circuit 1001 reads and executes the programs stored in the memory 1005, thereby executing the functions of the operation information acquisition unit 21, the inference unit 22c, the information provision unit 25c, the learning data acquisition unit 26c, the learning unit 27c, the attribute information acquisition unit 28, and the vehicle-related information acquisition unit 29. That is, the inference device 2c includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST21c-1, ST21c-2, and ST21c-3 to ST25c in FIG. 17 described above. It can also be said that the programs stored in the memory 1005 cause a computer to execute the processing procedures or methods of the operation information acquisition unit 21, the inference unit 22c, the information provision unit 25c, the learning data acquisition unit 26c, the learning unit 27c, the attribute information acquisition unit 28, and the vehicle-related information acquisition unit 29. The inference device 2c includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the information processing device 1c or the server 3.

[0246] In the above-described fourth embodiment, the inference device 2c is connected to one information processing device 1c, but this is merely an example. The inference device 2c can be connected to multiple information processing devices 1c. The inference device 2c acquires motion information, attribute information, and vehicle-related information from each of the information processing devices 1c installed in multiple different vehicles, and can infer the preferences of the occupants of each vehicle based on the acquired motion information, attribute information, and vehicle-related information. Note that the motion information, attribute information, and vehicle-related information output from the information processing device 1c are provided with information that can identify the vehicle.

[0247] Furthermore, in the above-described fourth embodiment, the model storage unit 23c of the inference device 2c may store multiple machine learning models according to the type of attribute or the type of vehicle-related information. The inference unit 22c selects, from the multiple machine learning models stored in the model storage unit 23c, a machine learning model according to the attribute information acquired by the attribute information acquisition unit 28 or the operation information acquired by the vehicle-related information acquisition unit 29, as the fourth machine learning model.

[0248] In addition, in the above-described fourth embodiment, the inference device 2c acquires vehicle-related information via the information processing device 1c, but this is merely an example. For example, the inference device 2c may acquire vehicle-related information directly from various devices provided in the vehicle. In this case, the information processing device 1c may be configured not to include the vehicle-related information collection unit 18 and the vehicle-related information output unit 19.

[0249] In the fourth embodiment, the inference device 2c may be mounted on a vehicle.

[0250] Furthermore, in the above-described fourth embodiment, the information processing device 1c is an in-vehicle device mounted on a vehicle, and the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 are provided in the in-vehicle device. Without being limited to this, for example, some of the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 may be mounted on the vehicle's on-board device, and the rest may be provided in the inference device 2c. Furthermore, for example, the image acquisition unit 11, the action detection unit 12, the action information output unit 13, the provided information acquisition unit 14, the provided information output unit 15, the attribute detection unit 16, the attribute information output unit 17, the vehicle-related information collection unit 18, and the vehicle-related information output unit 19 may all be provided in the inference device 2c.

[0251] Furthermore, in the above-described fourth embodiment, the model storage unit 23c of the inference device 2c may store a first machine learning model and a fourth machine learning model, and the inference unit 22c may be capable of inferring the occupant's preferences based on the first machine learning model and the motion information acquired by the motion information acquisition unit 21. Furthermore, the model storage unit 23c may further store a second machine learning model, and the inference unit 22c may be capable of inferring the occupant's preferences based on the motion information acquired by the motion information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the second machine learning model, or the model storage unit 23c may further store a third machine learning model, and the inference unit 22c may be capable of inferring the occupant's preferences based on the motion information acquired by the motion information acquisition unit 21, the vehicle-related information acquired by the vehicle-related information acquisition unit 29, and the third machine learning model.

[0252] As described above, according to the fourth embodiment, the inference device 2c is configured to include the action information acquisition unit 21 that acquires action information indicating actions performed by a vehicle occupant, the attribute information acquisition unit 28 that acquires attribute information indicating the attributes of the vehicle occupant, the vehicle-related information acquisition unit 29 that acquires vehicle-related information, and the inference unit 22c that infers the preferences of the occupant based on the action information acquired by the action information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. Therefore, when inferring the preferences of a vehicle occupant, the inference device 2c can prevent erroneous inference of the preferences of the occupant.

[0253] Furthermore, according to the fourth embodiment, the information processing system 100c includes an image acquisition unit 11 that acquires a captured image of a vehicle occupant, a motion detection unit 12 that detects a motion performed by the occupant based on the captured image acquired by the image acquisition unit 11, a motion information output unit 13 that outputs motion information indicating the motion of the occupant detected by the motion detection unit 12, an attribute detection unit 16 that detects an attribute of the occupant based on the captured image acquired by the image acquisition unit 11, and an attribute information output unit 17 that outputs attribute information indicating the attribute of the occupant detected by the attribute detection unit 16. The information processing system 100c is configured to include an information processing device 1c having the motion information output unit 13 and the attribute information output unit 28, an inference device 2c having a motion information acquisition unit 21 that acquires the motion information output by the motion information output unit 13, an attribute information acquisition unit 28 that acquires the attribute information output by the attribute information output unit 17, a vehicle-related information acquisition unit 29 that acquires vehicle-related information, and an inference unit 22c that infers the preferences of the vehicle occupant based on the motion information acquired by the motion information acquisition unit 21, the attribute information acquired by the attribute information acquisition unit 28, and the vehicle-related information acquired by the vehicle-related information acquisition unit 29. Therefore, when inferring the preferences of the vehicle occupant, the information processing system 100c can prevent erroneous inference of the preferences of the vehicle occupant.

[0254] In the above-described first to fourth embodiments, the information providing units 25, 25a, 25b, and 25c of the inference devices 2, 2a, 2b, and 2c output the advertisement output control information attached to the preference inference result to the output device 5 via the information processing devices 1, 1a, 1b, and 1c, but this is merely an example. The information providing units 25, 25a, 25b, and 25c of the inference devices 2, 2a, 2b, and 2c may directly output the preference inference result attached with the advertisement output control information to the output device 5. In this case, the information processing devices 1, 1a, 1b, and 1c are not required to include the provided information acquisition unit 14 and the provided information output unit 15.

[0255] Furthermore, the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. [Industrial Applicability]

[0256] The inference device of the present disclosure can prevent erroneous inference of the preferences of vehicle occupants. [Explanation of symbols]

[0257] 100, 100a, 100b, 100c Information processing system, 1, 1a, 1b, 1c Information processing device, 11 Image acquisition unit, 12 Action detection unit, 13 Action information output unit, 14 Provided information acquisition unit, 15 Provided information output unit, 16 Attribute detection unit, 17 Attribute information output unit, 18 Vehicle-related information collection unit, 19 Vehicle-related information output unit, 2, 2a, 2b, 2c Inference device, 21 Action information acquisition unit, 22, 22a, 22b, 22c Inference unit, 23, 23a, 23b, 23c Model memory unit, 24, 24a, 24b, 24c Inference result memory unit, 25, 25a, 25b, 25c Information provision unit, 26, 26a, 26b, 26c Learning data acquisition unit, 27, 27a, 27b, 27c Learning unit, 28 attribute information acquisition unit, 29 vehicle-related information acquisition unit, 3 server, 4 imaging device, 5 output device, 1001 processing circuit, 1002 input interface device, 1003 output interface device, 1004 processor, 1005 memory.

Claims

1. a motion information acquiring unit that acquires motion information indicating a motion performed by an occupant of the vehicle based on an image of the occupant captured by an imaging device mounted on the vehicle; an attribute information acquisition unit that acquires attribute information indicating any one of a plurality of attributes of the occupant based on the captured image; a model storage unit that stores a plurality of machine learning models corresponding to the plurality of types of attributes, the plurality of machine learning models receive the motion information and the attribute information as input and output preference information indicating the preferences of the occupant; an inference unit that selects, as a second machine learning model, a machine learning model according to the attribute information acquired by the attribute information acquisition unit from the plurality of machine learning models stored in the model storage unit, The vehicle control system further includes an inference unit that infers the preference of the occupant based on the motion information acquired by the motion information acquisition unit, the attribute information acquired by the attribute information acquisition unit, and the second machine learning model. Reasoning device.

2. an information providing unit that outputs a preference inference result in which the action information acquired by the action information acquiring unit, the attribute information acquired by the attribute information acquiring unit, and the preference information indicating the preference of the occupant inferred by the inference unit are associated with each other; 2. The inference device according to claim 1, comprising:

3. A motion information acquisition unit that acquires motion information indicating motions performed by an occupant of the vehicle based on an image of the occupant captured by an imaging device mounted on the vehicle; an attribute information acquisition unit that acquires attribute information indicating any one of a plurality of attributes of the occupant based on the captured image; a vehicle-related information acquisition unit that acquires vehicle-related information related to the vehicle; a model storage unit that stores a plurality of machine learning models corresponding to the types of the plurality of attributes or the types of the vehicle-related information, the plurality of machine learning models receive the motion information, the attribute information, and the vehicle-related information as input and output preference information indicating the preferences of the occupant; an inference unit that selects, from the plurality of machine learning models stored in the model storage unit, a machine learning model according to the attribute information acquired by the attribute information acquisition unit or the vehicle-related information acquired by the vehicle-related information acquisition unit as a fourth machine learning model, The vehicle information acquisition device further includes an inference unit that infers the preference of the occupant based on the motion information acquired by the motion information acquisition unit, the attribute information acquired by the attribute information acquisition unit, the vehicle-related information acquired by the vehicle-related information acquisition unit, and the fourth machine learning model. Reasoning device.

4. an information providing unit that outputs a preference inference result in which the action information acquired by the action information acquiring unit, the attribute information acquired by the attribute information acquiring unit, the vehicle-related information acquired by the vehicle-related information acquiring unit, and the preference information indicating the preference of the occupant inferred by the inference unit are associated with each other; 4. The inference device according to claim 3, comprising:

5. An image acquisition unit that acquires an image of an occupant of the vehicle using an imaging device mounted on the vehicle; a motion detection unit that detects a motion performed by the occupant based on the captured image acquired by the image acquisition unit; a motion information output unit that outputs motion information indicating the motion of the occupant detected by the motion detection unit; an attribute detection unit that detects one of a plurality of attributes of the occupant based on the captured image acquired by the image acquisition unit; an attribute information output unit that outputs attribute information indicating the attribute detected by the attribute detection unit; an information processing device having a model storage unit that stores a plurality of machine learning models corresponding to the plurality of types of attributes, the plurality of machine learning models receive the motion information and the attribute information as input and output preference information indicating the preferences of the occupant; moreover, a motion information acquisition unit that acquires the motion information output by the motion information output unit; an attribute information acquisition unit that acquires the attribute information output by the attribute information output unit; an inference unit that selects, as a second machine learning model, a machine learning model according to the attribute information acquired by the attribute information acquisition unit from the plurality of machine learning models stored in the model storage unit, an inference unit that infers the preference of the occupant based on the motion information acquired by the motion information acquisition unit, the attribute information acquired by the attribute information acquisition unit, and the second machine learning model; An inference device having An information processing system comprising:

6. An image acquisition unit that acquires an image of an occupant of the vehicle using an imaging device mounted on the vehicle; a motion detection unit that detects a motion performed by the occupant based on the captured image acquired by the image acquisition unit; a motion information output unit that outputs motion information indicating the motion of the occupant detected by the motion detection unit; an attribute detection unit that detects one of a plurality of attributes of the occupant based on the captured image acquired by the image acquisition unit; an attribute information output unit that outputs attribute information indicating the attribute detected by the attribute detection unit; an information processing device having a model storage unit that stores a plurality of machine learning models according to the types of the plurality of attributes or the types of vehicle-related information related to the vehicle; the plurality of machine learning models receive the motion information, the attribute information, and the vehicle-related information as input and output preference information indicating the preferences of the occupant; moreover, a motion information acquisition unit that acquires the motion information output by the motion information output unit; an attribute information acquisition unit that acquires the attribute information output by the attribute information output unit; a vehicle-related information acquisition unit that acquires the vehicle-related information related to the vehicle; an inference unit that selects, from the plurality of machine learning models stored in the model storage unit, a machine learning model according to the attribute information acquired by the attribute information acquisition unit or the vehicle-related information acquired by the vehicle-related information acquisition unit as a fourth machine learning model, an inference unit that infers the preferences of the occupant based on the action information acquired by the action information acquisition unit, the attribute information acquired by the attribute information acquisition unit, the vehicle-related information acquired by the vehicle-related information acquisition unit, and the fourth machine learning model. An information processing system comprising:

7. a step in which a motion information acquisition unit acquires motion information indicating a motion performed by an occupant of the vehicle based on a captured image of the occupant captured by an imaging device mounted on the vehicle; an attribute information acquisition unit acquiring attribute information indicating any one of a plurality of attributes of the occupant; an inference unit selecting, as a second machine learning model, a machine learning model corresponding to the attribute information acquired by the attribute information acquisition unit from among a plurality of machine learning models corresponding to each of the plurality of attribute types; a step in which the inference unit infers a preference of the occupant based on the motion information acquired by the motion information acquisition unit, the attribute information acquired by the attribute information acquisition unit, and the second machine learning model; and a method of inference comprising:

8. A step in which an action information acquisition unit acquires action information indicating actions performed by an occupant of the vehicle based on an image of the occupant captured by an imaging device mounted on the vehicle; an attribute information acquisition unit acquiring attribute information indicating any one of a plurality of attributes of the occupant; a vehicle-related information acquisition unit acquiring vehicle-related information related to the vehicle; an inference unit selecting, as a fourth machine learning model, a machine learning model corresponding to the attribute information acquired by the attribute information acquisition unit or the vehicle-related information acquired by the vehicle-related information acquisition unit from among a plurality of machine learning models corresponding to each of the plurality of types of attributes or the type of vehicle-related information; a step in which the inference unit infers a preference of the occupant based on the action information acquired by the action information acquisition unit, the attribute information acquired by the attribute information acquisition unit, the vehicle-related information acquired by the vehicle-related information acquisition unit, and the fourth machine learning model. and a method of inference comprising:

Images