Control system, control method and storage medium

Abstract

[Problem] To provide a control system, a control method and a storage medium, by which a user's surrounding environment can be changed using a mobile object according to the user's emotions. [Solution] A control system is provided, comprising: an estimation unit that estimates the user's emotions; and a mobile object control unit that controls a mobile object according to the estimated emotions to change the user's surrounding environment while tracking the user.

Description

[0001] This invention application is a divisional application of invention patent application No. 201580059112.7, whose application date is August 4, 2015 and entered the Chinese national phase on April 28, 2017 and whose invention name is “Control System, Control Method and Storage Medium”. Technical Field

[0002] The present disclosure relates to a control system, a control method and a storage medium. Background Art

[0003] Recently, technology for automatically controlling a robot to improve a user's life into a comfortable life has been proposed.

[0004] For example, Patent Document 1 below discloses a robot that estimates a person's emotional state based on the person's expression and delivers information to a family member who does not live with the person. Patent Document 2 below discloses a technology for estimating a user's emotions based on captured images and biometric information and changing the robot's behavior or changing the subject based on the emotion estimation result. Patent Document 3 below discloses a communication robot that judges a user's temperament based on the strength of the user's touch on the robot, etc., and determines the emotion based on the touch duration.

[0005] In addition, the following Patent Documents 4 and 5 propose devices for adjusting the angle of a sun umbrella by a motor according to the position of the sun.

[0006] Reference list

[0007] Patent Literature

[0008] Patent Document 1: JP 2013-246517A

[0009] Patent Document 2: JP 2012-59107A

[0010] Patent Document 3: JP 2008-278981A

[0011] Patent Document 4: WO 2013-020562

[0012] Patent Document 5: US 2006-0124122 Summary of the invention

[0013] Technical issues

[0014] However, the above patent documents do not consider the technology for tracking a mobile user and changing the user's surroundings. The technology described in patent documents 4 and 5 is based on the assumption that the person remains in the same place to control fixed equipment such as beach umbrellas and cannot change the surroundings of the moving person.

[0015] For example, if it suddenly starts to rain when the user is out, the user has a feeling of wanting to avoid the rain (or an uncomfortable feeling of getting wet due to the rain). The user's surroundings can be locally changed by controlling the moving object according to the user's feeling so that the user will not get wet by the rain, thereby improving the user's life.

[0016] Therefore, the present disclosure proposes a control system, a control method, and a storage medium, by which a mobile object can change the user's surrounding environment according to the user's emotions.

[0017] Solution to the problem

[0018] According to the present disclosure, there is provided a control system including: an estimation unit that estimates an emotion of a user; and a mobile object controller that controls a mobile object according to the estimated emotion to change a surrounding environment of the user while tracking the user.

[0019] According to the present disclosure, there is provided a control method including: estimating a user's emotion; and controlling a moving object to change a surrounding environment of the user while tracking the user according to the estimated emotion.

[0020] According to the present disclosure, there is provided a storage medium having a program stored therein, which, when executed by a computer, causes the computer to function as: an estimation unit that estimates a user's emotions; and a mobile object controller that controls a mobile object according to the estimated emotions to change the user's surroundings while tracking the user.

[0021] Advantageous Effects of the Invention

[0022] According to the present disclosure described above, a moving object can change the user's surrounding environment according to the user's emotions.

[0023] Note that the above effects are not necessarily limitative. With or in place of the above effects, any of the effects described in this specification or other effects that can be obtained according to this specification can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] [ Figure 1 ] Figure 1 It is an explanatory diagram of an overview of an environment conditioning system according to an embodiment of the present disclosure.

[0025] [ Figure 2 ] Figure 2 is an illustration of an example of tracking a moving object of a user.

[0026] [ Figure 3 ] Figure 3 is a block diagram showing an example of a configuration of a mobile object according to the present embodiment.

[0027] [ Figure 4 ] Figure 4 is a block diagram showing an example of the configuration of an environment adjustment server according to the present embodiment.

[0028] [ Figure 5 ] Figure 5 1 is an explanatory diagram of an example of an environmental heat map according to the present embodiment.

[0029] [ Figure 6 ] Figure 6 It is an explanatory diagram of an example of a sentiment heat map and a comprehensive heat map according to the present embodiment.

[0030] [ Figure 7 ] Figure 7 : is a block diagram showing an example of the configuration of the emotion server according to the present embodiment.

[0031] [ Figure 8 ] Figure 8 : is a flowchart showing the environment adjustment operation process of the environment adjustment system according to the present embodiment.

[0032] [ Fig. 9 ] Fig. 9 This is an explanatory diagram schematically illustrating a situation in which rain and sunlight are blocked by a mobile object according to the present embodiment.

[0033] [ Fig.10 ] Fig.10 This is an explanatory diagram of the expansion of the occlusion range of the moving object according to the present embodiment.

[0034] [ Fig.11 ] Fig.11 This is an explanatory diagram of the expansion of the occlusion range by a plurality of moving objects according to the present embodiment.

[0035] [ Fig.12 ] Fig.12 1 is a flowchart showing an operation process of changing an environment for shielding from rain according to the present embodiment.

[0036] [ Fig.13 ] Fig.13 is an explanatory diagram of the calculation of the relative position of a mobile object with respect to a target user during rain occlusion.

[0037] [ Fig.14 ] Fig.14 Detailed description is a flowchart showing an operation process of changing an environment for blocking sunlight according to the present embodiment.

[0038] [ Fig.15 ] Fig.15 is an explanatory diagram of the calculation of the relative position of a moving object with respect to a target user during sun shading.

[0039] [ Fig.16 ] Fig.16 : is a flowchart showing a process of changing an occlusion range based on an emotion value according to the present embodiment.

[0040] [ Fig.17 ] Fig.17 This is an explanatory diagram of lighting control by a moving object according to the present embodiment.

[0041] [ Fig.18 ] Fig.18 Detailed Description of the Invention This is a flowchart showing a lighting control process performed by a mobile object according to the present embodiment.

[0042] [ Fig.19 ] Fig.19 This is a diagram for explaining the calculation of the lighting direction (angle) and position of a moving object during lighting.

[0043] [ Fig. 20 ] Fig. 20 It is an explanatory diagram of the environmental change by the air curtain according to the present embodiment.

[0044] [ Fig.21 ] Fig.21 : is a flowchart showing an operation process of performing an environment change by an air curtain according to the present embodiment.

[0045] [ Fig. 22 ] Fig. 22 It is an explanatory diagram of the environmental change by blocking smoke according to the present embodiment.

[0046] [ Fig.23 ] Fig.23 : is a flowchart showing an operation process of performing an environment change by blocking smoke according to the present embodiment.

[0047] [ Fig.24 ] Fig.24 This is an explanatory diagram of the environment change by music playback according to the present embodiment.

[0048] [ Fig.25 ] Fig.25 1 is a flowchart showing an operation process of performing an environment change by music playback according to the present embodiment. DETAILED DESCRIPTION

[0049] Hereinafter, one or more preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, structural elements having substantially the same function and structure are denoted by the same reference numerals, and repeated description of these structural elements is omitted.

[0050] Additionally, the description will be given in the following order.

[0051] 1. Overview of the Environmental Conditioning System According to an Embodiment of the Present Disclosure

[0052] 2. Basic configuration and operation

[0053] 2-1. Configuration of moving objects

[0054] 2-2. Configuration of the environment adjustment server

[0055] 2-3. Configuration of the emotion server

[0056] 2-4. Environmental adjustment operation

[0057] 3. Example of environmental change

[0058] 3-1. Shelter from rain and sun

[0059] 3-2.Lighting

[0060] 3-3. Output of wind, water, etc.

[0061] 3-4. Music playback

[0062] 4. Conclusion

[0063] <<1. Overview of Environmental Conditioning System According to Embodiment of the Present Disclosure>>

[0064] First, refer to Figure 1 The overview of the environment conditioning system 1 according to an embodiment of the present disclosure is described. Figure 1 As shown, the environment adjustment system 1 according to the present embodiment includes a mobile object 6 that tracks a user to change the user's surrounding environment and an environment adjustment server 7 that controls the mobile object 6. The environment adjustment server 7 communicates with the mobile object 6 via an access point or the like, and performs movement control of the mobile object 6 or the like. The mobile object 6 may be a plurality of mobile objects 6a to 6c, and may be moved using a base station or the like installed outdoors as a base point, such as Figure 1 shown.

[0065] In addition, the environment adjustment server 7 is connected to the emotion server 2 and the heat map server 75 via the network 3, and acquires information required for movement control of the mobile object 6. The environment adjustment server 7 can provide a pleasant environment for the user by making the mobile object 6 track the user and locally changing the surrounding environment according to the user's emotion.

[0066] The mobile object 6 may be Figure 1 The small drone shown can be Figure 2 The mobile object 6 is shown flying while tracking the mobile user. Figure 1 The flying object shown may be, for example, Figure 2A mobile object 6' of robot type is shown moving on the ground while tracking a user, or a mobile object in water (not shown). Figure 1 and Figure 2 The illustrated forms of the moving objects 6 and 6 ′ are examples, and the present embodiment is not limited thereto.

[0067] In addition, if Figure 2 As shown, the user can wear a band-type communication terminal (hereinafter referred to as a smart band) 8 for acquiring biological information such as pulse and temperature. The smart band 8 transmits the acquired biological information to the mobile object 6. The biological information acquired from the user can be used as information for estimating the user's emotions.

[0068] The overview of the environment conditioning system 1 according to the present disclosure has been described. Figures 3 to 8 The specific configuration and basic operation of the devices included in the environmental conditioning system 1 are described.

[0069] <<2. Basic Configuration and Operation>>

[0070] <2-1. Configuration of moving objects>

[0071] Figure 3 is a block diagram showing an example of the configuration of the mobile object 6 according to the present embodiment. Figure 3 As shown, the mobile object 6 includes a communication unit 61 , a target object sensor 62 , a mobile object controller 63 and an environment sensor 64 . Figure 3 The illustrated configuration is an example and the mobile object 6 may further include, for example, a transformation controller 65 and an output unit 66 .

[0072] The communication unit 61 transmits / receives data to / from the environment adjustment server 7. For example, the communication unit 61 transmits sensor information acquired by the target object sensor 62 and the environment sensor 64 to the environment adjustment server 7, and receives control information including instructions of movement control and transformation control from the environment adjustment server 7.

[0073] The target object sensor 62 is a sensing unit that obtains information about the tracking target object. Here, the tracking target object corresponds to, for example, a user moving outdoors. Specifically, the target object sensor 62 obtains information about the user through, for example, a camera 62a, an infrared camera 62b, a radio signal receiver 62c, or a microphone array 62d.

[0074] The camera 62a is a general camera using visible light, and shoots a tracking target object to obtain a captured image. In addition, the infrared camera 62b is an example of a camera using non-visible light, and shoots a tracking target object to obtain an infrared image. The radio signal receiver 62c receives, for example, biometric information of the user transmitted from the smart band 8 worn by the user. In addition, the radio signal receiver 62c also detects beacons for specifying the location of the user, etc. The microphone array 62d acquires ambient sounds such as the footsteps of the user.

[0075] The specific examples of the target object sensor 62 described above are examples, and the target object sensor 62 may also include an ultrasonic sensor, a distance measurement sensor, and the like.

[0076] The mobile object controller 63 has the function of controlling the movement of the mobile object 6. Specifically, the mobile object controller 63 includes a power drive unit 63a and a posture controller 63b. Figure 3 As shown. The power drive unit 63a is implemented by, for example, a propeller, a wheel, or a walking leg, and operates according to the control of the mobile object controller 63 to track the target user. In addition, the posture controller 63b senses the posture of the mobile object 6 through a gyro sensor, etc., and controls the power drive unit 63a to adjust the inclination and height of the mobile object 6. The mobile object controller 63 according to the present disclosure can locally change the user's surrounding environment by controlling the flight path and height of the mobile object 6, so that the mobile object 6 becomes, for example, a shielding object that shields the user from rain or sunlight.

[0077] The environment sensor 64 is a sensing unit that acquires information about the surrounding environment. Specifically, the environment sensor 64 acquires environment information through, for example, a latitude and longitude positioning unit 64a or an altitude sensor 64b. Specific examples of the environment sensor 64 are not limited to Figure 3 The environmental sensors 64 are shown as examples and may also include temperature sensors, humidity sensors, etc. as needed.

[0078] The transformation controller 65 performs control to transform the shape of the mobile object 6 to expand the shielding range when shielding against rain or sunlight. Fig.10 and Fig.11 The description is as follows: extending the occlusion range by moving the object 6 .

[0079] The output unit 66 provides various outputs for changing the user's surrounding environment. For example, the output unit 66 is implemented by a lighting unit (lamp), a wind (air) output unit, a water output unit, a laser output unit, etc. In addition, the output unit 66 performs output control according to the control of the environment adjustment server 7.

[0080] The aforementioned mobile object 6 is equipped with a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and a nonvolatile memory to control each component of the mobile object 6 .

[0081] <2-2. Configuration of the environment adjustment server>

[0082] Figure 4 1 is a block diagram showing an example of the configuration of the environment adjustment server 7 according to the present embodiment. Figure 4 As shown, the environment adjustment server 7 includes a communication unit 71 , a controller 70 , and a user database (DB) 72 .

[0083] The communication unit 71 transmits / receives data to / from an external device. Specifically, for example, the communication unit 71 receives target object information and environmental information from the mobile object 6, and transmits control information for controlling the movement of the mobile object 6 to the mobile object 6. In addition, the communication unit 71 acquires predetermined data from the emotion server 2 and the heat map server 75 via the network 3.

[0084] The controller 70 controls each component of the environment adjustment server 7. Specifically, the controller 70 controls the mobile object 6 to track the target user to change the surrounding environment of the target user according to the user's mood. In addition, the controller 70 is implemented by a microcomputer including a CPU, ROM, RAM, and non-volatile memory.

[0085] Here, the controller 70 functions as a user setting and management unit 70a, a target object detection unit 70b, an emotion estimation unit 70c, an environment information acquisition unit 70d, a heat map generation / acquisition unit 70e, an emotion value calculation unit 70f, and a moving object controller 70g. Figure 4 shown.

[0086] The user setting and management unit 70a manages the registration, change and deletion of the information of the user using the present system. For example, the user's information includes the user's name, identification number, facial image, age, gender, hobbies, tastes, home address, workplace, behavior patterns, etc. The user's hobbies and tastes can be input by the user or can be automatically set by analyzing the user's life log, content uploaded to social media, text content of mail and messaging tools, conversations with friends and family, purchase history, schedule, etc. In addition, the user's behavior pattern can be automatically set by life log (specifically, past behavior history), purchase history, etc. The user setting and management unit 70a stores the user's information in the user DB 72 and manages the information.

[0087] The target object detection unit 70b detects a person located near the mobile object 6, an object carried by the person, and the like based on the target object sensor information transmitted from the mobile object 6. Specifically, the target object detection unit 70b may detect the target user, for example, by analyzing a captured image transmitted from the mobile object 6 and comparing the captured image with a facial image of a user registered in the user DB 72. Otherwise, the target object detection unit 70b may detect the target user based on user identification information received by the mobile object 6 from a radio communication device such as a smart band 8 worn by the user. When the target user is detected, the target object detection unit 70b outputs the detection result to the emotion estimation unit 70c. The above-mentioned target user detection method of the target object detection unit 70b is an example, and the target user detection method according to the present embodiment is not limited thereto.

[0088] The emotion estimation unit 70c estimates the emotion of the target user detected by the target object detection unit 70b. Specifically, the emotion estimation unit 70c estimates the emotion of the target user based on, for example, biological information (pulse, temperature, sweating amount, brain waves, etc.). Figure 2 The smart belt 8 shown acquires it, sends it from the smart belt 8 to the mobile object 6, and then sends it from the mobile object 6 to the environment adjustment server 7.

[0089] In addition, the emotion estimation unit 70c can estimate the emotion of the target user based on the target user's traits (gender, age, height, weight, personality, occupation, etc.), hobbies and tastes, and the surrounding environment. For example, in the case of a young woman, the emotion of wanting to avoid exposure to direct sunlight during the day (worrying about sunburn) and the emotion of being afraid of dark roads at night (night roads, roads without passers-by, etc.) are estimated. In addition, in the case of an office worker commuting or working, the emotion of not wanting to get wet in the rain due to the important documents he or she carries is estimated. Information on such traits, hobbies, and tastes of the target user can be obtained from the user DB 72. The emotion estimation unit 70c outputs the estimation result to the heat map generation / acquisition unit 70e.

[0090] The environmental information acquisition unit 70d acquires information about the surrounding environment from the mobile object 6 through the communication unit 71. Specifically, the environmental information acquisition unit 70d acquires data (latitude, longitude, altitude, etc.) sensed by the environmental sensor 64 of the mobile object 6 as environmental information. In addition, the environmental information acquisition unit 70d can acquire surrounding geographical information, building information, etc. as environmental information from a predetermined server according to the position (latitude and longitude information) of the mobile object 6. The environmental information acquisition unit 70d outputs the acquired environmental information to the heat map generation / acquisition unit 70e.

[0091] The heat map generation / acquisition unit 70e generates an emotion heat map that maps emotions geographically based on the estimation results of the emotion estimation unit 70c and the environmental information output from the environmental information acquisition unit 70d. More specifically, the heat map generation / acquisition unit 70e generates an emotion heat map that indicates whether there are many users with certain emotions in certain locations based on the estimation results of the emotions of multiple users. In addition, in addition to the estimation results of the emotion estimation unit 70c, the heat map generation / acquisition unit 70e can also collect emotions associated with locations based on the results obtained by analyzing the content written on social media and the content of the email. Specifically, the unpleasant feeling caused by getting wet in the rain, the frustration caused by wet documents, the frustration of not being able to open your umbrella when you come back from shopping because your hands are full of things, the discomfort caused by having to take a longer path on rainy days compared to sunny days, etc. are associated with the location and mapped to the location.

[0092] In addition, the heat map generation / acquisition unit 70e generates an environmental heat map around the current position of the target user based on the environmental information (latitude, longitude, altitude, geographic information, building information, etc.) output from the environmental information acquisition unit 70d. More specifically, the heat map generation / acquisition unit 70e generates an environmental heat map "rain" indicating a position that is easy to get wet in the rain, an environmental heat map "shadow" indicating a position corresponding to the shadow of a building, and the like. For example, because a position that is easy to get wet in the rain and a position that is not easy to get wet are identified based on the geographical terrain (geographic information) and artificial buildings such as arcades, underground shopping malls, and eaves, a position that is easy to get wet in the rain can be determined by simulating geographical features and buildings, or a position that is easy to get wet in the rain can be determined based on the rainfall actually measured by the mobile object 6, etc. In addition, information about a position that is easy to get wet in the rain can be manually input. Otherwise, the heat map generation / acquisition unit 70e can determine locations that are prone to getting wet in the rain by collecting records of the user actively calling the mobile object 6 to the location where he or she is located to block the rain or collecting records of the locations that actually get wet in the rain via various sensors such as a water drop sensor attached to the user's body.

[0093] The heat map generated in this manner may be stored in the heat map server 75 on the network or in a storage unit (not shown) of the environment adjustment server 7. Furthermore, the heat map may be updated periodically.

[0094] In addition, the heat map generation / acquisition unit 70e can generate a comprehensive heat map of the emotion heat map and the environment heat map.

[0095] Here, refer to Figure 5 and Figure 6The various heat maps generated by the heat map generation / acquisition unit 70e are described. The heat map generation / acquisition unit 70e maps a predetermined environment degree (for example, an area that is easily wet by rain) to a predetermined environment degree (for example, an area that is easily wet by rain). Figure 5 The map 40 of the target area shown in the left part of FIG. Figure 5 The environment heat map 41 is shown in the right part of FIG. When the environment heat map 41 is “rain”, the darker colored locations are more likely to get wet in the rain.

[0096] In addition, the heat map generation / acquisition unit 70e maps a predetermined emotion degree (e.g., unpleasant emotion due to getting wet in the rain) to Figure 5 The map 40 of the target area shown in the left part of FIG. Figure 6 4. When the emotion heat map 42 is "unpleasant", the darker colored locations represent locations where the user is more unhappy or frustrated because he or she gets wet in the rain.

[0097] In addition, the heat map generation / acquisition unit 70e integrates the environment heat map 41 and the emotion heat map 42 to generate Figure 6 The comprehensive heat map shown in the right part of FIG43. In this case, the environment heat map or the emotion heat map can be regarded as important by applying a specific weight to one of the heat maps.

[0098] The comprehensive heat map generated in this manner becomes information for determining “an area with a high value of protecting the user from rain” in a comprehensive manner.

[0099] The heat map generation / acquisition unit 70e outputs the generated integrated heat map to the mobile object controller 70g.

[0100] As described above, it is desirable to update the environment heat map and the mood heat map at fixed intervals because, for example, in the case of constructing a new building, the position that easily gets wet in the rain and the position of the shadow corresponding to the building change. In addition, it is also desirable to update the mood heat map at fixed intervals because it is considered that new emotions are added every day and the emotions may change according to the season, date, and time. For example, getting wet in the rain in the summer may not cause an unpleasant feeling, and cold rain in the winter may cause a very unpleasant feeling.

[0101] In addition, the heat map update frequency can have different periods depending on the heat map type. For example, in the case of a UV intensity heat map, whether a location is exposed to the sun varies depending on the time of day and season. Therefore, it is desirable to update the UV intensity map at short intervals. In addition, since the UV intensity also depends on the weather (cloudy, sunny, etc.), it is considered to generate separate UV intensity heat maps for cloudy and sunny days.

[0102] In addition, the heat map generation / acquisition unit 70e can predict and generate future environmental heat maps using repeated seasonal elements and information obtained from weather forecasts. In addition, the heat map generation / acquisition unit 70e can predict and generate future environmental heat maps based on underground passages or arcade construction plans. In addition, the environmental heat map generated by prediction can be updated in real time based on actual measurement values, or the accuracy of the prediction can be improved by feedback later.

[0103] Furthermore, with regard to emotion heat maps, for example, if periodicities such as there being many people in the office area on weekdays and fewer people on holidays can be extracted from past emotion heat map data, it can be predicted that there will be fewer people who become frustrated in the rain, or even that they can happily pass through a narrow area shielded from the rain without getting wet (or without being exposed to UV rays) because there is no crowding, etc.

[0104] The emotion value calculation unit 70f calculates the emotion value (i.e., relative emotion value) for the belongings of the target user and the person accompanying the target user detected by the target object detection unit 70b. Specifically, the emotion value calculation unit 70f obtains the interaction evaluation related to the object ID of the target user from the emotion server 2, and calculates the relative emotion value of the target user based on the interaction evaluation. Here, the interaction evaluation is an estimate of the interaction (i.e., behavior) performed by the target user relative to another person or object, and can be used when calculating the feeling of the target user relative to other people or objects (referred to as emotion value in the specification).

[0105] For example, the estimation of the interaction of the target user who frequently maintains the watch he or she owns is stored in the emotion server 2, and the estimation of the interaction of the watch is saved in the case of special design for it, and the emotion value calculation unit 70f obtains such interaction evaluation of the target user and calculates the emotion value. Although the estimation value calculation method is not particularly limited, for example, the interaction with the specific object (related object) of the target user can be classified by the interaction type, and the estimation of all interactions about the specific object can be averaged by using a weighting function according to the interaction type. The object ID of the target user can be obtained from the user DB 72.

[0106] The mobile object controller 70h determines the area where the environment needs to be changed on the route where the target user moves based on the integrated heat map output from the heat map generation / acquisition unit 70e, and controls the mobile object 6 to change the user's surrounding environment. The determination of the area where the environment needs to be changed is performed based on whether the value of the integrated heat map is higher than a predetermined threshold. In this way, for example, when the target user passes by a position where the user is likely to get wet in the rain, the surrounding environment of the target user is locally changed by causing the mobile object 6 to track the target user to block the rain by utilizing the integrated heat map obtained by integrating the emotional heat map and the environmental heat map, so that the target user can be in a pleasant state.

[0107] Although the environment adjustment is performed by the mobile object 6 so that the target user is not in an unpleasant state during the movement by using the integrated heat map obtained by integrating the emotion heat map reflecting the emotion estimation results of multiple users in this embodiment with the environment heat map generated based on the environment information such as geographical features, the present embodiment is not limited thereto. For example, when the emotion of the user becomes unpleasant, the mobile object controller 70g can control the mobile object 6 to locally change the surrounding environment according to the estimation result of the emotion estimation unit 70c so that the user has a pleasant feeling.

[0108] In addition, when the surrounding environment of the target user is changed by the mobile object 6, the mobile object controller 70h can perform control to expand the shielding range by the mobile object 6 or shield the target user's belongings or people with the target user in priority to the target user based on the emotional value (estimated degree) of the target user's belongings or people with the target user obtained by the emotional value calculation unit 70f.

[0109] The configuration of the environment adjustment server 7 according to the present embodiment has been described in detail. Figure 7 The configuration of the emotion server 2 according to the present embodiment is described.

[0110] <2-3. Configuration of the emotion server>

[0111] Figure 7 is a block diagram showing an example of the configuration of the emotion server 2 according to the present embodiment. Figure 7 As shown, the emotion server 2 includes a communication unit 21 , a controller 20 , an object DB 22 , and an emotion information DB 24 .

[0112] The communication unit 21 is connected to the environment adjustment server 7 through the network 3, and returns the interaction evaluation associated with the object ID of the target user specified by the environment adjustment server 7. In addition, the communication unit 21 receives interaction information from a sensing device (not shown) attached to / installed on each object (person or object).

[0113] The controller 20 controls each component of the emotion server 2. In addition, the controller 20 is implemented by a microcomputer including a CPU, a ROM, a RAM, and a non-volatile memory. In addition, the controller 20 according to the present embodiment functions as an interactive storage controller 20a, an estimation unit 20b, an object management unit 20c, and a related object search unit 20d.

[0114] The interaction storage controller 20a controls the interaction information received from the sensing device attached to / mounted on the object to be stored in the emotion information DB 24. The sensing device includes a humidity sensor, a temperature sensor, a vibration sensor, an infrared sensor, a camera, a tactile sensor, a gyro sensor, etc., and senses the interaction of one object with another object.

[0115] The estimation unit 20b estimates the interaction stored in the emotion information DB 24. Although the interaction evaluation method is not particularly limited, for example, when the interaction is more desirable for the object, the estimation unit 20b evaluates the interaction applied to the object as higher, and specifically assigns it a score in the range of -1 to 1. The estimation result is associated with the interaction and stored in the emotion information DB 24.

[0116] The object management unit 20 c performs management such as registration, change, and deletion of information on objects stored in the object DB 22 .

[0117] The related object search unit 20d searches the object DB 22 and the emotion information DB 24 for an object with which an interaction has been generated with respect to the request object ID as a related object.

[0118] The object DB 22 is a storage unit that stores the object ID of each object. In addition, the object DB 22 stores various types of information about the object, such as product name, product type, manufacturer ID, model, and manufacturing date and time, in addition to the object ID.

[0119] The emotion information DB 24 is a storage unit that stores interactions between objects and estimates of the interactions.

[0120] The specific configuration of the emotion server 2 according to the present embodiment has been described.

[0121] <2-4. Environmental Adjustment Operation>

[0122] Next, we will refer to Figure 8 The environmental conditioning operation in the environmental conditioning system including the above-mentioned apparatus is described. Figure 8 : is a flowchart showing the environmental adjustment operation processing of the environmental adjustment system according to the present embodiment. The flowchart shows the process from the start to the end of applying environmental changes when the user moves from a certain place to a destination. The environmental adjustment server 7 obtains the weather and the like of the user's moving route according to the user's destination, moving route and time, compares the weather with the user's mood, and controls the mobile object 6 to change the user's surrounding environment so that the user can move in a more pleasant state.

[0123] Specifically, first, in step S103 , the environment adjustment server 7 acquires environment information about the current location and destination of the target user and destination arrival time information.

[0124] Subsequently, in step S106 , the emotion estimation unit 70 c of the environment adjustment server 7 acquires the user's characteristics, hobbies, tastes, biological information, and the like.

[0125] Next, in step S109, the emotion estimation unit 70c estimates the emotion of the user. Here, the emotion estimation unit 70c can estimate the emotion of the user in the current environment on the route from the current location to the destination based on the user's characteristics, hobbies, tastes, biometric information, etc. For example, when the user is a woman in her 30s, the fact that the user usually carries a parasol is extracted from past behavior history, conversations, writings, etc., and the fact that sunscreen and parasols are purchased are extracted from the purchase history, and it is estimated that the woman has a feeling of worrying about ultraviolet rays, direct sunlight, and sunburn (unpleasant emotions). On the other hand, when it is extracted that the user usually enjoys the taste of sunbathing, it is estimated that the user has a feeling of enjoying ultraviolet rays, direct sunlight, and sunbathing (pleasant emotions). In addition, the emotion estimation unit 70c can estimate the user's emotions (pleasant / unpleasant feelings) about the current surrounding environment based on the user's biometric information (body temperature, pulse, heartbeat, sweating, brain waves, etc.).

[0126] Subsequently, the controller 70 of the environment adjustment server 7 determines whether it is necessary to change the environment based on the estimation result in step S112. Specifically, for example, when the user has a feeling of worrying about ultraviolet rays, direct sunlight, and sunburn (unpleasant emotions), the controller 70 determines that it is necessary to change the environment if the weather at the current location and the destination is sunny and exposed to direct sunlight. On the other hand, when the user has a feeling of enjoying sunbathing (pleasant emotions), the controller 70 determines that it is not necessary to change the environment if the weather at the current location and the destination is sunny and exposed to direct sunlight. In this way, whether the user has pleasant emotions or unpleasant emotions for a particular environment depends on the user's traits, hobbies, tastes, etc., and the controller 70 can determine whether it is necessary to change the environment based on the user's traits, hobbies, tastes, biometric information, etc.

[0127] Next, in the case where the environment needs to be changed ("Yes" in S112), in step S115, the heat map generation / acquisition unit 70e acquires and generates an environmental heat map according to the necessary environmental changes. For example, when the environment needs to block ultraviolet rays and direct sunlight, the heat map generation / acquisition unit 70e acquires from the heat map server 75 an environmental heat map representing places that are easily exposed to direct sunlight in an area including a moving route from the current position to the destination. In addition, the heat map generation / acquisition unit 70e can reflect the current environmental information about the current position and destination in the acquired environmental heat map to update the environmental heat map. In addition, when the corresponding environmental heat map cannot be acquired, the heat map generation / acquisition unit 70e can generate a corresponding environmental heat map based on the current environmental information about the current position and destination, etc.

[0128] Subsequently, in step S118, the heat map generation / acquisition unit 70e acquires and generates an emotion heat map according to the necessary environmental changes. For example, the heat map generation / acquisition unit 70e acquires from the heat map server 75 an emotion heat map of the emotions of multiple users who are worried about ultraviolet rays and direct sunlight, unpleasant emotions due to exposure to direct sunlight, etc. in an area including a moving route from the current position to the destination. In addition, the heat map generation / acquisition unit 70e can reflect the current emotions of the target user in the acquired emotion heat map to update the emotion heat map. In addition, when the corresponding emotion heat map cannot be acquired, the heat map generation / acquisition unit 70e can generate a corresponding emotion heat map based on the current emotions of the target user, etc.

[0129] Thereafter, in step S121, the heat map generation / acquisition unit 70e integrates the acquired environment heat map with the emotion heat map to generate a comprehensive heat map. For example, by integrating the locations of unpleasant emotions generated by exposure to direct sunlight with the locations that are easily exposed to direct sunlight, the area where the target user needs to be protected from direct sunlight is highly identified. The heat map generation / acquisition unit 70e outputs the generated comprehensive heat map to the mobile object controller 70g.

[0130] Subsequently, in step S124, the mobile object controller 70g refers to the integrated heat map to determine whether there is a heat map area above a threshold value on the user's moving route.

[0131] Thereafter, when there is a heat map area higher than the threshold value ("Yes" in S124), in step S127, the mobile object controller 70g controls the mobile object 6 to change the surrounding environment of the target user. Specifically, the mobile object controller 70g causes the mobile object 6 to track the target user to fly in a position shielded from rain or sunlight, blow cold or warm air, turn on the lights, or play music to change the surrounding environment of the target user. The environmental change will be described in detail later.

[0132] Then, in step S130, the environment adjustment server 7 determines whether the user has arrived at the destination. For example, when the user has arrived at the destination, the environment adjustment operation ends, so the mobile object controller 70g controls the mobile object 6 to return to the predetermined base station.

[0133] When the user has not arrived at the destination (No in S130), in step S133, the environment adjustment server 7 acquires the user's current position information and the environment information of the current position through the target object detection unit 70b and the environment information acquisition unit 70d.

[0134] Then, in step S136, the heat map generation / acquisition unit 70e updates the environment heat map based on the acquired environment information, and integrates the environment heat map with the emotion heat map to update the integrated heat map. The updated integrated heat map is output to the mobile object controller 70g, and steps S124 to S136 are repeated until the user arrives at the destination.

[0135] The operation process of the environment adjustment system according to the present embodiment has been described in detail. Next, a specific example of the environment change according to the present embodiment will be described through a plurality of specific examples.

[0136] <<3. Examples of environmental changes>>

[0137] <3-1. Protection from rain and sunlight>

[0138] First, refer to Figures 9 to 16 A case where the environment changes in which rain and sunlight are blocked by the mobile object 6 is described.

[0139] Fig. 9 6 is an explanatory diagram of the general situation of blocking rain and sunlight by the mobile object 6 of this embodiment. Fig. 9 As shown, when the mobile object 6 is a flying object, the mobile object can fly and track the mobile user while remaining above the user to act as an umbrella (mobile object 6-1) or a parasol (mobile object 6-2), so that the user is not exposed to rain or direct sunlight, thereby locally changing the user's surroundings.

[0140] Here, the mobile object 6 may include a transformation controller 65 for extending the shielding range as described above, and the shielding range for shielding rain or sunlight is extended by transformation control to apply environmental changes to a friend with the target user, a baby carriage, or a large luggage, etc. For example, Fig.10 As shown in FIG. 1 , the shielding range of the mobile object 6 can be extended by changing the size of the mobile object 6 having the plate member 651 extending in a circular shape from the center of the mobile object 6. Otherwise, Fig.11As shown, multiple moving objects 6a and 6b are connected to block a wider range. Specifically, in order to expand the blocking range of the main moving object 6a, another moving object 6b near the main moving object 6a is called, and the plate 653 set in the moving object 6a is connected to the moving object 6b when they are close to each other (for example, the magnet set in the contact point of the moving object 6b is turned on, etc.). Then, as shown in FIG. Fig.11 As shown, by controlling the two moving objects to fly a predetermined distance away from each other, the shielding range can be extended by the plate 653. The slave moving object 6b can fly in the same flight mode as that of the master moving object 6a.

[0141] This occlusion range can be set manually by the target user, or automatically set by the mobile object controller 70g of the environmental conditioning server 7 by tracking through facial recognition, extracting features of clothing or belongings from captured images, or extracting specific tags attached to belongings.

[0142] (Rain protection operation)

[0143] Next, we will refer to Fig.12 The operation process when the environment change for shielding against rain is performed by the mobile object 6-1 is described. Fig.12 : is a flowchart showing the operation process for environmental change of shielding from rain according to the present embodiment.

[0144] like Fig.12 As shown, first, in step 203, the mobile object controller 70g of the environment adjustment server 7 sets the initial position of the target user. Figure 8 As described, the mobile object controller 70g determines the area where the environment needs to be changed on the moving route from the current position of the target user to the destination, and controls the mobile object 6-1. In this case, when the target user passes through the area where the environment needs to be changed, the mobile object controller 70g sets the corresponding target area as the initial position of the target user, and starts the mobile object 6-1 with a base station close to the target area as the starting point to fly. This embodiment is based on the following situation: wherein each mobile object 6-1 with a base station installed outdoors or the like as a starting point flies, and when the target user moves to the next area, the flying mobile object 6-1' with a base station installed in the next area as the starting point tracks the target user. However, this embodiment is not limited to this, and for example, the mobile object 6-1 owned by the target user can be controlled to continuously track the target user during the movement of the target user.

[0145] Next, in step S206, the target user is detected by the mobile object 6-1. Specifically, for example, based on the captured image captured by the camera 62a included in the target object sensor 62 of the mobile object 6, the target object detection unit 70b of the environment adjustment server 7 performs image analysis to recognize the facial image of the target user to detect the target user. In addition, the target user detection accuracy can be improved by detecting a mark (e.g., an infrared reflector or an infrared LED) attached to the target user via the infrared camera 62b.

[0146] Subsequently, when the target user is detected (YES in S209), the mobile object controller 70g acquires the direction and amount of rain falling on the target user based on the detection value of the rain / wind direction sensor (an example of the environmental sensor 64) provided in the mobile object 6-1 in step S212. Fig.13 The rainfall unit vector Rv(x, y, z) of the rain shown.

[0147] Thereafter, the mobile object controller 70g controls the mobile object 6-1 to perform rain shielding by moving the mobile object 6-1 to the space between the target user and the rain. Here, the relative position of the mobile object 6-1 moving with respect to the target user is obtained as follows.

[0148] Fig.13 6-1 is an explanatory diagram of the calculation of the relative position of the mobile object 6-1 with respect to the target user during the rain shelter period. Fig.13 As shown, using the direction of rainfall Rv (x, y, z), the position of the target user T (x, y, z), and the height dh obtained by subtracting the height of the target user from the height of the mobile object 6-1, the position O (x, y, z) of the mobile object 6-1 is obtained according to the following formula 1. Rv_z represents the z component of Rv.

[0149] (Formula 1)

[0150] O=(T-Rv·dh / Rv_z)

[0151] The mobile object controller 70g controls the mobile object 6-1 to move to the obtained position O(x, y, z). In addition, when a wide range needs to be blocked, such as when there are multiple users, the mobile object controller 70g controls the blocking range to be expanded by connecting multiple mobile objects 6-1 or transforming the mobile object 6-1.

[0152] Thereafter, in step S218, the mobile object controller 70g calculates the movement vector of the target user based on the previous position and the current position of the target user. In addition, the mobile object 6-1 flies and tracks the target user while maintaining the relative position O (x, y, z) relative to the target user, and keeps the target user in sight while tracking the target user through the camera 62a or receiving radio waves (Bluetooth (registered trademark) or Wi-Fi (registered trademark)) transmitted from a communication terminal (smart device, smartphone, etc.) owned by the target user through the radio signal receiver 62c.

[0153] Subsequently, in step S221, the mobile object controller 70g determines whether the target user has arrived at the destination, and terminates the tracking flight and environment adjustment control by the mobile object 6-1 when the target user has arrived at the destination.

[0154] On the other hand, when the target user is not detected ("No" in S209), if a fixed time has not passed ("No" in S224), the mobile object controller 70g in step S227 estimates the current position of the target user moving toward the destination based on the position of the target user finally detected, the moving vector calculated at this time, and the current elapsed time.

[0155] Then, in step S230, the mobile object controller 70g controls the mobile object 6-1 to move toward the estimated current position of the target user.

[0156] In a case where the fixed time has elapsed without the target user being detected (YES in S224), the mobile object controller 70g sets the initial position of the target user again (S203).

[0157] (Operation of blocking sunlight)

[0158] Next, we will refer to Fig.14 The operation process when an environmental change of blocking sunlight is performed by the mobile object 6 - 2 is described. Fig.14 FIG. 1 is a flowchart showing an operation process for environmental change for blocking sunlight according to the present embodiment. Fig.12 The operation process for shielding from rain is the same as the process described above.

[0159] like Fig.14 As shown, first, in step S303, the mobile object controller 70g of the environment adjustment server 7 sets the initial position of the target user.

[0160] Then, in step S306, the target user is detected by the mobile object 6-2.

[0161] Subsequently, when the target user is detected ("Yes" in S309), in step S312, the mobile object controller 70g acquires the position of the sun and the direction of the sunlight relative to the target user based on the detection values ​​of the illuminance sensor, the solar azimuth acquisition unit, and the calendar / clock unit (an example of the environmental sensor 64) provided in the mobile object 6-2. Fig.15 The solar position acquisition unit corresponding to the example of the environmental sensor 64 can acquire the position and direction of the sun based on the current position information (latitude, longitude and altitude) acquired by the latitude and longitude positioning unit 64a and the altitude sensor 64b and the current date and time information detected by the calendar / clock unit.

[0162] Next, in step S315, the mobile object controller 70g controls the mobile object 6-2 to perform sun shielding by moving the mobile object 6-2 to the space between the target user and the sun. Here, the relative position of the mobile object 6-2 moving relative to the target user is obtained as follows.

[0163] Fig.15 6-2 is an explanatory diagram of the calculation of the relative position of the mobile object 6-2 with respect to the target user during the period of blocking sunlight. Fig.15 As shown, using the direction of sunlight Rv (x, y, z), the position of the target user T (x, y, z), and the height dh obtained by subtracting the height of the target user from the height of the mobile object 6-2, the position O (x, y, z) of the mobile object 6-2 is obtained according to the following formula 2. Sv_z represents the z component of Sv.

[0164] (Formula 2)

[0165] O=(T-Sv·dh / Sv_z)

[0166] The mobile object controller 70g controls the mobile object 6-2 to the position O(x, y, z) acquired in this way.

[0167] Thereafter, in step S318, the mobile object controller 70g calculates the movement vector of the target user based on the previous position and the current position of the target user. In addition, the mobile object 6-2 flies and tracks the target user while maintaining the relative position O (x, y, z) relative to the target user, and keeps the target user in sight while tracking the target user through the camera 62a or receiving radio waves (Bluetooth (registered trademark) or Wi-Fi (registered trademark)) transmitted from a communication terminal (smart device, smartphone, etc.) owned by the target user.

[0168] Subsequently, in step S321, the mobile object controller 70g determines whether the target user has arrived at the destination, and terminates the tracking flight and environment adjustment control by the mobile object 6-2 when the target user has arrived at the destination.

[0169] On the other hand, when the target user is not detected ("No" in S309), if a fixed time has not passed ("No" in S324), the mobile object controller 70g in step S327 estimates the current position of the target user moving toward the destination based on the position of the target user finally detected, the moving vector calculated at this time, and the current elapsed time.

[0170] Then, in step S330, the mobile object controller 70g controls the mobile object 6-2 to move toward the estimated current position of the target user.

[0171] When the fixed time has passed without the target user being detected (YES in S324), the mobile object controller 70g sets the initial position of the target user again (S303).

[0172] (Change the occlusion range based on the emotion value)

[0173] In this embodiment, the shielding range can be changed based on the target user's emotional value. Therefore, it is possible to provide an environmental change depending on the target user's feelings by preferentially shielding objects that are important to the target user's daily life, shielding people who are important to the target user, etc. This will refer to Fig.16 Describe in detail.

[0174] Fig.16 FIG. 4 is a flowchart showing a process of changing the occlusion range based on the emotion value according to the present embodiment. Fig.16 As shown, first, in step S410, the emotion value calculation unit 70f of the environment adjustment server 7 obtains the object ID of the target user.

[0175] Next, in step S412, an object (eg, a baby carriage or luggage) moving together with the target user or a person moving together with the target user is detected by the target object detection unit 70b.

[0176] Subsequently, in step S415, the emotion value calculation unit 70f acquires the object ID of the detected object or person. This object ID (also referred to as a related object ID) may be pre-registered in the user DB 72, or may be obtained by the related object search unit 20d (refer to Figure 7 ) inquires the emotion server 2 to search in the object DB 22 based on the analysis result of the captured image acquired by the camera 62a of the mobile object 6.

[0177] Then, the emotion value calculation unit 70f specifies the object ID of the target user and the object ID of the object or person moving with the target user, and acquires the interaction evaluation between the target user and the object or person from the emotion server 2. The interaction evaluation is an estimated value (e.g., a numerical value in the range of -1.0 to 1.0) of the target user's behavior with respect to the person, such as conversation or mail, or the target user's behavior with respect to the object, such as preservation, maintenance, wearing, or appreciation.

[0178] Subsequently, in step S421, the emotion value calculation unit 70f calculates the emotion value of the target user for the object / person. Although the emotion value calculation formula is not particularly limited, for example, the interaction between the target user and the specific object (related object) can be classified by interaction type, and the interaction evaluation of the specific object can be averaged using a weighting function according to the interaction type (refer to the following formula).

[0179] (Formula 3)

[0180] Ave(ω intA ·(Ave(E intA ·TW(t))),ω intB ·(Ave(E intB ·TW(t))),ω intC ·(Ave(E intC ·TW(t))),…)

[0181] here,

[0182] ω intA : The weight of interaction A

[0183] E intA : Estimated value of interaction A

[0184] TW(t): weighting function for the elapsed time of the estimate

[0185] The above formula 3 corresponds to a case where the emotion value is calculated based on the interaction evaluation between people and a weighting function TW(t) of the elapsed time for the estimated value is used. t specifies the interaction time (or estimated time), and weighting is performed according to the time elapsed since the interaction was performed. This is because in the case of a relationship between people, the most recent interaction is considered to be more important than the past interaction.

[0186] Next, in step S424, the mobile object controller 70g determines whether the occlusion range needs to be changed based on the emotion value calculated by the emotion value calculation unit 70f. Specifically, when the relative emotion value of the target user relative to the object / person is higher than the threshold, the mobile object controller 70g determines that the object / person is occluded together with the target user or is occluded in priority to the target user.

[0187] When the blocking range needs to be changed (YES in S424), in step S427, the mobile object controller 70g instructs the mobile object 6 to change the blocking range.

[0188] According to the present embodiment, as described above, when the target user walks with an expensive antique, for example, the antique can be shielded by the mobile object 6 so that the antique is not exposed to the sun or rain. When the target user is with children or friends, they can share the shielding range.

[0189] <3-2. Lighting>

[0190] Next, other environment adjustment control performed by the mobile object 6 of the present embodiment will be described. Fig.17 6-3 is an explanatory diagram of the lighting control performed by the mobile object 6-3 of this embodiment. Fig.17 As shown, when it is estimated that the target user has an emotion of fear of walking on a dark road at night, the target user can be made comfortable by performing environmental adjustment control of lighting the road around the target user by the mobile object 6-3. This will refer to Fig.18 Describe in detail.

[0191] Fig.18 6-3 is a flowchart showing the lighting control process performed by the mobile object 6-3 of this embodiment. Fig.18 As shown, first, in step S503, the mobile object controller 70g of the environmental adjustment server 7 makes the mobile object 6-3 fly to the set initial position of the target user to detect the target user through the target object detection unit 70b based on the captured image of the infrared camera device 62b (for night roads) of the mobile object 6-3.

[0192] Then, in step S506, the mobile object controller 70g sets the flying height of the mobile object 6-3. The flying height can be set in advance.

[0193] Next, in step S509, the mobile object controller 70g calculates the moving direction (unit vector Fv) of the target user.

[0194] Thereafter, in step S512, the mobile object controller 70g determines whether the volume of the footsteps of the target user is greater than a threshold value based on the audio data acquired through the microphone array 62d of the mobile object 6-3.

[0195] Furthermore, in step S515, the mobile object controller 70g determines whether a change in the moving speed of the target user exceeds a threshold value based on the captured image acquired by the infrared camera 62b or the like of the mobile object 6-3.

[0196] When the footsteps are greater than a threshold value ("yes" in S512) or the movement speed changes by more than a threshold value ("yes" in S515), in step S518, the mobile object controller 70g determines whether the target user's fear level exceeds the threshold value. When a person walks alone on the road at night, a pedestrian who feels fear increases his or her walking speed or his or her footsteps become louder. Therefore, such information becomes a barometer of the degree of fear of the surrounding environment. In addition, the degree of fear can be calculated based on changes in the user's biological information such as heart rate. The user's heart rate can be obtained by the smart band 8 worn by the user. Otherwise, an area that particularly frightens the user according to collective intelligence or personal traits can be specified based on a comprehensive heat map (for example, a combination of an emotional heat map indicating a sense of fear and an environmental heat map indicating a place where there is a small amount of traffic and danger is prone to occur) to calculate the degree of fear.

[0197] When the degree of fear exceeds the threshold value ("Yes" in S518), in step S521, the mobile object controller 70g performs adjustment to increase the lighting intensity or expand the lighting range through the lighting unit (an example of the output unit 66). Otherwise, the mobile object controller 70g controls the speaker (an example of the output unit 66) to sound an alarm.

[0198] Then, after adjustment according to the degree of fear, or when the volume or speed change of the footsteps does not exceed the threshold value, in step S524, the mobile object controller 70g controls the lighting unit (an example of the output unit 66) to illuminate the front of the moving direction of the target user. Here, for example, the lighting direction and lighting position according to the mobile object 6-3 are obtained as follows.

[0199] Fig.19 6-3 is an explanatory diagram of calculation of the illumination direction (angle) and position of the moving object 6-3 during illumination. Fig.19 As shown, when the position O (x, y, z) of the mobile object 6-3, the position T (x, y, z) of the target user, the moving direction unit vector Fv (x, y, z), and the distance td between the target user and the lighting position are preset to specified values, the lighting position P is obtained according to the following formula 4.

[0200] (Formula 4)

[0201] P=T+Fv·td

[0202] Therefore, the unit vector Lv according to the lighting angle of the lighting unit is obtained according to the following Formula 5.

[0203] (Formula 5)

[0204] Lv=(PO) / |PO|

[0205] In addition, the mobile object controller 70g causes the mobile object 6-3 to fly while tracking the target user, maintains the lighting angle Lv of the lighting unit relative to the target user, and keeps the target user within sight while tracking the target user through the infrared camera device 62a or receiving radio waves (Bluetooth (registered trademark) or Wi-Fi (registered trademark)) transmitted from a communication terminal (smart device, smartphone, etc.) owned by the target user.

[0206] <3-3. Output of wind, water, etc.>

[0207] Furthermore, the mobile object 6 according to the present embodiment can provide a pleasant environment on hot / cold days by emitting warm wind, cold wind, air, mosquito repellent, water, etc. from the output unit 66, blocking cigarette smoke or pollen, or stopping insects such as mosquitoes so that the target user has a pleasant feeling. Whether the target user has an unpleasant feeling due to heat, cold, cigarette smoke, insects, etc. can be estimated by the emotion estimation unit 70c based on the target user's characteristics, hobbies, tastes, bio-information, etc. In addition, in the present embodiment, sensing of the surrounding environment is performed by a humidity sensor, a temperature sensor, an air quality sensor, a pollen sensor, an odor sensor, etc. as examples of the environment sensor 64.

[0208] The following will refer to Figure 20 to Figure 23 An implementation method for changing the surrounding environment of a target user by wind, water, etc. is described in detail.

[0209] (Air Curtain)

[0210] Fig. 20 FIG. 1 is a schematic diagram of environmental change through an air curtain according to this embodiment. Fig. 20 As shown, the mobile object 6-4 that flies and tracks the target user can change the environment into a pleasant environment by outputting cool wind by the wind output unit (an example of the output unit 66) when the target user has an unpleasant feeling due to sweating in hot weather. In addition, when the target user has an unpleasant feeling because he or she has a tendency to be bitten by mosquitoes and there are insects near him or her, the mobile object 6-4 can change the environment into a pleasant environment by spraying mosquito repellent chemicals.

[0211] Fig.21 FIG. 4 is a flowchart showing the operation process of changing the environment using the air curtain according to the present embodiment. Fig.21 As shown, first, in step S603, the mobile object controller 70g of the environment adjustment server 7 makes the mobile object 6-4 fly to the set initial position of the target user to detect the target user through the target object detection unit 70b based on the image captured by the camera 62a of the mobile object 6-4, etc.

[0212] Then, in step S606, the mobile object controller 70g sets the flying height of the mobile object 6-4. The flying height can be set in advance.

[0213] Subsequently, in step S609, the mobile object controller 70g calculates the moving direction of the target user.

[0214] Thereafter, in step S612, the mobile object controller 70g controls the mobile object 6-4 to output (blow) cold / warm wind or spray mosquito repellent chemicals around the target user. The air flow rate and temperature are adjusted based on the target user's mood, surrounding humidity, temperature, etc.

[0215] Furthermore, in step S615, the mobile object controller 70g controls the mobile object 6-3 to fly and track the target user.

[0216] (Blocks cigarette smoke)

[0217] Next, we will refer to Fig. 22 and Fig.23 The environmental change of blocking cigarette smoke by the mobile object 6-5 is described. Although cigarette smoke blocking can be achieved by the above-mentioned air curtain, the mobile object 6-5 can discharge water to extinguish the cigarette 50 near it, such as Fig. 22 Drainage control may be performed only on areas where smoking is prohibited on the road according to regulations or the like.

[0218] Fig.23 FIG. 2 is a flowchart showing an operation process of changing the environment by blocking smoke according to the present embodiment. Fig.23 As shown, first, in step S623, the mobile object controller 70g of the environment adjustment server 7 makes the mobile object 6-5 fly to the set initial position of the target user to detect the target user through the target object detection unit 70b based on the image captured by the camera 62a of the mobile object 6-5, etc.

[0219] Then, in step S626, the mobile object controller 70g searches for the light of the cigarette by performing a 360° infrared search around the mobile object 6-5 based on the image captured by the infrared camera device 62b of the mobile object 6-5, etc. Otherwise, the mobile object controller 70g may measure the wind direction using a wind direction sensor provided in the mobile object 6-5, and search intensively with infrared rays for the windward direction of the cigarette flowing toward the target user.

[0220] Next, in step S629, the cigarette to be extinguished is detected by the target object detection unit 70b.

[0221] Then, when the cigarette to be extinguished is detected (YES in S629), in step S632, the mobile object controller 70g instructs the mobile object 6-5 to drain water from the drainage unit (an example of the output unit 66) to extinguish the cigarette.

[0222] Furthermore, in step S635, the mobile object controller 70g controls the mobile object 6-5 to fly and track the target user.

[0223] <3-4. Music Playback>

[0224] In addition, the mobile object 6 according to the present embodiment can provide a pleasant environment by playing music via a speaker (an example of the output unit 66). For example, when the emotion estimation unit 70c estimates that the target user has a feeling of frustration or unhappiness due to surrounding noise, the target user can have a pleasant feeling by playing music that the target user likes according to the characteristics, hobbies and tastes of the target user through the mobile object 6.

[0225] Specifically, the mobile object 6-6 flying and tracking the target user plays music above the target user, such as Fig.24 As shown. Therefore, the target user listens to music without wearing the headphone speakers or earphone speakers, and can therefore hear the surrounding environment sounds at the same time and obtain the effect of increased safety while walking or jogging. In addition, music can be enjoyed even during cycling where the use of headphone speakers or earphone speakers is legally prohibited. In addition, it is also possible to make only the target user located below the mobile object 6-6 hear the music by utilizing this technology or beamforming technology with a speaker array.

[0226] Fig.25 1 is a flowchart showing the operation process of environment change by music playback according to the present embodiment. Fig.25 As shown, first, in step S643, the mobile object controller 70g of the environmental adjustment server 7 makes the mobile object 6-6 fly to the set initial position of the target user to detect the target user through the target object detection unit 70b based on the image captured by the camera device 62a of the mobile object 6-6, etc.

[0227] Then, in step S646, the mobile object controller 70g sets the flying height of the mobile object 6-6. The flying height can be set in advance.

[0228] Subsequently, in step S649, the mobile object controller 70g calculates the moving direction of the target user.

[0229] Thereafter, in step S652, the mobile object controller 70g instructs the mobile object 6-6 to play music through the speaker (an example of the output unit 66). The music to be played may be selected based on the hobbies, tastes, and past music playing history of the target user.

[0230] Furthermore, in step S655, the mobile object controller 70g controls the mobile object 6-6 to fly and track the target user.

[0231] <<4. Conclusion>>

[0232] As described above, in the environment adjustment system 1 according to the embodiment of the present disclosure, by causing the mobile object 6 to track the moving target user, the surrounding environment can be locally changed according to the emotion of the target user.

[0233] The preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, and the present disclosure is not limited to the above examples. Those skilled in the art may find various changes and modifications within the scope of the appended claims, and it should be understood that they will naturally fall within the technical scope of the present disclosure.

[0234] For example, a computer program may be written for causing hardware such as CPU, ROM, and RAM included in the above-described mobile object 6 and environment adjustment server 7 to exhibit the functions of the mobile object 6 and environment adjustment server 7. Furthermore, a computer-readable storage medium having the computer program stored therein may be provided.

[0235] In addition, in the above-mentioned embodiment, the movement of the mobile object 6 is controlled by the environment adjustment server 7, but the environment adjustment system according to the present embodiment is not limited thereto, and the movement control may be performed by the controller (CPU) of the mobile object 6. For example, the mobile object 6 may have a plurality of components including Figure 4 At least some of the functional components in the environment adjustment server 7 shown (target object detection unit 70b, environment information acquisition unit 70d, emotion estimation unit 70c, heat map generation / acquisition unit 70e, emotion value calculation unit 70f and mobile object controller 70g).

[0236] In addition, the functions of at least two of the mobile objects 6-1 to 6-6 can be combined. For example, in the case of a dark road on a rainy day, two environmental changes of shielding rain and illuminating the road at night are provided. In this case, the mobile object controller 70g can predict the flight time and the time to use each function based on the comprehensive heat map, and select the required function, battery capacity, and body.

[0237] In addition, in addition to avoiding objects falling overhead (for example, blocking rain), blocking light (for example, blocking sunlight), light exposure, voice output, blowing, water spraying and chemical spraying, the environmental changes of the environmental adjustment system of this embodiment can be achieved through image projection, sound insulation, gas absorption, liquid absorption, heat generation, control of robot arms, etc.

[0238] Furthermore, although the emotion value is calculated by the emotion value calculation unit 70 f of the environment conditioning server 7 in the environment conditioning system 1 according to the present embodiment, the present disclosure is not limited thereto, and the emotion value may be calculated by the emotion server 2 .

[0239] In addition, the effects described in this specification are merely illustrative or exemplary effects, not restrictive. That is, with or in place of the above effects, the technology according to the present disclosure can achieve other effects that are clear to those skilled in the art based on the description of this specification.

[0240] Additionally, the present technology may also be configured as follows.

[0241] (1) A control system comprising:

[0242] an estimating unit that estimates a user's emotion; and

[0243] A mobile object controller controls a mobile object to change the user's surrounding environment according to the estimated emotion.

[0244] (2) The control system according to (1), wherein the estimation unit estimates the emotion of the user based on biological information of the user.

[0245] (3) The control system according to (1) or (2), wherein the estimation unit estimates the emotion of the user based on traits, hobbies, and tastes of the user.

[0246] (4) The control system according to (3), wherein the estimation unit estimates the emotion of the user with reference to a surrounding environment of the user.

[0247] (5) The control system according to any one of (1) to (4), further including an emotion value calculation unit that calculates the user's emotion value for the specific object based on the interaction evaluation for the specific object.

[0248] (6) The control system according to (5), wherein the mobile object controller performs control to adjust a variation range according to the emotion value for the specific object.

[0249] (7) The control system according to any one of (1) to (6), further comprising a generation unit that generates an emotion heat map based on the emotions of the plurality of users estimated by the estimation unit,

[0250] The mobile object controller controls the mobile object according to the emotion heat map to change the user's surrounding environment.

[0251] (8) The control system according to (7),

[0252] wherein the generating unit generates an environment heat map based on information about the user's surrounding environment, and

[0253] The mobile object controller controls the mobile object to change the user's surrounding environment according to the emotion heat map and the environment heat map.

[0254] (9) The control system according to (8), wherein the generation unit predicts and generates a future emotion heat map or an environment heat map.

[0255] (10) A control system according to any one of (1) to (9), wherein, when the estimation unit estimates that the user has an unpleasant emotion, the mobile object controller controls the mobile object to change the surrounding environment of the user so that the user has a pleasant emotion.

[0256] (11) A control system according to (10), wherein, when the estimation unit estimates that the user's emotion toward a specific environment is unpleasant, the mobile object controller controls the mobile object to change the specific environment around the user so that the user has a pleasant emotion.

[0257] (12) A control system according to (11), wherein the mobile object controller determines an area on the moving route to the destination where the environment is to be changed based on an emotion heat map and an environment heat map according to the estimated emotions of the user relative to the specific environment, and controls the mobile object responsible for the area to change the surrounding environment of the user.

[0258] (13) The control system according to any one of (1) to (12), wherein the mobile object is a flying object, a ground mobile object, or an underwater mobile object.

[0259] (14) A control method comprising:

[0260] Estimating the user's sentiment; and

[0261] The user's surrounding environment is changed by controlling a moving object according to the estimated emotion.

[0262] (15) A storage medium storing a program for causing a computer to:

[0263] an estimating unit that estimates a user's emotion; and

[0264] A mobile object controller controls a mobile object to change the user's surrounding environment according to the estimated emotion.

[0265] Reference numerals list

[0266] 1 Environmental Control System

[0267] 2 Emotion Server

[0268] 20 Controller

[0269] 20a Interactive Storage Controller

[0270] 20b Estimation Unit

[0271] 20c Object Management Unit

[0272] 20d Related Object Search Unit

[0273] 21 Communication unit

[0274] 22 Object Database

[0275] 24 Emotion Information Database

[0276] 3 Network

[0277] 6 Moving Objects

[0278] 61 Communication unit

[0279] 62 Target object sensor

[0280] 63 Mobile Object Controller

[0281] 64 Environmental Sensors

[0282] 65 Transform Controller

[0283] 66 Output Units

[0284] 7 Environmental Adjustment Server

[0285] 70 Controller

[0286] 70a User Settings and Management Unit

[0287] 70b Object Detection Unit

[0288] 70c Sentiment Estimation Unit

[0289] 70d Environmental information acquisition unit

[0290] 70e Heatmap generation / acquisition unit

[0291] 70f Emotion value calculation unit

[0292] 70g Mobile Object Controller

[0293] 71 Communication unit

[0294] 72 User Database

Claims

1. A control system, comprising: an estimation unit, which estimates the user's emotions; as well as a mobile object controller that controls a mobile object to change the user's surroundings while tracking the user according to the estimated emotions, The changes include: 1) physically protecting the user from the elements of the surrounding environment by forming a shield, creating an air curtain, or draining water by the moving object in the vicinity of the user; or 2) Controlling the mobile object to emit sound or light to the user according to the estimated emotion.

2. The control system according to claim 1, wherein: The estimation unit estimates the emotion of the user based on at least one of: biological information of the user; traits, hobbies, and tastes of the user.

3. The control system according to claim 2, wherein: The estimation unit estimates the emotion of the user with reference to the surrounding environment of the user. 4 . The control system according to claim 1 , further comprising an emotion value calculation unit which calculates the emotion value of the user with respect to the specific object based on an interaction evaluation with the specific object.

5. The control system according to claim 4, wherein: The mobile object controller performs control to adjust a variation range according to the emotion value for the specific object.

6. The control system according to claim 4, wherein: The emotion value calculation unit is configured to classify the interaction between the user and the specific object according to the interaction type, and calculate the user's emotion value for the specific object by averaging the interaction evaluations for the specific object using a weighting function according to the interaction type.

7. The control system according to claim 1, further comprising a generating unit that generates an emotion heat map based on the emotions of the plurality of users estimated by the estimating unit, in, The mobile object controller controls the mobile object to change the user's surrounding environment according to the emotion heat map.

8. The control system according to claim 7, in, The generating unit generates an environment heat map based on information about the surrounding environment of the user, and The mobile object controller controls the mobile object to change the user's surrounding environment according to the emotion heat map and the environment heat map.

9. The control system according to claim 8, wherein: The generating unit predicts and generates a future emotion heat map or an environment heat map.

10. The control system according to claim 1, wherein: The mobile object controller is also configured to determine areas where the environment needs to be changed on the moving route from the user's current location to the destination, and when the user passes through each area where the environment needs to be changed, control the mobile object near the area to track the user.

11. The control system according to claim 1, wherein: The mobile object controller is further configured to control the mobile object to continuously track the user during movement of the user.

12. The control system according to claim 1, wherein: The mobile object controller is configured to track the user by maintaining a relative position of the mobile object with respect to the user.

13. The control system according to claim 1, wherein: The mobile object controller is further configured to control the mobile object to return to a predetermined base station after the user reaches a destination.

14. The control system according to claim 1, wherein: The mobile object controller is configured to simultaneously control a plurality of mobile objects to track the user.

15. The control system according to claim 1, wherein: When the estimation unit estimates that the user has an unpleasant emotion, the mobile object controller controls the mobile object to change the surrounding environment of the user so that the user has a pleasant emotion.

16. The control system according to claim 15, wherein: When the estimation unit estimates that the emotion of the user with respect to a specific environment is unpleasant, the mobile object controller controls the mobile object to change the specific environment around the user so that the user has a pleasant emotion.

17. The control system according to claim 16, wherein: The mobile object controller determines an area on a moving route to a destination where the environment is to be changed based on an emotional heat map and an environmental heat map according to the estimated emotions of the user relative to the specific environment, and controls the mobile object responsible for the area to change the user's surrounding environment.

18. The control system according to claim 1, wherein: The mobile object is a flying object, a ground mobile object or an underwater mobile object.

19. A control method, comprising: Estimate user sentiment; as well as controlling a mobile object to change the user's surroundings according to the estimated emotion while tracking the user, The changes include: 1) physically protecting the user from the elements of the surrounding environment by forming a shield, creating an air curtain, or draining water by the moving object in the vicinity of the user; or 2) Controlling the mobile object to emit sound or light to the user according to the estimated emotion.

20. A storage medium storing a program which, when executed by a computer, causes the computer to function as: an estimating unit that estimates a user's emotion; and a mobile object controller that controls a mobile object to change the user's surroundings while tracking the user according to the estimated emotions, The changes include: 1) physically protecting the user from the elements of the surrounding environment by forming a shield, creating an air curtain, or draining water by the moving object in the vicinity of the user; or 2) Controlling the mobile object to emit sound or light to the user according to the estimated emotion.

Images