Analysis device, analysis method, analysis program, analysis system, and learning data generation method

Abstract

To provide a technique for properly identifying a change factor of an emotion of a subject contributing to community building, facility building, etc.SOLUTION: An exemplary analysis device detects an emotion of a subject according to the movement of the subject, detects an emotion change position where the emotion has changed, based on the detected emotion, and identifies environmental information corresponding to the emotion change position as change factor information of the emotion.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an analysis device, an analysis method, an analysis program, an analysis system, and a learning data generation method for identifying factors that cause changes in the emotions of a subject. [Background technology]

[0002] One way to promote the region, attract customers, and secure sales is to appeal to the good points of the town and its facilities, but fundamentally, it is necessary to create towns and facilities that people will want to visit.

[0003] If the appeal of a town, facility, etc. is not known, then it is sufficient to promote its appeal, but if it does not have any appeal in the first place, then the first step is to create it. However, even if a town or facility has appeal and is well known, there are cases where people do not come. This is thought to be because there are negative factors that make it hard to visit the town, facility, etc. in the town, facility, etc. or in its surrounding area, or there is a lack of positive factors that make it hard to visit the town, facility, etc. in the town, facility, etc. or in its surrounding area. Making improvements to reduce the negative factors or increase the positive factors will attract people to the town, facility, etc., or will attract even more people than they do now.

[0004] On the other hand, quiet residential areas have a need to keep non-residents away from visiting. If this need is not met, it is likely that there are negative factors that discourage people from visiting the area or its surroundings that are lacking, or there are positive factors that discourage people from visiting the area or its surroundings that are present. Increasing the negative factors or reducing the positive factors, and making improvements that do not impede the convenience of residents, will lead to a decrease in visits by non-residents.

[0005] When building towns and facilities, designs are based on psychology and other academic fields that visualize the human mind (for example, the relationship between the color temperature of lighting and the mind). For example, lighting colors, tree placement, wall heights, and wall color schemes are designed to create an environment that is comfortable for people without creating a feeling of oppression.

[0006] One way to improve existing towns and facilities to make them more accessible to people is to visualize the negative and positive elements through expert knowledge and visitor surveys, and then improve those elements. For example, if the negative element is "traffic congestion," improvements can be implemented to alleviate that congestion. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2018-55550 A Summary of the Invention [Problem to be solved by the invention]

[0008] However, the environments of target locations vary widely, and there are some areas where the knowledge of experts can be applied and some that cannot. To supplement the knowledge of experts, surveys of visitors are conducted, but there are issues such as the inability to identify negative or positive elements from the survey unless the location left a very strong impression, or the inability of the visitors themselves to verbalize what negative or positive elements they perceived.

[0009] Furthermore, in Patent Document 1, the degree of satisfaction with a facility is calculated based on the emotions of the occupants after the vehicle has stayed at the facility. However, Patent Document 1 only grasps the emotions of people who actually stayed at the facility, and is unable to grasp the emotions of people who do not stay at the facility (people who do not visit), and is only able to grasp the degree of the occupants' emotions toward the facility as a whole, which poses a problem in that it is not possible to find an appropriate and specific direction for improvement.

[0010] In view of the above circumstances, an object of the present invention is to provide a technology for appropriately identifying factors that cause changes in the emotions of subjects, which contributes to urban development, facility development, and the like. [Means for solving the problem]

[0011] An exemplary analysis device of the present invention detects the emotions of a subject according to the subject's movements, detects the emotion change position where the emotion has changed based on the detected emotion, and identifies environmental information corresponding to the emotion change position as information on the cause of the change in emotion. [Effects of the Invention]

[0012] According to the exemplary embodiment of the present invention, information on singular points where a subject's emotions change can be obtained. The type and appearance of facilities present at singular points where emotions change, as well as the surrounding environment, are likely to be negative or positive factors when the subject visits the singular point. In other words, such singular points are expected to have factors that determine whether or not the subject can visit the point. Therefore, by analyzing the environment around the singular point based on the location information of the singular point, it becomes possible to obtain information useful for urban development, etc. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment; [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of a server according to a first exemplary embodiment of the present invention. [Figure 3] A diagram showing an example of a psychological plane table [Figure 4] A diagram showing an example of a static environmental change database. [Figure 5] FIG. 10 is a diagram showing an example of a target dynamic environment change type table; [Figure 6] A diagram showing an example of a dynamic environmental change database. [Figure 7] Flowchart of analysis process executed by the controller [Figure 8] An example of an emotion change database [Figure 9] FIG. 1 is a diagram showing an example of an analytical information database. [Figure 10] An example of a database after statistical processing [Figure 11] An example of an analysis result map [Figure 12] FIG. 10 shows an example of an analysis result map with a pop-up display. [Figure 13] FIG. 10 is a diagram showing an example of the configuration of a server according to a second exemplary embodiment of the present invention. [Figure 14] Flowchart of the learning process executed by the controller [Figure 15] Flowchart of inference processing executed by the controller DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0015] First Embodiment [Analysis System] 1 is a diagram showing an example of the configuration of an analysis system SYS1 according to an embodiment of the present invention. The analysis system SYS1 is a system that identifies factors that may cause changes in the emotions of a subject (driver D1 of vehicle V1) as the subject moves along a route from a first point to a second point (for example, factors that cause changes in the subject's emotions regarding a linear series of stores along a main road).

[0016] As shown in FIG. 1, the analysis system SYS1 includes a drive recorder 1 and a server 2 (computer).

[0017] The drive recorder 1 is mounted on a vehicle V1. In this embodiment, the vehicle V1 also mounts an on-board camera C1, an on-board microphone M1, a GPS sensor SN2, a timing device TM1, and an airbag device AB1 in addition to the drive recorder 1. The drive recorder 1, the driving assistance device 3, the G sensor SN1, the on-board camera C1, the on-board microphone M1, the ultrasonic sensor SN1, the GPS sensor SN2, the heart rate sensor SN3, and the timing device TM1 are connected to an in-vehicle communication network provided inside the vehicle V1.

[0018] The server 2 is located outside the vehicle V1 (for example, in an analysis service center building, etc.). The server 2 may be a physical server or a virtual server. The server 2 may be configured by one server or by multiple servers.

[0019] The drive recorder 1 and the server 2 communicate with each other via a network NT1. The drive recorder 1 is an information acquisition terminal that collects data based on commands from the server 2 (such as a command to transmit specified data within a specified area) and transmits the data to the server 2 via the network NT1. In this embodiment, the drive recorder 1 collects the output of the heart rate sensor SN3 as information related to the subject's emotions and transmits it to the server 2, and collects the output of the GPS sensor SN2 as the subject's location information and transmits it to the server 2. In addition to the information related to the subject's emotions and the subject's location information, the drive recorder 1 also transmits the output (date and time data) of the timing device TM1 and the subject's attributes (gender, age, etc.) registered in the drive recorder 1 to the server 2 via the network NT1.

[0020] The server 2 is an analysis device that uses data acquired from the drive recorder 1 to detect the subject's emotions according to the subject's movements, detects emotion change locations based on the detected emotions, and identifies environmental information corresponding to the emotion change locations as emotion change factor information. While only one vehicle V1 is shown in FIG. 1 , it is desirable to have a large number of vehicles V1 equipped with drive recorders 1 that can communicate with the server 2. If there are multiple vehicles V1 equipped with drive recorders 1 that can communicate with the server 2, multiple subjects can be used, and statistical processing of data groups consisting of emotion change locations and corresponding environmental information for multiple subjects can be performed to identify emotion change factor information at the emotion change locations, thereby reducing the impact of personal preferences on emotion change factor information. In this embodiment, multiple subjects are used.

[0021] The vehicle-mounted camera C1 is configured with one or more cameras that capture images of the surroundings of the vehicle V1. The vehicle-mounted camera C1 captures images of the surroundings of the vehicle V1 and generates vehicle-mounted camera images.

[0022] The in-vehicle microphone M1 is composed of one or more microphones that pick up sounds around the vehicle V1 and convert them into audio signals, which are electrical signals.

[0023] The ultrasonic sensor (ultrasonic sonar) SN1 detects objects (vehicles, people, etc.) present around the vehicle V1, and generates and outputs object detection information.

[0024] The GPS sensor SN2 receives signals from multiple GPS (Global Positioning System) satellites and generates and outputs vehicle position information based on the received signals. The vehicle position information generated by the GPS sensor SN2 represents the current position of the vehicle V1 by longitude and latitude.

[0025] The heart rate sensor SN3 detects the heart rate of the subject. The heart rate sensor S12 is an optical heart rate (pulse) sensor that is disposed on the steering wheel of the vehicle V1, for example.

[0026] The timing device TM1 generates and outputs a signal corresponding to the current date and time. The timing device TM1 has, for example, an internal battery, so that it can operate without an external power supply and keep accurate time and date. Note that the GPS sensor SN2 has a timing function based on signals from GPS satellites, so the GPS sensor SN2 can also be used as the timing device TM1.

[0027] [server] 2 is a diagram showing an example of the configuration of the server 2 according to the first exemplary embodiment of the present invention. The server 2 includes a communication unit 21, a storage unit 22, and a controller 23.

[0028] The communication unit 21 transmits and receives any signal to and from the drive recorder 1. Note that the controller 23 can transmit and receive any information to and from various devices installed in the vehicle V1 using the communication unit 21 and the drive recorder 1, but in the following, descriptions of the communication unit 21 and the drive recorder 1 may be omitted.

[0029] The storage unit 22 is configured to include non-volatile memory such as ROM (Read Only Memory) or flash memory, and volatile memory such as RAM (Random Access Memory). The storage unit 12 stores drawing data 221, actual measurement data 222, psychological plane data 223, a static environmental change database 224, a target dynamic environmental change type table 225, a dynamic environmental change database 226, an emotion change database 227, a program PG1, and map data MP1.

[0030] The drawing data 221 is data generated based on existing data such as survey maps created by prior surveying and design drawings created when designing roads, etc. The drawing data 221 is divided into predetermined ranges called meshes, and each mesh is filed separately. The drawing data for the area to be used for processing, etc. is selected from the drawing data 221 and used. The drawing data 221 includes, for example, road width, number of lanes, presence or absence of sidewalks, presence or absence of guardrails, etc.

[0031] The measured data 222 is data to supplement the drawing data 221, and is data generated based on data such as images, positions, and distances collected by a traveling vehicle. The measured data 222 includes not only newly generated data but also data whose contents have been updated due to construction work, etc. It should be noted that data generated based on data such as images, positions, and distances collected by a vehicle V1 driven by a subject can also be used as the measured data 222.

[0032] The psychological plane data 223 is data forming the psychological plane PP shown in FIG. 3, which estimates emotions from two types of emotional index values ​​based on biosignals, etc. The psychological plane PP shown in FIG. 3 is the psychological plane disclosed, for example, in Japanese Patent Application Laid-Open No. 2019-63324, and includes a data group associating two types of emotional index values ​​(the average value of the heartbeat interval, i.e., the R-R interval (hereinafter, the average R-R interval) and the standard deviation of the amplitude of the low-frequency component of the R-R interval (hereinafter, the low-frequency (LF) amplitude standard deviation)) with corresponding emotional information (emotion type). According to the psychological plane PP shown in FIG. 3, positive emotions (happiness, surprise) and negative emotions (sadness, worry) can be distinguished and determined based on the average R-R interval and the LF amplitude standard deviation. Emotions other than positive emotions (happiness, surprise) and negative emotions (sadness, worry) are defined as neutral emotions. Note that an emotion estimation method that can separate positive emotions (e.g., joy, surprise, etc.) from negative emotions (e.g., sadness, worry, etc.) other than the emotion estimation method using the psychological plane PP shown in Fig. 3 may also be used. When an emotion estimation method other than the emotion estimation method using the psychological plane PP shown in Fig. 3 is used, a sensor, a data table, an AI model, etc. that corresponds to the emotion estimation method are used.

[0033] The static environmental change database 224 is a database in which positions (locations where static environmental changes occur) are linked to static environmental change types, as shown in FIG. 4, for example. The static environmental change database 224 is created by the controller 23 based on the drawing data 221 and the actual measurement data 222. Note that in FIG. 4, the direction of change is not distinguished among the static environmental change types, but for example, "road width changes by more than a threshold value" may be subdivided into "road width widens by more than a threshold value" and "road width narrows by more than a threshold value." Furthermore, the static environmental change database 224 may be stored in the drive recorder 1 instead of the server 2.

[0034] 5, the target dynamic environmental change type table 225 is a table that defines the type of dynamic environmental change that is to be registered as data in the dynamic environmental change database 226. The target dynamic environmental change type table 225 may be stored in the drive recorder 1 instead of the server 2. When the target dynamic environmental change type table 225 is stored in the server 2, if the controller 23 detects a dynamic environmental change that is to be registered as data in the dynamic environmental change database 226 from data sent from the drive recorder 1 (datasets such as captured image data, audio data, acceleration information of the vehicle V1, date and time information, etc.), the content of the data is added to the dynamic environmental change database 226, and the dynamic environmental change database 226 is updated. When the target dynamic environmental change type table 225 is stored in the drive recorder 1, if the controller of the drive recorder 1 detects a dynamic environmental change that is to be registered in the dynamic environmental change database 226 from the data collected by the drive recorder 1 (datasets such as captured image data, audio data, acceleration information of the vehicle V1, date and time information, etc.), the contents of the data are added to the dynamic environmental change database 226 and the dynamic environmental change database 226 is updated.

[0035] 6, the dynamic environmental change database 226 is a database in which a position (a central position of occurrence of a dynamic environmental change), a dynamic environmental change type, a dynamic environmental change content, a date and time (a central date and time of occurrence of a dynamic environmental change), and captured image data acquired by the drive recorder 1 when the dynamic environmental change occurred are linked together. Note that the dynamic environmental change database 226 may be stored in the drive recorder 1 instead of the server 2.

[0036] The emotion change database 227 will be described later.

[0037] The program PG1 is a program to be executed by the controller 23.

[0038] The map data MP1 includes road data used to represent road shapes. The map data is divided into predetermined areas called meshes (for example, 100m square), and each mesh is stored as a separate file. The map data for the area to be used for processing, etc. is selected from the map data MP1 and used.

[0039] The controller 23 comprehensively controls the operation of each component in the server 2. The controller 23 includes, as hardware resources, an arithmetic processing unit including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The controller 23 includes function blocks 231 to 236.

[0040] The controller 23 is a program execution device (computer) capable of executing any program. The controller 23 executes the program PG1, thereby realizing each function of the controller 23 (including each function of the functional blocks 231 to 236). All of the operations of the controller 23 described in this embodiment may be operations that are realized by the controller 23 executing the program PG1. The program PG1 may be composed of multiple programs.

[0041] The following describes each functional block of the controller 23. Functional blocks 231, 232, 233, 234, 235, and 236 are a static environment information determination unit, a dynamic environment information determination unit, an emotion estimation unit, a position detection unit, an emotion change factor information determination unit, and an analysis result map generation unit, respectively.

[0042] The static environment information identification unit 231 identifies static environment information from the first location to the second location based on the static environment change database 224. The static environment information is information in which the temporal change in the content related to the environment is smaller than a threshold value.

[0043] The dynamic environment information identification unit 232 identifies dynamic environment information from the first location to the second location based on the dynamic environment change database 226. The dynamic environment information is information in which the temporal fluctuation of the environmental content in the time period around the subject's movement is greater than a threshold value.

[0044] In this embodiment, the static environment information identification unit 231 and the dynamic environment information identification unit 232 are separate functional blocks, but the static environment information identification unit 231 and the dynamic environment information identification unit 232 may be integrated into an environment information identification unit that identifies both static environment information and dynamic environment information. When the static environment information identification unit 231 and the dynamic environment information identification unit 232 are integrated, the static environment change database 224 and the dynamic environment change database 226 are also integrated.

[0045] The emotion estimation unit 233 estimates (detects) the emotion of the subject based on the output of the heartbeat sensor SN3 and the psychological plane data 223, and identifies changes in the subject's emotion. In this embodiment, the emotion estimation unit 233 identifies both a transition from a positive emotion or a neutral emotion to a negative emotion, and a transition from a negative emotion or a neutral emotion to a positive emotion, as changes in the subject's emotion. Note that the emotion estimation unit 233 may identify only one of a transition from a positive emotion or a neutral emotion to a negative emotion, and a transition from a negative emotion or a neutral emotion to a positive emotion, as changes in the subject's emotion. When detecting only negative factors when the subject reaches a singular point of emotion change, the emotion estimation unit 233 identifies only a transition from a positive emotion or a neutral emotion to a negative emotion as changes in the subject's emotion. In addition, when detecting only positive factors when the subject visits a singular point of emotional change, the emotion estimation unit 233 identifies only a transition from a negative emotion or a neutral emotion to a positive emotion as a change in the subject's emotion.

[0046] The position detection unit 234 detects the position of the subject based on the output of the GPS sensor SN2. The position detection unit 234 detects the date and time and the attributes of the subject in addition to the position of the subject. The position detection unit 234 detects the date and time at the time of detecting the position of the subject based on the output of the timing device TM1. The position detection unit 234 detects the attributes of the subject at the time of detecting the position of the subject based on the information registered in the drive recorder 1.

[0047] The timing at which the dynamic environmental information identification unit 232, the emotion estimation unit 233, and the position detection unit 234 acquire information is synchronized, and the timing at which the dynamic environmental information identification unit 232, the emotion estimation unit 233, and the position detection unit 234 output information is also synchronized. Examples of synchronization methods include a method in which each unit generates a data record at sampling position intervals (e.g., every 1 m) or sampling time intervals (e.g., every 1 second), each data record is stored, and a data record in which an emotion change has occurred is extracted in post-processing, and a method in which each unit generates a data record when an emotion change has occurred, and each data record is stored.

[0048] The emotion change factor information identification unit 235 identifies environmental information corresponding to the location where the subject's emotion changed as information about factors that changed the subject's emotion. By setting the subject's movement as a movement to visit a town, facility, etc., it is possible to estimate the identified factors that changed the subject's emotion as negative or positive factors for visiting the town, facility, etc.

[0049] The analysis result map generating section 236 generates an analysis result map showing the locations of the factors that caused changes in the subject's emotions identified by the emotion change factor information identifying section 235.

[0050] [Analysis process flowchart] Fig. 7 is a flowchart of the analysis process executed by the controller 23. The analysis process is realized by the controller 13 executing the above-mentioned program PG1. The analysis process shown in Fig. 4 starts when the server 2 receives an instruction from the operator to start the analysis process.

[0051] First, in step S10, the controller 23 sets a first location and a second location. For example, if it is desired to estimate the negative and positive factors associated with a visit from a certain location to a certain facility, the certain location may be set as the first location, and the entrance to the certain facility may be set as the second location. The controller 23 sets the first and second locations according to the operation of an operator on an operating device such as a keyboard or mouse connected to the server. After step S10, the process proceeds to step S20.

[0052] In step S20, the static environment information identification unit 231 identifies static environment changes (static environment information) from the first location to the second location based on the static environment change database 224. After step S20, the process proceeds to step S30.

[0053] In step S30, the controller 23 requests the drive recorder 1 to collect and provide information about the subject's journey from the first location to the second location. After step S30, the process proceeds to step S40.

[0054] In step S40, the controller 23 updates the dynamic environmental change database 226 based on the data sent from the drive recorder 1. After step S40, the process proceeds to step S50.

[0055] In step S50, the dynamic environment information identification unit 232 identifies dynamic environment changes (dynamic environment information) occurring between the first location and the second location, based on the dynamic environment change database 226. After step S50, the process proceeds to step S60.

[0056] In step S60, the emotion estimation unit 233 estimates (detects) the emotion of the subject based on the output of the heartbeat sensor SN3 and the psychological plane data 223, and if there is a change in the emotion of the subject, updates the emotion change database 227. After step S60, the process proceeds to step S70.

[0057] 8, the emotion change database 227 is a database in which a position (for example, a central position where an emotion change occurs), a date and time (for example, a central date and time where an emotion change occurs), the content of the emotion change, and the captured image data acquired by the drive recorder 1 when the emotion change occurred are linked together. Note that the emotion change database 227 may be stored in the drive recorder 1 instead of the server 2.

[0058] The content of emotional changes can be categorized into, for example, a transition from a positive or neutral emotion to a negative emotion (first major category) and a transition from a negative or neutral emotion to a positive emotion (second major category). Subcategories within the first major category include a transition from joy to sadness, a transition from joy to worry, etc., and subcategories within the second major category include a transition from neutral to joy, a transition from neutral to surprise, etc. Linking the date and time (e.g., the central date and time of the occurrence of emotional changes) with the content of emotional changes in the emotional change database 227 makes it possible to identify factors that cause emotional changes for each time period, date, season, etc., and thereby identify factors that cause emotional changes in subjects that contribute to effective urban development, facility development, etc. for target time periods, dates, seasons, etc. It is also effective to add the attributes of the subjects to the emotional change database 227. This allows the attributes of the subjects to be linked to the content of emotional changes, making it possible to identify the factors that cause emotional changes for each attribute of the subject, and to identify the factors that cause emotional changes in subjects that will contribute to effective urban development, facility development, etc. for subjects with target attributes.

[0059] In step S70, controller 23 determines whether or not the subject has arrived at the second location based on the detection result of position detection unit 234. If the subject has not arrived at the second location, the process returns to step S30, and if the subject has arrived at the second location, the process proceeds to step S80.

[0060] In step S80, the emotion change factor information identification unit 235 of the controller 23 creates the analytical information database 228 using the data from the static environmental change database 224, the dynamic environmental change database 226, and the emotion change database 227. The analytical information database 228 can identify the static environmental changes and dynamic environmental changes that caused changes in the subject's emotions. After step S80, the process proceeds to step S90.

[0061] 9, the analytical information database 228 is a database in which a location, a date and time, the content of an emotional change, the content of a dynamic environmental change, the content of a static environmental change, and captured image data acquired by the drive recorder 1 when an emotional change or an environmental change occurs are linked together. The analytical information database 228 includes a data group in which the static environmental change database 224 and the emotional change database 227 match in location (including not only a perfect match but also cases with a certain range of error), and also includes a data group in which the dynamic environmental change database 226 and the emotional change database 227 match in location and date and time (including not only a perfect match but also cases with a certain range of error). Furthermore, the analytical information database 228 also includes a data group in the dynamic environmental change database 226 that does not match the emotional change database 227 in at least one of the location and date and time (including not only a perfect match but also cases with a certain range of error). A data group in the dynamic environmental change database 226 that does not match (including not only a perfect match but also cases with a certain range of error) at least one of the location and date and time with the emotion change database 227 can be used as a dynamic environmental change that does not have much effect on emotions.

[0062] In step S90, the controller 23 determines whether the number of subjects has reached a target number. After step S90, if the number of subjects has not reached the target number, the process returns to step S30, and if the number of subjects has reached the target number, the process proceeds to step S100. Note that the controller 23 may set a target number for each attribute of the subjects and determine that the number of subjects has reached the target number when the number of subjects for all attributes has reached the target number. In this case, even if emotion change factors are identified for each attribute of the subjects, the influence of personal preferences on emotion change factors can be suppressed.

[0063] In step S100, the controller 23 statistically processes the data group included in the analytical information database 228 to create a statistically processed database 229. For example, the statistically processed database 229 is created by selecting a data group at positions where emotion changes have been detected a threshold number of times or more in the analytical information database 228. After step S100, the process proceeds to step S110.

[0064] The post-statistical processing database 229 is a database in which, as shown in FIG. 10, the location, the type of static environmental change, the subject's attributes, the date and time, the content of the emotional change, the content of the dynamic environmental change, and the captured image data acquired by the drive recorder 1 when the emotional change or the environmental change occurred are linked together.

[0065] In step S110, the analysis result map generation unit 236 generates an analysis result map MP2 from the post-statistical processing database 229 and the map data MP1. When step S110 is completed, the analysis process ends. The analysis result map MP2 is displayed on, for example, a display device connected to the server 2.

[0066] Fig. 11 is a diagram showing an example of an analysis result map MP2. The analysis result map MP2 shown in Fig. 11 is an image in which icons N1 and N2 indicating positions where factors that caused the subject's emotion to change to negative occurred and icons P1 and P2 indicating positions where factors that caused the subject's emotion to change to positive occurred are superimposed on a map image MP generated from map data MP1.

[0067] When an operator selects an icon on the analysis result map MP2, for example, by using a mouse connected to the server and clicking the icon while the cursor is on the icon, the controller 23 adds detailed information about the icon to the analysis result map MP2 and displays it.

[0068] Fig. 12 is a diagram showing an example of an analysis result map MP2 having a pop-up display POP. The analysis result map MP2 shown in Fig. 12 is an image that displays detailed information about an icon N1 in the form of a pop-up display POP. Possible detailed information about the icon N1 includes, for example, the number of subjects whose emotions changed to negative (more specifically, the number of subjects on weekdays and weekends), the details of the factors that caused the change to negative emotions, and the percentage of subjects who changed to positive emotions by type.

[0069] In addition, by using date and time information and the subject's attributes, the analysis result map MP2 may be an analysis result map specialized for summer, an analysis result map specialized for daytime, an analysis result map specialized for a certain attribute of the subject, etc.

[0070] Second Embodiment The second exemplary embodiment of the present invention is based on the first embodiment described above. Regarding matters not specifically described in the second embodiment, the description of the first embodiment also applies to the second embodiment unless there is a contradiction. However, when interpreting the description of the second embodiment, the description of the second embodiment may take precedence over any contradiction between the first and second embodiments.

[0071] 13 is a diagram showing an example configuration of the server 2 according to the second exemplary embodiment of the present invention. The controller 23 includes a learning data generation unit 237, a learning unit 28, and an AI (Artificial Intelligence) model execution unit 239 instead of the emotion change factor information identification unit 235.

[0072] The storage unit 22 stores the AI ​​model MD1. The AI ​​model MD1 is an AI model that estimates changes in the subject's emotions from environmental information corresponding to changes in the environment corresponding to the position of the subject.

[0073] [Flowchart of learning process and inference process] Fig. 14 is a flowchart of the learning process executed by the controller 23. The learning process is realized by the execution of the above-mentioned program PG1 by the controller 13. The learning process shown in Fig. 14 starts when the server 2 receives an instruction from the operator to start the learning process.

[0074] The flowchart shown in Fig. 14 is obtained by replacing step S60 of the flowchart shown in Fig. 7 with step S61, removing step S80 from the flowchart shown in Fig. 7, and replacing steps S100 and S110 of the flowchart shown in Fig. 7 with step S91. Detailed description of the same processes as in Fig. 7 will be omitted.

[0075] In step S61, the training data generation unit 237 uses the static environmental change database 224, the dynamic environmental change database 226, and the emotion change database 227 to generate (update) supervised training data in which environmental information corresponding to the environmental change corresponding to the subject's position is used as an input value and the emotion change corresponding to the subject's position is used as a correct value. The emotion change corresponding to the subject's position that is the correct value includes emotions that do not change (emotions not included in the emotion change database 227). In other words, the supervised training data is generated using data from the analysis information database 228, such as that shown in FIG. 9, for example. The server 2 performs the processing of step S61 and is therefore a training data generation device that generates supervised training data.

[0076] The input values ​​of the supervised learning data (environmental information corresponding to changes in the environment corresponding to the position of the subject) may be the static environmental change types in the static environmental change database 224 and the dynamic environmental change contents in the dynamic environmental change database 226, or may be captured image data in the emotion change database 227, or may be both the static environmental change types in the static environmental change database 224 and the dynamic environmental change contents in the dynamic environmental change database 226 and the captured image data in the emotion change database 227. Note that when the input values ​​of the supervised learning data (environmental information corresponding to changes in the environment corresponding to the position of the subject) are captured image data in the emotion change database 227, the server 2 does not need to store the drawing data 221, the measured data 222, the static environmental change database 224, the target dynamic environmental change type table 225, and the dynamic environmental change database 226.

[0077] In step S91, the learning unit 238 provides the AI ​​model MD1, which is a neural network, with the supervised learning data generated by the learning data generation unit 237. The AI ​​model MD1 infers emotional changes corresponding to the subject's location from environmental changes corresponding to the subject's location. The learning unit 238 evaluates the error between the emotional changes corresponding to the subject's location inferred by the AI ​​model MD1 and the emotional changes corresponding to the subject's location provided as ground truth data. The learning unit 238 then performs learning of the AI ​​model MD1 to reduce the error using a learning algorithm such as backpropagation. The learning here is supervised machine learning, and the parameters (weights, etc.) of the AI ​​model MD1 are adjusted during the learning. The learning ends when a predetermined learning termination condition is met, such as when the error converges to a sufficiently small value. The AI ​​model MD1 after learning (trained AI model MD1) is stored in the storage unit 22.

[0078] FIG. 15 is a flowchart of the inference processing executed by the controller 23. This flowchart is a processing flowchart used when generating data for analysis of collected environmental information, for example, when generating the analytical information database 228. Note that the processing of the flowchart can also be executed in such a way that data for analysis is generated while collecting environmental information with a drive recorder while the vehicle is traveling. The inference processing is realized by the controller 13 executing the above-mentioned program PG1. The inference processing shown in FIG. 15 starts when the server 2 receives an instruction from an operator to start the inference processing. Steps S100 and S110 of the flowchart shown in FIG. 15 are the same as steps S100 and S110 of the flowchart shown in FIG. 7.

[0079] In step S92, the AI ​​model execution unit 239 inputs environmental information corresponding to the environmental change determined as the analysis target in accordance with the operator's operation on an operating device such as a keyboard or mouse connected to the server into the AI ​​model MD1 (trained AI model MD1), and proceeds to step S93. In step S93, the AI ​​model execution unit 239 creates (updates) the analytical information database 228 using the output (estimated data regarding the subject's emotional change) of the AI ​​model MD1 (trained AI model MD1) and the corresponding input (environmental information). In other words, if there is a change in the emotion estimated by the AI ​​model MD1, the environmental change input to the AI ​​model MD1 (trained AI model MD1) can be identified (estimated) as the emotion change factor. After step S93, proceed to step S100.

[0080] In step S100, the controller 23 statistically processes the data group included in the analytical information database 228 to create a statistically processed database 229. For example, the statistically processed database 229 is created by selecting a data group at positions where emotion changes have been detected a threshold number of times or more in the analytical information database 228. After step S100, the process proceeds to step S110.

[0081] In step S110, the analysis result map generation unit 236 generates an analysis result map MP2 from the post-statistical processing database 229 and the map data MP1. When step S110 is completed, the analysis process ends. The analysis result map MP2 is displayed on, for example, a display device connected to the server 2.

[0082] In this embodiment, the impact of environmental changes on human emotions can be simulated using the AI ​​model MD1 (trained AI model MD1). Therefore, various changes can be virtually made to the environment, and changes in human emotions in response to these various environmental changes can be confirmed. Therefore, in this embodiment, the scope of investigation can be expanded when identifying (estimating) factors that change the subject's emotions.

[0083] <Notes, etc.> Various technical features disclosed in the description of the present invention may be modified in various ways without departing from the spirit of the technical creation. Furthermore, multiple embodiments and modifications disclosed in the description of the present invention may be combined to the extent possible.

[0084] In the above-described embodiment, the subject is the driver D1, but the subject may be, for example, a person traveling on foot. When the subject is traveling on foot, a smart watch, a smartphone, or the like can be used as the data collection device. Note that when the subject is traveling on foot, the values ​​of the setting data, etc., of the target dynamic environment change type table 225 must be set to values ​​suitable for walking.

[0085] In the above-described embodiment, a visit to one facility is assumed, but multiple sets of first and second locations may be set to assume visits to multiple facilities of the same type (e.g., library, community center, etc.), tendency (e.g., business owner, etc.).

[0086] In the above-described embodiment and modified example, information on factors that cause changes in the subject's emotions on the route from the first point to the second point is identified, but information on factors that cause changes in the subject's emotions in an area may also be identified. When information on factors that cause changes in the subject's emotions in an area is identified, data (data records, table data) similar to those in the above-described embodiment and modified example is generated for each road in the area (each so-called link (road section between intersections)), and processing similar to that in the above-described embodiment and modified example is performed. Note that when information on factors that cause changes in the subject's emotions in an area is identified, route setting becomes area setting (data collection / analysis for movement within the area).

[0087] In the above-described embodiment and modified examples, the controller 23 of the server 2 is configured to include the emotion deduction unit 233 and the position detection unit 234. However, the controller 23 of the server 2 may not include the emotion deduction unit 233 and the position detection unit 234, and a controller provided in the drive recorder 1 may include functional units similar to the emotion deduction unit 233 and the position detection unit 234. [Explanation of symbols]

[0088] 1. Drive recorder 2. Server 21. Communications Department 22...Storage section 23. Controller C1...In-vehicle camera D1... driver M1...In-car microphone NT1 Network SN1 Ultrasonic Sonar SN2 GPS Sensor SN3 Heart Rate Sensor SYS1: Analysis System TM1...Timekeeping device V1...Vehicle

Claims

1. Detecting the subject's emotions according to the subject's movements; Detecting an emotion change position where the emotion has changed based on the detected emotion; An analysis device that identifies environmental information corresponding to the emotion change position as emotion change factor information.

2. 2. The analysis device according to claim 1, wherein the analysis device statistically processes a group of data consisting of the emotion change positions and corresponding environmental information for a plurality of subjects, and identifies the emotion change factor information at the emotion change positions.

3. The analysis device according to claim 1 , wherein the environmental information includes static environmental information whose content varies over time less than a threshold value.

4. The analysis device according to claim 1 or 3, wherein the environmental information includes dynamic environmental information whose content varies over time in a time period around the subject's movement more than a threshold value.

5. The analysis device according to claim 1 , wherein the change in emotion includes at least one of a transition from a positive emotion or a neutral emotion to a negative emotion, and a transition from a negative emotion or a neutral emotion to a positive emotion.

6. Detecting the subject's emotions according to the subject's movements; Detecting an emotion change position where the emotion has changed based on the detected emotion; and identifying environmental information corresponding to the emotion change position as emotion change factor information.

7. Detecting emotions of the subject according to the subject's movements; Detecting an emotion change position where the emotion has changed based on the detected emotion; identifying environmental information corresponding to the emotion change position as emotion change factor information; An analysis program that causes a computer to execute the above.

8. An analysis system including an information acquisition terminal and a server device that collects and analyzes information acquired by the information acquisition terminal, The information acquisition terminal Detecting the subject's emotions according to the subject's movements; Detecting an emotion change position where the emotion has changed based on the detected emotion; transmitting information on the emotion change position and emotion change information at the emotion change position to the server; The server device acquiring environmental information corresponding to the emotion change position; identifying the environmental information corresponding to the emotion change position as emotion change factor information at the emotion change position; Analysis system.

9. The information acquisition terminal acquiring a surrounding photographed image at the emotion change position; The surrounding captured image is transmitted to the server. The server device The surrounding captured image is included in the environmental information corresponding to the emotion change position. The analysis system according to claim 8 .

10. The analysis system according to claim 8 or 9, wherein the information acquisition terminal is an in-vehicle device.

11. Obtain environmental information at the target location, inputting the acquired environmental information into an estimation model that estimates the emotions felt by a person who has reached the target point based on the environmental information; An analysis system that identifies a factor that causes the emotion output by the estimation model as the environmental information at the target location, The estimation model is Detecting the subject's emotions according to the subject's movements; Detecting an emotion change position where the emotion has changed based on the detected emotion; Detecting environmental information corresponding to the emotion change position; generating learning data using the detected environmental information as input data and the detected emotion as correct answer data; The artificial intelligence model is generated by learning the generated learning data. Analysis system.

12. A method for generating learning data for an artificial intelligence model that estimates emotions of a moving person, comprising: Detect emotions according to the subject's movements, Detecting an emotion change position where the emotion has changed based on the detected emotion; Detecting environmental information corresponding to the emotion change position; generating learning data using the detected environmental information as input data and the detected emotion as correct answer data; Training data generation method.

Images