Information processing device, information processing method, and program

Abstract

To provide information more appropriately aligned with needs of a user.SOLUTION: An information processing device includes a processing unit that acquires a search instruction related to an area entered by a user, acquires reference information that belongs to an area that includes a position expressed by position information from a database that aggregates reference information including the position information of a person and biological information of the person at the position expressed by the position information when the search instruction is acquired, derives a state index value related to the physicomental state of the user predicted in the area on the basis of the acquired reference information, and generates area information related to the area on the basis of the derived state index value.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there is a technology in which, in response to a user's operation to specify a desired area, images, text, etc. that have been previously associated with the area and stored are displayed on a display unit as tourist information along with a map of the area (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-159820 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the information that can be obtained from images and text is limited, and it is not possible to obtain information such as, for example, what state of mind and body the user will be in when actually visiting the area (for example, how relaxed or excited the user will be). For this reason, the above-mentioned conventional technology has a problem in that it is not always possible to provide information that meets the user's needs.

[0005] An object of the present invention is to provide information that more appropriately meets the needs of users. [Means for solving the problem]

[0006] In order to solve the above problems, the information processing device according to the present invention comprises: Obtaining search instructions related to a region entered by a user; When the search instruction is received, the method obtains reference information, in which the location represented by the location information belongs to a certain area, from a database in which reference information including location information of a person and biometric information of the person when the person was at the location represented by the location information is accumulated; deriving a state index value related to a predicted mental and physical state of the user in the certain area based on the acquired reference information; generating area information for the certain area based on the derived condition index value; A processing unit is provided. Effect of the Invention

[0007] According to the present invention, it is possible to provide information that more appropriately meets the needs of a user. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information providing system. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of a terminal device. [Diagram 3] FIG. 2 is a block diagram showing a functional configuration of a server. [Figure 4] FIG. 13 is a diagram showing a display screen of a display unit on which a face guide is displayed. [Diagram 5] FIG. 11 is a diagram showing an example of a pulse waveform detected from video data. [Figure 6] An example of the results of frequency analysis of a pulse waveform is shown below. [Figure 7] FIG. 2 is a diagram showing an example of biometric information. [Figure 8] FIG. 11 is a diagram showing an example of the contents of a reference information DB. [Figure 9] 10 is a flowchart showing a control procedure for pulse wave analysis processing in the terminal device. [Figure 10] 10 is a flowchart showing a control procedure for pulse wave analysis processing in a server. [Figure 11] 13 is a flowchart showing a control procedure for video analysis processing. [Figure 12] FIG. 2 is a diagram showing a top screen. [Figure 13] FIG. 11 is a diagram showing an example of reference information extracted based on a search condition. [Figure 14] 13 is a diagram showing evaluation criteria for excitement level and a display mode of the evaluation results. FIG. [Figure 15] 11A and 11B are diagrams showing evaluation criteria for a relaxation degree and a display mode of the evaluation results. [Figure 16] FIG. 13 is a diagram showing the evaluation criteria for the degree of blood circulation promotion and the display mode of the evaluation results. [Figure 17] FIG. 1 is a diagram showing an example of a map. [Figure 18] FIG. 13 is a diagram showing an example of the contents of a candidate area table. [Figure 19] FIG. 13 is a diagram showing an example of a list. [Figure 20] 10 is a flowchart showing a control procedure for area information display processing in the terminal device. [Figure 21] 13 is a flowchart showing a control procedure for area information display processing in the server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] (Configuration of the information provision system) FIG. 1 is a diagram showing a configuration of an information providing system 1. As shown in FIG. The information providing system 1 (information processing system) includes a plurality of terminal devices 10 and a server 20 (information processing device). The information providing system 1 is used by a plurality of users. Each user uses one terminal device 10. The plurality of terminal devices 10 are connected to the server 20 via a network N, and are capable of transmitting and receiving data to and from the server 20. The network N is, for example, the Internet, but is not limited to this. The communication path between the terminal devices 10 and the server 20 may include a wireless communication path. In response to a request from the terminal device 10, the information providing system 1 provides information related to various areas (areas) such as tourist spots to the user of the terminal device 10.

[0011] The terminal device 10 is a device that is mainly carried and used by a user, such as a smartphone. An application program (hereinafter, referred to as "application 131" (see FIG. 2)) for receiving an information provision service by the information provision system 1 is installed in the terminal device 10. Note that the terminal device 10 is not limited to a smartphone, and may be a tablet terminal, a notebook PC, or a device that is worn on the user's body, such as a smart watch.

[0012] The server 20 updates a reference information DB232 (database) (see Figs. 3 and 8) that accumulates reference information 40 (location bioinformation) including user (person) location information 42 and the user's bioinformation 45 when the user was at the location indicated by the location information 42, based on the information transmitted from each terminal device 10. In addition, the server 20 generates area information including a map 81 (see Fig. 17) or a list 82 (see Fig. 19), etc., based on the reference information DB232 in response to a request from the terminal device 10, and displays it on the terminal device 10.

[0013] FIG. 2 is a block diagram showing the functional configuration of the terminal device 10. As shown in FIG. The terminal device 10 includes a CPU 11 (Central Processing Unit), a RAM 12 (Random Access Memory), a storage unit 13, a display unit 14 (output unit), an operation unit 15, an image capturing unit 16, a position information acquiring unit 17, a communication unit 18, and a bus 19. The individual units of the terminal device 10 are connected to each other via the bus 19.

[0014] The CPU 11 is a processor that reads and executes various programs such as the application 131 stored in the storage unit 13, and controls the operation of each unit of the terminal device 10 by performing various arithmetic processing. The terminal device 10 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 11 of the present embodiment may be executed by the multiple processors. In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel.

[0015] The RAM 12 provides a working memory space for the CPU 11 and stores temporary data. A reference data processing flag 121 used in the pulse wave analysis process described later is stored in the RAM 12. The reference data processing flag 121 may be stored in the storage unit 13.

[0016] The storage unit 13 is a non-transitory recording medium readable by the CPU 11 as a computer, and stores programs such as the application 131 and various data. The storage unit 13 includes a non-volatile memory such as a flash memory. The programs are stored in the storage unit 13 in the form of computer-readable program codes. The data stored in the storage unit 13 includes reference data 452 used in pulse wave analysis, which will be described later.

[0017] Display unit 14 displays various information such as an operation screen of application 131 under the control of CPU 11. As display unit 14, for example, a liquid crystal display device that performs display using a dot matrix method can be used, but is not limited to this.

[0018] The operation unit 15 accepts an input operation by the user and outputs an input signal corresponding to the input operation to the CPU 11. The operation unit 15 includes a touch panel provided overlaid on the display screen of the display unit 14, and detects contact of the user's finger or the like as an input operation by this touch panel. The operation unit 15 may also include hardware buttons together with or instead of the touch panel, and may be capable of accepting an input operation by this hardware button.

[0019] The photographing unit 16 includes a camera, and generates photographed data by photographing a moving image using the camera. The photographing unit 16 of this embodiment includes an in-camera capable of photographing the display unit 14 side of the terminal device 10, and an out-camera capable of photographing the side opposite to the display unit 14 side (rear side). The in-camera can photograph the face of the user while the user is viewing the display unit 14. The photographing unit 16 splits the light reflected from the subject into light of red, green, and blue wavelength bands using a color filter provided on the imaging element of the camera, and generates image data having pixel values ​​according to the intensity of the light of each color. The generated image data of the moving image is stored in the storage unit 13.

[0020] The position information acquisition unit 17 receives and decodes radio waves transmitted from a positioning satellite of a Global Positioning Satellite System (GNSS) such as a Global Positioning System (GPS) to calculate the current position. The position information acquisition unit 17 calculates the current position under the control of the CPU 11 and outputs the result to the CPU 11. Note that the method of calculating the current position by the position information acquisition unit 17 is not limited to a method using radio waves transmitted from a positioning satellite, and may be, for example, a method of identifying a positional relationship with a beacon based on a signal from a beacon installed at a predetermined position.

[0021] The communication unit 18 performs a communication operation in accordance with a predetermined communication standard. The communication unit 18 transmits and receives data to and from the server 20 via the network N through this communication operation.

[0022] FIG. 3 is a block diagram showing the functional configuration of the server 20. As shown in FIG. The server 20 includes a CPU 21 (processing unit, computer), a RAM 22, a storage unit 23, a communication unit 24, and a bus 25. The various units of the server 20 are connected to each other via the bus 25. The server 20 may further include an operation unit, a display unit, and the like that are used by an administrator of the server 20.

[0023] The CPU 21 is a processor that reads and executes the program 231 stored in the storage unit 23 and performs various arithmetic processing to control the operation of each unit of the server 20. The server 20 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 21 of this embodiment may be executed by the multiple processors. In this case, a "processing unit" is configured by the multiple processors. In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel.

[0024] The RAM 22 provides a working memory space for the CPU 21 and stores temporary data.

[0025] The storage unit 23 is a non-transitory recording medium readable by the CPU 21 as a computer, and stores the program 231 and various data. The storage unit 23 includes a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD). The program 231 is stored in the storage unit 23 in the form of a computer-readable program code. The data stored in the storage unit 23 includes the above-mentioned reference information DB 232 and candidate area table 233. Specific contents of the reference information DB 232 and candidate area table 233 will be described later.

[0026] The communication unit 24 performs a communication operation in accordance with a predetermined communication standard. The communication unit 24 transmits and receives data to and from the terminal device 10 via the network N through this communication operation.

[0027] (Overview of the operation of the information provision system) Next, an overview of the operation of the information providing system 1 will be described. The information providing method by the information providing system 1 includes the following database updating step and area information display step.

[0028] In the database update step, the reference information DB 232 is updated in the server 20 based on information transmitted from each of the multiple terminal devices 10 to the server 20. In detail, the terminal device 10 transmits date and time information 41, position information 42 of the terminal device 10 and the user, attribute information 43 of the user, environmental information 44 (see FIG. 8), and video data capturing the user's face to the server 20. The server 20 detects the user's pulse wave from the received video data, analyzes the pulse wave waveform, and generates biometric information 45 (see FIGS. 7 and 8) including various pulse wave index values ​​(biometric index values). The server 20 registers the reference information 40 (see FIG. 8) including the biometric information 45, date and time information 41, position information 42, attribute information 43, and environmental information 44 in the reference information DB 232. By generating and registering the reference information 40 at different times for each of the multiple terminal devices 10, the reference information DB 232 accumulates the reference information 40 of multiple users in various areas at various dates and times.

[0029] When a request for providing area information is transmitted from a certain terminal device 10 to the server 20, the area information display step is initiated. The server 20 extracts (acquires) reference information 40 corresponding to the search conditions included in the request from the reference information DB 232, and generates area information for an area that matches the search conditions based on the extracted reference information 40 and transmits it to the terminal device 10. The terminal device 10 displays the received area information on the display unit 14. As described above, the area information includes, for example, a map 81 or a list 82.

[0030] The area information provided by the information providing system 1 includes status information related to at least one of the user's psychological state and physical state (mental and physical state) predicted in the area. The status information related to the psychological state includes, for example, information such as "excitement level" (which can also be rephrased as "thrill level") indicating how excited and thrilled the user feels, or "relaxation level" indicating how relaxed the user feels. The status information related to the physical state includes, for example, information on "blood circulation promotion level" indicating how much blood circulation is promoted by entering a hot spring, etc. By displaying area information including such status information, the user can predict, for example, how relaxed or excited the user will be in the area he or she wants to go to.

[0031] The operations in the database update step and the area information display step will be described in detail below.

[0032] (Database update step) First, a method for generating biometric information 45 based on video data capturing a user's face in the database update step will be described. When a predetermined operation is performed by the user or when a predetermined start timing arrives while the application 131 is being executed on the terminal device 10, the terminal device 10 transitions to a standby state for capturing a moving image of the user's face. In the standby state, face guides G1 to G3 and the like for assisting in capturing a face image are displayed on the display unit 14.

[0033] FIG. 4 is a diagram showing the display screen of the display unit 14 on which face guides G1 to G3 are displayed. In the standby state, as shown in FIG. 4, face guides G1 to G3 and an image captured by the in-camera of the image capturing unit 16 (i.e., the user's face) are displayed superimposed on the display unit 14. By adjusting the position and orientation of the face so that the contour of the face approximately matches the face guide G1 and the right eye and left eye are within the face guides G2 and G3, respectively, during video capture, it becomes easier to capture a video suitable for face recognition and pulse wave detection, which will be described later. In the standby state, video capture is started when the user performs a predetermined capture start operation. The capture time is, for example, about one minute in total. The capture time does not necessarily have to be one continuous minute, and may include interruptions. 4, for example, a video of the face may be captured by the in-camera while various information display screens are displayed by the application 131. This allows the user to make effective use of the time spent capturing the video.

[0034] The captured video data is compressed using a predetermined compression method and then transmitted to the server 20. A suitable compression method is a method that compresses each frame image without compressing it in the time axis direction, such as the Motion JPEG method. If there is sufficient communication capacity and storage capacity, the uncompressed video data may be used for analysis. In addition, the terminal device 10 transmits date and time information 41, location information 42, attribute information 43, and environmental information 44 to the server 20 together with the video data.

[0035] When the video data and the like are received, the server 20 analyzes the video data. In analyzing the video data, a face is detected by a predetermined image recognition process, and a region of interest (ROI) for detecting a pulse wave is set inside the face. Fig. 4 shows an example of ROI1 set on the forehead, ROI2 set on the nose, ROI3 and ROI4 set on the left and right cheeks, and ROI5 set on the chin. Among these, a ROI (in this embodiment, ROI3 or ROI4) of a predetermined part is set.

[0036] In the video data, the luminance of the green component corresponding to the absorption wavelength band by hemoglobin contained in blood varies over time according to the increase or decrease in blood flow in the skin. More specifically, hemoglobin has an absorption characteristic having a peak in the green wavelength band of approximately 520 nm to 600 nm. Most of this wavelength band overlaps with the main transmission wavelength band (approximately 500 nm to 570 nm) of the green filter of the imaging element of the in-camera of the imaging unit 16. Therefore, when the blood flow in the blood vessels of the skin increases in response to pulsation, the amount of absorption of green light increases in response to the increase in blood flow, and the luminance value of the green pixels in the video data decreases. In addition, when the blood flow decreases, the amount of absorption of green light decreases in response to the decrease in blood flow, and the luminance value of the green pixels in the video data increases. Therefore, the pulse wave can be detected based on the change in the pixel value of the green pixels in the video data. More specifically, in each frame image of the video data, the average value of the luminance value of the green pixels in the ROI is derived, and the time change can be used as pulse wave waveform data.

[0037] FIG. 5 is a diagram showing an example of a pulse waveform detected from video data. As shown in FIG. 5, the pulse waveform is expressed as luminance value data that repeatedly increases and decreases. In this embodiment, in order to make it easier to intuitively grasp the increase in blood flow, the luminance value data is a converted luminance (converted luminance) that increases as the blood flow increases. More specifically, when the luminance detected by the imaging element is output as an 8-bit detected luminance value, the converted luminance value is obtained by subtracting the detected luminance value from the maximum luminance value of 255. In FIG. 5, the pulse waveform is depicted as a smooth curve, but the actual pulse waveform includes a number of different high-frequency components that are omitted in FIG. 5. FIG. 6 shows an example of the result of frequency analysis of the pulse waveform.

[0038] The characteristics of the pulse wave waveform can be expressed by a number of pulse wave indices (biological indices). The pulse wave indices include pulse wave amplitude (hereinafter also referred to as "PA"), baseline, heart rate (hereinafter also referred to as "HR"), LF, HF, LF / HF, CVRR, and total power (hereinafter also referred to as "TP"). Among these, PA and baseline are pulse wave indices related to the state of blood flow, and LF, HF, LF / HF, CVRR, and TP are pulse wave indices related to the activity state of the autonomic nerves. PA is the average value of the brightness difference between a certain minimum peak P1 and the next maximum peak P2 in the pulse waveform shown in Fig. 5. A large PA indicates a strong pulse and good blood circulation. The baseline is the average value of the luminance values ​​representing the pulse waveform. As described above, in this embodiment, a converted luminance value is used, so a large baseline indicates a large amount of blood and congestion. HR is the number of times the luminance value increases and decreases repeatedly per minute (for example, the number of minimum peaks P1 or maximum peaks P2). As shown in FIG. 6, LF is the sum of power spectrum density in the frequency band of 0.04 to 0.15 Hz. LF reflects both vasomotor sympathetic and parasympathetic nerve activity. The lower limit of LF, 0.04 Hz, corresponds to 25 seconds. Therefore, in order to properly detect this frequency component, the shooting time of the video is set to about 1 minute, which is sufficiently longer than 25 seconds. HF is the sum of the power spectral density in the frequency band from 0.15 to 0.4 Hz, as shown in Figure 6. HF mainly reflects the activity of the parasympathetic nervous system. LF / HF is the ratio of LF to HF. Since LF and HF have the above-mentioned characteristics, a larger LF / HF value indicates a higher predominance of the sympathetic nervous system, and a smaller LF / HF value indicates a higher predominance of the parasympathetic nervous system. CVRR represents the variation in the heartbeat interval FFI (Foot-to-Foot Interval) shown in Figure 5. In detail, CVRR is calculated by the following formula. CVRR(%)=(Standard deviation of FFI / Average of FFI)×100 A higher CVRR value indicates greater parasympathetic dominance. TP is equivalent to the sum of LF and HF. TP reflects the total activity of the autonomic nervous system. Based on the values ​​of these pulse wave indices derived from the pulse waveform data, biometric information 45 is generated.

[0039] FIG. 7 is a diagram showing an example of the biometric information 45. As shown in FIG. 7 corresponds to, among the pulse wave indices described above, PA and baseline related to the blood flow state, and HR, LF / HF, CVRR, and TP related to the autonomic nerves. In more detail, the biological information 45 includes, for each pulse wave index 451, reference data 452, measurement data 453 (pulse wave index value, biological index value), and a value of rate of change 454. The reference data 452 is a pulse wave index value (reference value) measured for each user at rest (e.g., when at home or before leaving for a trip). The reference data 452 is stored in the storage unit 13 of each terminal device 10 used by each user. The reference data 452 may be an average value of multiple measurements at a predetermined timing (e.g., every morning). The measurement data 453 is a value of the pulse wave index measured at a time other than when the reference data 452 was acquired (for example, at a travel destination, etc.). Note that it is preferable to take pictures with direct lighting (built-in LED, etc.) turned on in order to keep the brightness constant during each measurement. The rate of change 454 indicates the degree to which the measurement data 453 fluctuates with respect to the reference data 452. The rate of change 454 is a value obtained by subtracting 1 from the ratio of the measurement data 453 to the reference data 452 and multiplying the result by 100. The user's mental and physical state at the time of capturing the video is reflected in the measurement data 453 and the rate of change 454 of the biometric information 45. The generated biometric information 45 is registered in the reference information DB 232.

[0040] FIG. 8 is a diagram showing an example of the contents of the reference information DB 232. As shown in FIG. One record (data row) in the reference information DB 232 corresponds to one piece of reference information 40. One piece of reference information 40 includes one piece of video data transmitted from a certain terminal device 10, and date and time information 41, location information 42, attribute information 43, and environment information 44 transmitted from the terminal device 10 to the server 20 together with the video data. The date and time information 41 indicates the date and time when the video was captured by the terminal device 10. The location information 42 includes columns for "area name," "latitude," and "longitude." "Latitude" and "longitude" indicate the latitude and longitude of the terminal device 10 acquired by the location information acquisition unit 17 when the terminal device 10 shoots a video. "Area name" indicates the area to which the latitude and longitude belong. The type and size of the area are not particularly limited. For example, the area may be an administrative district such as a city, ward, town, or village, a tourist spot (mountains, plateaus, lakes, rivers, coasts, islands, hot springs, etc.), a tourist facility, an amusement facility, etc. If the latitude and longitude do not belong to any area, the area name may be left blank. The attribute information 43 represents attributes of the user of the terminal device 10, and includes columns of “gender”, “occupation”, and “age”. The columns included in the attribute information 43 are not limited to these. The contents of the attribute information 43 are input by the user on the application 131 of the terminal device 10, for example. The environmental information 44 indicates the environment around the terminal device 10 and the user when the location information 42 was acquired, and includes columns for "weather" and "season." The environmental information 44 is input by the user on the application 131 of the terminal device 10, for example. The weather may be automatically determined by a sensor (not shown) provided in the terminal device 10. The season may also be determined from the date and time information 41. In the present embodiment, the reference information 40 includes the change rate 454 of the biological information 45. However, without being limited thereto, the reference information 40 may include the reference data 452, the measurement data 453, and the like, of the biological information 45.

[0041] Next, a pulse wave analysis process executed by the CPU 11 of the terminal device 10 and the CPU 21 of the server 20 to realize the operation of the database update step described above will be described.

[0042] FIG. 9 is a flowchart showing a control procedure for pulse wave analysis processing in the terminal device 10. The pulse wave analysis process is started when a specified operation for starting pulse wave analysis is performed while the application 131 is being executed on the terminal device 10, or when a specified start timing arrives. When the pulse wave analysis process is started, the CPU 11 of the terminal device 10 sets the reference data processing flag 121 to "0" (step S101). The reference data processing flag 121 is 1-bit data, and when the value is "1", it indicates that the reference data 452 is acquired in the pulse wave analysis process, and when the value is "0", it indicates that the measurement data 453 and the rate of change 454 are acquired in the pulse wave analysis process.

[0043] The CPU 11 determines whether or not the reference data 452 is stored in the storage unit 13 (step S102). If it is determined that the reference data 452 is stored ("YES" in step S102), the CPU 11 transmits the reference data 452 to the server 20 (step S103). If it is determined that the reference data 452 is not stored (step S104), the CPU 11 sets the reference data processing flag 121 to "1".

[0044] When step S103 or S104 is completed, the CPU 11 acquires the date and time information 41, the position information 42, the attribute information 43, and the environment information 44, and transmits them to the server 20 (step S105). The CPU 11 also causes the display unit 14 to display face guides G1 to G3 (step S106), and causes the imaging unit 16 to start capturing a moving image of the face in response to a user's operation to instruct the start of capturing (step S107). The CPU 11 determines whether a predetermined capturing time (e.g., one minute) has elapsed (step S108). If it is determined that the capturing time has elapsed ("YES" in step S108), the CPU 11 causes the imaging unit 16 to end capturing, compresses the generated moving image data in the Motion JPEG format, and transmits the compressed data to the server 20 (step S109). Note that some or all of the transmission of the reference data 452 in step S103, the transmission of the date and time information 41, the position information 42, the attribute information 43 and the environmental information 44 in step S105, and the transmission of the video data in step S109 may be performed all at once.

[0045] The CPU 11 repeatedly determines whether or not an analysis result has been received from the server 20 (step S110). If it is determined that an analysis result has been received ("YES" in step S110), the CPU 11 determines whether or not the reference data processing flag 121 is "1" (step S111). If it is determined that the reference data processing flag 121 is "1" ("YES" in step S111), the CPU 11 stores the received analysis result in the storage unit 13 as the reference data 452 (step S112), and sets the reference data processing flag 121 to "0" (step S113).

[0046] When step S113 is completed, or when it is determined in step S111 that the reference data processing flag 121 is "0" ("NO" in step S111), the CPU 11 causes the display unit 14 to display a result screen including the received analysis results (step S114). The result screen includes, for example, the biological information 45 shown in Fig. 7. However, if steps S112 and S113 have been completed, the received analysis results represent the reference data 452, and therefore the measurement data 453 and the rate of change 454 are not displayed.

[0047] The CPU 11 determines whether or not an operation to end the pulse wave analysis has been performed (step S115). If the CPU 11 determines that the operation has not been performed ("NO" in step S115), the process returns to step S106, and if the CPU 11 determines that the operation has been performed ("YES" in step S115), the pulse wave analysis process ends.

[0048] FIG. 10 is a flowchart showing the control procedure of the pulse wave analysis process in the server 20. When the pulse wave analysis process is started, the CPU 21 determines whether or not the reference data 452 has been received from the terminal device 10 (step S201), and if it is determined that the reference data 452 has been received ("YES" in step S201), the CPU 21 stores the reference data 452 in the memory unit 23 (step S202).

[0049] The CPU 21 determines whether the date and time information 41, the position information 42, the attribute information 43, and the environment information 44 have been received (step S203). If it is determined that these pieces of information have not been received ("NO" in step S203), the CPU 21 returns the process to step S201. If it is determined that the above information has been received ("YES" in step S203), the CPU 21 adds a record to the reference information DB 232 and registers the fields (data) of the above information for each column (step S204). Note that if some of the date and time information 41, the position information 42, the attribute information 43, and the environment information 44, including the position information 42, have been received, the process may branch to "YES" in step S203. In this case, the CPU 21 registers only the received information in the record in step S204.

[0050] When step S204 is completed, or when it is determined in step S201 that the reference data 452 has not been received ("NO" in step S201), the CPU 21 determines whether or not video data has been received from the terminal device 10 (step S205). When it is determined that video data has not been received ("NO" in step S205), the CPU 11 returns the process to step S201, and when it is determined that video data has been received ("YES" in step S205), the CPU 11 executes video analysis processing (step S206).

[0051] FIG. 11 is a flowchart showing a control procedure for the video analysis process. When the video analysis process is called, the CPU 21 performs a predetermined image recognition process on the video data to detect a face (step S206a). The CPU 21 sets an ROI at a predetermined part of the detected face (step S206b). The CPU 21 derives the average value of the brightness values ​​of the green pixels in the set ROI for each frame image to detect a pulse wave (generate pulse waveform data) (step S206c). Since the waveform of the pulse wave detected here contains fluctuations in DC components due to body movement, breathing, etc., the CPU 21 performs a predetermined filtering process on the pulse waveform data to remove the DC components (step S206d). The CPU 21 detects peak values ​​(brightness values ​​of the minimum peak P1 and the maximum peak P2) in the pulse waveform (step S206e). The CPU 21 derives a pulse rate (HR) based on the number of minimum peaks P1 or maximum peaks P2 per minute, etc. (step S206f). CPU 21 derives the pulse wave amplitude (PA) based on the peak value detection result in step S206e (step S206g). CPU 21 derives a baseline based on the luminance value of the pulse waveform (step S206h). CPU 21 performs frequency analysis processing on the pulse waveform data (step S206i) and derives the values ​​of LF / HF, CVRR, and total power (TP) (step S206j). When step S206j ends, the CPU 21 ends the moving image analysis process, and returns the process to the pulse wave analysis process of FIG.

[0052] 10 (step S206) is completed, CPU 21 determines whether or not reference data 452 has been saved in step S202 (step S207). If it is determined that reference data 452 has been saved ("YES" in step S207), CPU 21 derives rate of change 454 of each piece of measurement data 453 (pulse wave index value) derived in the video analysis process (step S208). CPU 21 also transmits each piece of measurement data 453 (pulse wave index value) and rate of change 454 to terminal device 10 as the analysis result (step S209). On the other hand, if it is determined that the reference data 452 is not stored ("NO" in step S207), the CPU 21 transmits each measurement data 453 (pulse wave index value) to the terminal device 10 as the analysis result (step S210). When step S209 or S210 ends, the CPU 21 ends the pulse wave analysis process.

[0053] (Area information display step) Next, the detailed operation of the area information display step will be described. When a predetermined operation is performed on the terminal device 10 while the application 131 is being executed, a top screen 50 for specifying search conditions for area information is displayed on the display unit 14.

[0054] FIG. 12 is a diagram showing the top screen 50. As shown in FIG. The top screen 50 displays a text box 51 for inputting the name of the area to search, drop-down lists 52a-52c for specifying search conditions related to attributes, drop-down lists 52d, 52e for specifying search conditions related to the environment, check boxes 53a-53c for specifying search criteria, and a search button 54 for starting the search.

[0055] By selecting the search button 54 with an area name entered in the text box 51, a map 81 (see FIG. 17) of the entered area can be displayed as area information. By selecting the search button 54 with the text box 51 left blank, a list 82 (see FIG. 19) including multiple areas extracted based on the search perspective specified by the check boxes 53a to 53c can be displayed as area information.

[0056] The drop-down lists 52a to 52c allow the user to specify gender, occupation, and age as search conditions related to attributes. When a search condition related to an attribute is specified, area information is generated and displayed based on the reference information 40 including attribute information 43 that matches the specified attribute. When user attribute information is stored in the terminal device 10, an attribute that matches the attribute information may be automatically set as a search condition.

[0057] The drop-down lists 52d and 52e allow the user to specify weather and season as search conditions related to the environment, respectively. When a search condition related to the environment is specified, area information is generated and displayed based on the reference information 40 including the environment information 44 that matches the specified environment. When the terminal device 10 can determine the surrounding environment, the environment that matches the determined environment may be automatically set as a search condition.

[0058] When any or all of the check boxes 53a to 53c are selected (checked), area information is generated and displayed from the viewpoint corresponding to the selected check box. In detail, by selecting the check boxes 53a to 53c, area information can be searched from the viewpoints of "excitement level", "relaxation level", and "blood circulation promotion level", respectively. Note that the search viewpoints are not limited to the three shown in FIG. 12, and may include other viewpoints such as "stress level", "tension level", etc.

[0059] When the search button 54 is selected on the top screen 50, reference information 40 that matches the search conditions is extracted from the reference information DB 232 based on the current search condition specification status, i.e., the input status of the text box 51, the drop-down lists 52a to 52e, and the check boxes 53a to 53c.

[0060] FIG. 13 is a diagram showing an example of reference information 40 extracted based on the search conditions. FIG. 13 illustrates an example of an extraction result of the reference information 40 when the area name "B" is specified in the text box 51, and "summer" is specified in the drop-down list 52e, and the search button 54 is selected. In accordance with the above-mentioned search condition specification status, the reference information 40 is extracted in which the "area name" in the location information 42 is "B" and the "season" in the environmental information 44 is "summer". In this way, the extracted reference information 40 is the reference information 40 whose location indicated by the location information 42 belongs to a certain area according to the search condition. Furthermore, when a certain environment (here, "summer") is specified as the search condition, the reference information 40 including the environmental information 44 indicating the certain environment is acquired from the reference information DB 232. Although not shown in FIG. 13, when a certain attribute is specified as a search condition, the reference information 40 including the attribute information 43 representing the certain environment is acquired from the reference information DB 232.

[0061] When reference information 40 is extracted in accordance with the search conditions, an average value of the rate of change 454 is derived for each pulse wave index of biological information 45. This average value is used as condition index value 30 relating to the predicted mental and physical condition of the user in the area to be searched. Based on the derived condition index value 30, the area is evaluated from the search perspective selected by check boxes 53a to 53c. In this embodiment, the average value of the multiple change rates 454 in the multiple extracted reference information 40 is used as the condition index value 30, but instead, any representative value of the multiple change rates 454, for example the median, may be used.

[0062] FIG. 14 is a diagram showing the evaluation criteria for excitement level and the display mode of the evaluation results. When the checkbox 53a is selected, the excitement level predicted based on the state index value 30 related to the change rate of the pulse rate (ΔHR) and / or the state index value 30 related to the change rate of the total power (ΔTP) is evaluated. FIG. 14 shows an example of the evaluation criteria when the excitement level is evaluated in five levels from the lowest LV1 to the highest LV5. When the state index value 30 related to the change rate of the pulse rate (hereinafter referred to as "R1") is used as the evaluation criterion, if R1 ≤ -20% is satisfied, the excitement level is determined to be the lowest LV1. If -20% < R1 ≤ -10% is satisfied, it is determined to be LV2. If -10% < R1 < 10% is satisfied, it is determined to be LV3. If 10% ≤ R1 < 20% is satisfied, it is determined to be LV4. If 20% ≤ R1 is satisfied, the excitement level is determined to be the highest LV5. When the state index value 30 related to the change rate of TP (hereinafter referred to as "R2") is used as the evaluation criterion, if R2 ≤ -20% is satisfied, the excitement level is determined to be the lowest LV1. If -20% < R2 ≤ -10% is satisfied, it is determined to be LV2. If -10% < R2 < 10% is satisfied, it is determined to be LV3. If 10% ≤ R2 < 20% is satisfied, it is determined to be LV4. If 20% ≤ R2 is satisfied, the excitement level is determined to be the highest LV5. In addition, another evaluation criterion calculated from the state index value 30 related to the change rate of the pulse rate and the state index value 30 related to the change rate of TP may be set, and it may be determined which of LV1 to LV5 the excitement level corresponds to based on the evaluation criterion.

[0063] The evaluation results of the excitement level (LV1 to LV5) are displayed by the label 71 of pattern A or pattern B shown in FIG. 14, for example, on the map 81 or the list 82. In pattern A, the excitement level is represented by the size of the heart-shaped label 71. Specifically, the lower the excitement level, the smaller the label 71 is displayed, and the higher the excitement level, the larger the label 71 is displayed. In Pattern B, the degree of excitement is represented by the color of the label 71. Specifically, the color of the label 71 is adjusted so that the colder the color, the lower the degree of excitement, and the warmer the color, the higher the degree of excitement. For example, the color of the label 71 can be blue at LV1, red at LV5, and intermediate colors at LV2 to LV4. In either Pattern A or B, the label 71 may include the display of the value of R1 or R2. The label 71 (and the labels 72 and 73 described later) is an aspect of the state information related to the magnitude of the state index value 30.

[0064] FIG. 15 is a diagram showing the evaluation criteria for the degree of relaxation and the display mode of the evaluation results. When the checkbox 53b is selected, the predicted degree of relaxation is evaluated based on the state index value 30 related to the change rate of LF / HF (ΔLF / HF) and / or the state index value 30 related to the change rate of CVRR (ΔCVRR). FIG. 15 shows an example of the evaluation criteria when the degree of relaxation is evaluated in five levels from the lowest LV1 to the highest LV5. When the state index value 30 related to the change rate of LF / HF (hereinafter referred to as "R3") is used as the evaluation criterion, if 20% ≦ R3 is satisfied, it is determined as LV1 with the lowest degree of relaxation; if 10% ≦ R3 < 20% is satisfied, it is determined as LV2; if -10% < R3 < 10% is satisfied, it is determined as LV3; if -20% < R3 ≦ -10% is satisfied, it is determined as LV4; if R3 ≦ -20% is satisfied, it is determined as LV5 with a high degree of relaxation. When the state index value 30 related to the change rate of CVRR (hereinafter referred to as "R4") is used as the evaluation criterion, if R4 ≦ -20% is satisfied, it is determined as LV1 with the lowest degree of relaxation; if -20% < R4 ≦ -10% is satisfied, it is determined as LV2; if -10% < R4 < 10% is satisfied, it is determined as LV3; if 10% ≦ R4 < 20% is satisfied, it is determined as LV4; if 20% ≦ R4 is satisfied, it is determined as LV5 with the highest degree of relaxation. In addition, another evaluation criterion calculated from the state index value 30 related to the change rate of LF / HR and the state index value 30 related to the change rate of CVRR may be set, and based on this evaluation criterion, it may be determined which of LV1 to LV5 the degree of relaxation corresponds to.

[0065] The evaluation result (LV1 to LV5) of the degree of relaxation is also represented by the label 72 of pattern A or pattern B, similar to the degree of excitement. Regarding the color of pattern B, the label 72 may be displayed in a color such that the lower the degree of relaxation, the warmer the color system, and the higher the degree of relaxation, the colder the color system.

[0066] FIG. 16 is a diagram showing the evaluation criteria for the degree of blood circulation promotion and the display mode of the evaluation result. When the check box 53c is selected, the predicted degree of blood circulation promotion is evaluated based on the state index value 30 related to the change rate (ΔPA) of the pulse wave amplitude (PA) and / or the state index value 30 related to the change rate (Δbaseline) of the baseline. FIG. 15 shows an example of the evaluation criteria when the degree of blood circulation promotion is evaluated in five levels from the lowest LV1 to the highest LV5. When the state index value 30 related to the change rate of PA (hereinafter referred to as "R5") is used as the evaluation criterion, when R5≤−20% is satisfied, the degree of blood circulation promotion is determined to be the lowest LV1, when −20%<R5≤−10% is satisfied, it is determined to be LV2, when −10%<R5<10% is satisfied, it is determined to be LV3, when 10%≤R5<20% is satisfied, it is determined to be LV4, and when 20%≤R5 is satisfied, the degree of blood circulation promotion is determined to be the highest LV5. When the state index value 30 related to the change rate of the baseline (hereinafter referred to as "R6") is used as the evaluation criterion, when R6≤−20% is satisfied, the degree of blood circulation promotion is determined to be the lowest LV1, when −20%<R6≤−10% is satisfied, it is determined to be LV2, when −10%<R6<10% is satisfied, it is determined to be LV3, when 10%≤R6<20% is satisfied, it is determined to be LV4, and when 20%≤R6 is satisfied, the degree of blood circulation promotion is determined to be the highest LV5. In addition, a separate evaluation criterion may be set that is calculated from the condition index value 30 relating to the rate of change of PA and the condition index value 30 relating to the rate of change of the baseline, and the degree of blood circulation promotion may be determined to correspond to one of LV1 to LV5 based on that evaluation criterion.

[0067] The evaluation result of the blood circulation promotion level (LV1 to LV5) is also represented by the mark 73 of pattern A or pattern B, similar to the excitement level.

[0068] When the evaluation of the excitement level, relaxation level, or blood circulation promotion level based on the state index value 30 is completed, if an area name is input in the text box 51, map data of the map 81 of the area, which includes signs 71-73 indicating the evaluation result, is generated. The map data is transmitted to the terminal device 10, whereby the map 81 is displayed on the display unit 14 of the terminal device 10.

[0069] FIG. 17 is a diagram showing an example of a map 81. As shown in FIG. 17 illustrates a case where the reference information 40 shown in FIG. 13 is extracted based on the search conditions, and the excitement level and the relaxation level are specified as search criteria by check boxes 53a and 53b. Here, the excitement level is evaluated by the state index value 30 related to the rate of change of the pulse rate, and the relaxation level is evaluated by the state index value 30 related to the rate of change of the LF / HF. As shown in Fig. 13, the condition index value 30 related to the rate of change in the pulse rate is "10%", so the excitement level is determined to be level 4 according to the evaluation criteria shown in Fig. 14. Therefore, on the map 81 shown in Fig. 17, the second largest sign 71 is displayed according to pattern A. Furthermore, as shown in Fig. 13, the state index value 30 related to the rate of change of LF / HF is "-20%", so the relaxation level is determined to be LV5 according to the evaluation criteria shown in Fig. 15. Therefore, on the map 81 shown in Fig. 17, the largest sign 72 is displayed according to pattern A.

[0070] On the top screen 50 shown in FIG. 12, when the search button 54 is selected with the text box 51 left blank, a plurality of candidate areas to be listed in the list 82 are determined. For each of the determined plurality of areas, reference information 40 is extracted as shown in FIG. 13, and a condition index value 30 is derived. The condition index value 30 for each of the plurality of areas is stored in a candidate area table 233 shown in FIG. 18. In FIG. 18, the condition index value 30 derived for each of the determined plurality of areas "A", "B", ... "Z" is stored. The method of determining the plurality of areas is not particularly limited, and may be, for example, a predetermined number of areas in descending order of the number of times that the map 81 has been displayed by other users.

[0071] When candidate area table 233 is generated, candidate area table 233 is sorted by condition index value 30 of the pulse wave index serving as the evaluation criterion for the search perspective selected from among check boxes 53a to 53c. For example, when check box 53b is selected, candidate area table 233 is sorted by condition index value 30 relating to the rate of change of LF / HF or the rate of change of CVRR. When sorting is completed, list 82 including a predetermined number of areas from the top of the sorted results is displayed.

[0072] FIG. 19 is a diagram showing an example of the list 82. FIG. 19 illustrates an example of a list 82 in the case where the candidate zone table 233 shown in FIG. 18 has been generated and the check box 53b corresponding to the relaxation level has been selected on the top screen 50. It is also assumed that the state index value 30 relating to the rate of change of LF / HF is used as the evaluation criterion for the relaxation level. In the list 82, zone names "Z", "G", "A", ... are listed based on the result of sorting the candidate zone table 233 by the state index value 30 relating to the rate of change of LF / HF. The relaxation level corresponding to each zone is displayed by the indicators 72 of pattern B. Note that state information in a form different from the indicators 72, for example, state information representing the relaxation level by numerical values, symbols, figures, or the like, may be displayed.

[0073] Photographs and text related to tourist information of the area may also be displayed on the map 81 shown in FIG. 17 and the list 82 shown in FIG. In addition, in this embodiment, the representative value of the rate of change 454 of the measurement data 453 (pulse wave index value) is set as the condition index value 30, but instead, the representative value of the measurement data 453 (pulse wave index value) may be used as the condition index value 30 to evaluate the excitement level, etc.

[0074] Next, a description will be given of an area information display process executed by the CPU 11 of the terminal device 10 and the CPU 21 of the server 20 to realize the operation of the above-mentioned area information display step. FIG. 20 is a flowchart showing a control procedure for the area information display process in the terminal device 10. As shown in FIG. The area information display process is started when a predetermined operation for starting search and display of area information is performed while the application 131 is being executed on the terminal device 10. When the area information display process is started, the CPU 11 of the terminal device 10 displays the top screen 50 shown in FIG. 12 on the display unit 14 (step S301). The CPU 11 repeatedly determines whether or not an operation to instruct search execution, that is, an operation to select the search button 54, has been performed (step S302). If it is determined that the operation has been performed ("YES" in step S302), the CPU 11 determines whether or not a search condition related to an attribute and / or environment has been specified by the drop-down lists 52a to 52e (step S303). If it is determined that the search condition has been specified ("YES" in step S303), the CPU 11 transmits information on the search condition related to the specified attribute and / or environment to the server 20 (step S304). When step S304 is completed or when the process branches to "NO" in step S303, the CPU 11 transmits information on the search viewpoint specified by the check boxes 53a to 53c to the server 20 (step S305).

[0075] The CPU 11 determines whether or not the user has specified the name of the area to be searched in the text box 51 (step S306). If it is determined that the area name has been specified ("YES" in step S306), the CPU 11 transmits information of the specified area name to the server 20 (step S307). Note that two or more or all of the information transmissions in steps S304, S305, and S307 may be executed at once. After the end of step S307, the CPU 11 repeatedly determines whether or not the data of the map 81 has been received from the server 20 (step S308). If it is determined that the data of the map 81 has been received from the server 20 ("YES" in step S308), the CPU 11 causes the display unit 14 to display area information including the map 81 based on the received data (step S311).

[0076] On the other hand, if it is determined that the area name has not been specified ("NO" in step S306), the CPU 11 transmits a request to the server 20 to create the list 82 (step S309). Two or more or all of the information transmissions in steps S304, S305, and S309 may be executed at once. After the end of step S309, the CPU 11 repeatedly determines whether or not the data of the list 82 has been received from the server 20 (step S310). If it is determined that the data of the list 82 has been received from the server 20 ("YES" in step S310), the CPU 11 causes the display unit 14 to display area information including the map 81 based on the received data (step S311).

[0077] When step S311 is completed, the CPU 11 determines whether or not an operation has been performed to end the search for area information (step S312). If it is determined that the operation has not been performed ("NO" in step S312), the CPU 11 determines whether or not an operation has been performed to search for the next area information (step S313). If it is determined that an operation has been performed to perform the next search ("YES" in step S313), the CPU 11 returns the process to step S301, and if it is determined that the operation has not been performed ("NO" in step S313), the CPU 11 returns the process to step S312. If it is determined that an operation has been performed to end the search for area information ("YES" in step S312), the CPU 11 ends the area information display process.

[0078] FIG. 21 is a flowchart showing a control procedure for the area information display process in the server 20. When the area information display process is started, the CPU 21 of the server 20 resets the search conditions for searching area information (step S401). The CPU 21 determines whether or not it has received search condition information related to attributes and / or environments (step S402), and if it is determined that the information has been received ("YES" in step S402), it adds the contents of the specified attributes and / or environments to the search conditions (step S403).

[0079] When step S403 is completed, or when branching to "NO" in step S402, the CPU 21 determines whether or not information specifying an area name has been received (step S404). When it is determined that the information has been received ("YES" in step S404), the CPU 21 executes steps S405 to S409 for generating data for the map 81.

[0080] The CPU 21 adds the specified area name to the search conditions (step S405). The CPU 21 acquires reference information 40 that satisfies the search conditions from the reference information DB 232 (step S406). The CPU 21 derives a condition index value 30 corresponding to each pulse wave index value (step S407). The CPU 21 determines the markers 71-73 to be displayed in accordance with the evaluation criteria shown in Figs. 14-16, based on the condition index value 30 corresponding to the received search perspective (step S408). The CPU 21 generates map data of a map 81 corresponding to the area name, with the markers 71-73 superimposed thereon (step S409).

[0081] On the other hand, if it is determined in step S404 that information specifying the area name has not been received ("NO" in step S404), CPU 21 determines whether or not a request to create list 82 has been received (step S410). If it is determined that the request has not been received ("NO" in step S410), CPU 21 returns the process to step S402, and if it is determined that the request has been received ("YES" in step S410), CPU 21 executes steps S411 to S416 to generate data for list 82.

[0082] CPU 21 determines candidates for a plurality of area names to be listed in list 82, and generates a plurality of records corresponding to the plurality of area names in candidate area table 233 (step S411). CPU 21 selects one area name from the plurality of area name candidates and sets it as a search condition (step S412). CPU 21 acquires reference information 40 that satisfies the search condition from reference information DB 232 (step S413). CPU 21 derives condition index value 30 corresponding to each pulse wave index value, and registers it in a record in candidate area table 233 (step S414). CPU 21 determines whether condition index value 30 has been derived for all area names, and if it is determined that condition index value 30 has not been derived for any area name ("NO" in step S415), the process returns to step S412. If it is determined that status index values ​​30 have been derived for all area names ("YES" in step S415), the CPU 21 sorts the candidate area table 233 by the status index value 30 corresponding to the received search perspective, and generates list data for list 82 (step S416).

[0083] When step S409 or S416 ends, the CPU 21 transmits the generated map data or list data to the terminal device 10 (step S417). The process of transmitting the map data or list data corresponds to the process of causing the display unit 14 of the terminal device 10 to output area information relating to a certain area based on the derived state index value 30.

[0084] The CPU 21 determines whether or not an operation for terminating the search for area information has been performed on the terminal device 10 (step S418). If the CPU 21 determines that the operation has not been performed ("NO" in step S418), the CPU 21 returns the process to step S401, and if the CPU 21 determines that the operation has been performed ("YES" in step S418), the CPU 21 terminates the area information display process.

[0085] (Modification) Next, a modified example of the above embodiment will be described. Below, differences from the above embodiment will be described, and a description of features common to the above embodiment will be omitted. In the above embodiment, a pulse wave is detected in one ROI, and the measurement data 453 and the change rate 454 of each pulse wave index are set as the biological information 45. Alternatively, a pulse wave may be detected in two different ROIs of one video data, and the difference between the measurement data 453 derived based on each pulse wave waveform may be set as the biological information 45. For example, a first measurement data 453 may be obtained from a first pulse wave waveform detected in ROI1 of the forehead shown in FIG. 4, a second measurement data 453 may be obtained from a second pulse wave waveform detected in ROI2 of the nose, and the difference obtained by subtracting the first measurement data 453 from the second measurement data 453 may be registered in the reference information 40 as the biological information 45. A pulse wave can be detected relatively stably in the forehead, whereas in the nose, the blood flow rate is likely to change in response to changes in the mental and physical state due to stress or the like, and the change is likely to appear as a change in the PA, baseline, etc. in the pulse wave waveform. Therefore, by taking the difference in the measurement data 453 of PA and baseline between the ROI1 of the forehead and the ROI2 of the nose, it is possible to generate the biological information 45 and the reference information 40 that reflect the change in mental and physical state due to stress, etc. PA and baseline can be measured from a short video (for example, about 10 seconds). Therefore, according to the method of this modification, it is possible to accurately evaluate the search aspect such as stress level by shooting a video for a shorter period of time without using pulse wave indices related to the autonomic nerves such as LF / HF and TP, which require shooting for about one minute. It is also possible to register the above-mentioned reference data for the difference in advance, and derive the rate of change of the difference of the measurement data 453 from the reference data for the difference in the measurement data 453 and register it in the reference information 40.

[0086] (effect) As described above, the server 20 according to the present embodiment includes a CPU 21, and the CPU 21 acquires a search instruction related to a zone input by a user. When the search instruction is acquired, the CPU 21 acquires reference information 40 in which a position represented by the position information 42 belongs to a certain zone from a reference information DB 232 in which reference information 40 including a person's position information 42 and a person's biometric information 45 when the person was at the position represented by the position information 42 is accumulated, and based on the acquired reference information 40, a condition index value 30 related to a predicted mental and physical state of the user in a certain zone is derived, and zone information related to the certain zone is generated based on the derived condition index value 30. By displaying this biometric information on the display unit 14, it is possible to provide the user with zone information reflecting a predicted mental and physical state in the zone, which could not be provided by the conventional technology. Therefore, it is possible to provide information more appropriately in line with the needs of the user. For example, it is possible to provide appropriate information about zones that are candidates for travel destinations to a user who is traveling in search of relaxation or a user who is traveling in search of excitement. In addition, by using the reference information 40 of a person who was actually in a certain area, it is possible to accurately predict the mental and physical state in a certain zone.

[0087] The area information also includes a map 81 of a certain area and signs 71-73 related to the magnitude of the state index value 30. This makes it possible to provide the user with visually easy-to-understand area information including information related to the predicted mental and physical state. For example, by displaying places with a high level of relaxation on the map 81 and indicating the level of relaxation with the sign 72, a relaxation point map can be provided as tourist information. Similar information can also be provided regarding the level of excitement and the level of blood circulation promotion.

[0088] Furthermore, the CPU 21 generates information of a list 82 including a plurality of signs 71-73 relating to a plurality of different areas as area information. This makes it possible to provide information relating to predicted mental and physical states for a plurality of areas in a list format. Therefore, for example, suitable travel destination candidates can be provided to a user who is traveling in search of relaxation or a user who is traveling in search of excitement.

[0089] Furthermore, the reference information 40 includes attribute information 43 relating to human attributes, and the CPU 21 acquires the reference information 40 including the attribute information 43 representing a certain attribute from the reference information DB 232, and derives the state index value 30 based on the acquired reference information 40. This makes it possible to provide more appropriate area information to a user having a certain attribute.

[0090] Furthermore, the reference information 40 includes environmental information 44 related to the environment around the person when the position information 42 was acquired, and the CPU 21 acquires the reference information 40 including the environmental information 44 representing a certain environment from the reference information DB 232, and derives the condition index value 30 based on the acquired reference information 40. This makes it possible to provide area information including information related to a predicted mental and physical condition in a certain area under a certain environment.

[0091] Furthermore, the biological information 45 includes a pulse wave index value (biological index value) derived based on a pulse waveform detected from a video of a person, or a change rate 454 of the pulse wave index value relative to reference data 452, and the CPU 21 acquires from the reference information DB 232 a plurality of reference information 40 in which the position indicated by the position information 42 belongs to a certain area, and derives a representative value of the pulse wave index value or a representative value of the change rate 454 in the plurality of biological information 45 included in the acquired plurality of reference information 40 as the condition index value 30. By using the pulse wave index value in this way, the reference information DB 232 accumulates the reference information 40 reflecting the state of blood flow and the activity state of the autonomic nerves. By using such reference information 40, the mental and physical state can be predicted with higher accuracy.

[0092] The biometric index value is a value of a pulse wave index related to a pulse waveform detected in one ROI in a video, and the pulse wave index is PA, baseline, HR, LF / HF, CVRR, or TP. This makes it possible to obtain biometric information 45 by a simple process of detecting a pulse wave in a single ROI and analyzing the pulse waveform.

[0093] Moreover, the pulse wave index value in this modification is the difference between the pulse wave index value associated with a first pulse waveform detected in ROI 1 in the video and the pulse wave index value associated with a second pulse waveform detected in ROI 2 different from ROI 1. This makes it possible to acquire appropriate biological information 45 in a shorter period of time than when a single ROI is used.

[0094] Furthermore, the CPU 11 causes the display unit 14 to display (output) the area information, so that area information that meets the needs of the user can be presented to the user.

[0095] Furthermore, in the information processing method according to the present embodiment, the CPU 21 acquires reference information 40 in which the position indicated by the position information 42 belongs to a certain area from a reference information DB 232 in which reference information 40 including a person's position information 42 and the person's biometric information 45 when the person was in the position indicated by the position information 42 is accumulated, derives a condition index value 30 related to the predicted mental and physical condition of the user in the certain area based on the acquired reference information 40, and causes the display unit 14 to display area information related to the certain area based on the derived condition index value 30. This makes it possible to provide information that more appropriately meets the needs of the user.

[0096] Moreover, the program 231 according to this embodiment causes the CPU 21 to execute a process of acquiring reference information 40 in which the position represented by the position information 42 belongs to a certain area from a reference information DB 232 in which reference information 40 including a person's position information 42 and biometric information 45 of the person when the person was in the position represented by the position information 42 is accumulated, a process of deriving a condition index value 30 related to a predicted mental and physical condition of the user in the certain area based on the acquired reference information 40, and a process of displaying area information related to the certain area on the display unit 14 based on the derived condition index value 30. This makes it possible to provide information that more appropriately meets the needs of the user.

[0097] (others) The present invention is not limited to the above-described embodiment, but may be modified in various ways. For example, the processes executed by the server 20 in the above embodiment may be executed by the terminal device 10. In this case, the terminal device 10 corresponds to the "information processing device", and the CPU 11 of the terminal device 10 corresponds to the "processing unit". Furthermore, the processing executed by the server 20 in the above embodiment may be shared and executed by the terminal device 10 and the server 20. In this case, the configuration consisting of the terminal device 10 and the server 20 corresponds to an "information processing device", and the configuration consisting of the CPU 11 of the terminal device 10 and the CPU 21 of the server 20 corresponds to a "processing unit".

[0098] In the above embodiment, the pulse wave is detected based on a video of the face captured by the terminal device 10 to generate the biometric information 45, but the method of generating the biometric information 45 is not limited to this. For example, when the terminal device 10 is a wearable terminal that is worn on the user's body (for example, a smart watch worn on the wrist), the pulse wave may be detected by a pulse wave detection unit provided in the terminal device 10, and the biometric information 45 may be generated by analyzing the waveform of the pulse wave. In this case, the generation of the biometric information 45 may be performed by either the CPU 11 of the terminal device 10 or the CPU 21 of the server 20. Also, a pulse wave detection unit may be provided in a wearable terminal separate from the terminal device 10 to detect the pulse wave, and pulse wave waveform data or the analysis result thereof may be transmitted to the terminal device 10 that is connected to the wearable terminal device by short-range wireless communication or the like.

[0099] Furthermore, the biological information is not limited to information based on a pulse waveform, but may be information generated based on, for example, blood pressure, respiratory rate, or the like.

[0100] The reference information DB 232 may be stored in an external storage device of the server 20.

[0101] In addition, in the above embodiment, an example has been described in which the server 20 performs pulse wave analysis and generates and controls the display of area information, but this is not limited to this, and a server that performs pulse wave analysis and a server that generates and controls the display of area information may be provided separately.

[0102] Furthermore, the process of outputting the area information is not limited to the process of displaying the area information on the display unit 14, but may be, for example, the process of printing the area information on a printing unit.

[0103] In the above description, an example has been disclosed in which the flash memory of the storage unit 13 and the HDD and SSD of the storage unit 23 are used as computer-readable media for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, information recording media such as flash memory and CD-ROMs can be applied. In addition, carrier waves are also applied to the present invention as a medium for providing data for the program according to the present invention via a communication line.

[0104] Furthermore, it goes without saying that the detailed configuration and detailed operation of each component of the information providing system 1 in the above embodiment can be modified as appropriate without departing from the spirit and scope of the present invention.

[0105] Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0106] 1 Information provision system 10 Terminal Equipment 11 CPU 14 Display section (output section) 20 Server (information processing device) 21 CPU (processing unit, computer) 23 Memory section 232 Reference Information DB (Database) 30 Status Indicator Value 40 References 41 Date and Time Information 42 Location information 43 Attribute information 44 Environmental information 45 Biometric information 451 Pulse Wave Index 452 Reference Data (Reference Value) 453 Measurement data (pulse wave index value, biometric value) 454 Rate of change 71~73 Sign (Status Information) 81 Map (area information) 82 List (Area Information) ROI1~ROI5 Regions of interest

Claims

1. Obtaining search instructions related to a region entered by a user; When the search instruction is received, the method obtains reference information, in which the location represented by the location information belongs to a certain area, from a database in which reference information including location information of a person and biometric information of the person when the person was at the location represented by the location information is accumulated; deriving a state index value related to a predicted mental and physical state of the user in the certain area based on the acquired reference information; generating area information for the certain area based on the derived condition index value; An information processing device comprising a processing unit.

2. The area information includes a map of the certain area and status information related to a magnitude of the status index value. The information processing device according to claim 1 .

3. the area information includes status information related to a magnitude of the status index value, The processing unit generates, as the area information, information of a list including a plurality of pieces of status information relating to a plurality of the certain areas different from each other. The information processing device according to claim 1 .

4. the reference information includes attribute information related to attributes of the person; The processing unit includes: obtaining the reference information including the attribute information representing an attribute from the database; deriving the condition index value based on the obtained reference information; The information processing device according to claim 1 .

5. the reference information includes environmental information relating to an environment around the person at the time the location information was acquired, The processing unit includes: obtaining the reference information from the database, the reference information including the environment information representing an environment; deriving the condition index value based on the obtained reference information; The information processing device according to claim 1 .

6. The biological information includes a biological index value derived based on a pulse waveform detected from a video of the person, or a rate of change of the biological index value with respect to a predetermined reference value, The processing unit includes: Obtaining from the database a plurality of pieces of reference information whose locations represented by the location information belong to the certain area; deriving a representative value of the biological index values ​​or a representative value of the change rates in the plurality of biological information included in the plurality of acquired reference information as the state index value; The information processing device according to claim 1 .

7. the biological index value is a value of a pulse wave index related to a pulse waveform detected in one region of interest in the moving image, The pulse wave index is the amplitude of the pulse waveform, the baseline of the pulse waveform, the pulse rate, LF / HF, CVRR, or total power. The information processing device according to claim 6.

8. the biological index value is a difference between a value of a pulse wave index associated with a first pulse waveform detected in a first region of interest in the moving image and a value of the pulse wave index associated with a second pulse waveform detected in a second region of interest in the moving image that is different from the first region of interest, The pulse wave index is the amplitude of the pulse waveform, the baseline of the pulse waveform, the pulse rate, LF / HF, CVRR, or total power. The information processing device according to claim 6.

9. The processing unit causes an output unit to output the generated area information. The information processing device according to claim 1 .

10. 1. A computer-implemented information processing method, comprising: Obtaining search instructions related to a region entered by a user; When the search instruction is received, the method obtains reference information, in which the location represented by the location information belongs to a certain area, from a database in which reference information including location information of a person and biometric information of the person when the person was at the location represented by the location information is accumulated; deriving a state index value related to a predicted mental and physical state of the user in the certain area based on the acquired reference information; generating area information for the certain area based on the derived condition index value; Information processing methods.

11. On the computer, obtaining search instructions related to the area entered by the user; a process of acquiring, when the search instruction is acquired, reference information in which the location represented by the location information belongs to a certain area from a database in which reference information including location information of a person and biometric information of the person when the person was at the location represented by the location information is accumulated; deriving a state index value related to a predicted mental and physical state of the user in the certain area based on the acquired reference information; generating area information for the certain area based on the derived condition index value; A program that executes the following.

Images