Processing device

Abstract

This processing device includes a content display unit that, when estimating the biological state of an estimation subject using image data in which the face of the estimation subject is set to be as at least an estimation target, displays, on a screen, predetermined content different from information on the estimation of the biological state. The content display unit ends the displaying of the content in accordance with the progress of the biological state estimation.

Description

Processing device

[0001] The present invention relates to a processing device, a processing method, and a recording medium.

[0002] Techniques for estimating a biological state such as a pulse or respiration using image data are known.

[0003] For example, Patent Document 1 describes a pulse wave detection device that detects a pulse wave, which is a variation in the volume of blood in a blood vessel, using time-series images of the face of a subject. According to Patent Document 1, the pulse wave detection device includes a region of interest setting unit, a luminance data generation unit, a displacement data generation unit, a pulse wave component extraction unit, and an output unit. For example, the region of interest setting unit sets a region of interest of the face of the subject in the time-series images of the face of the subject. Further, the luminance data generation unit generates luminance data, which is time-series data of representative values of luminance values of a predetermined color component of pixels in the region of interest. Subsequently, the displacement data generation unit generates displacement data indicating the movement of the face of the subject based on the time-series images. Then, the pulse wave component extraction unit extracts a signal component in the luminance data due to the pulse wave using the displacement data. Thereafter, the output unit outputs information on the pulse wave based on the extracted signal component.

[0004] As a related technique, for example, there is Patent Document 2. Patent Document 2 describes an information processing device that recommends content based on biological information. According to Patent Document 2, the information processing device acquires biological information representing a biological reaction shown by a user during the reproduction of content and evaluations of each of the reproduced contents for a plurality of users, and based on the acquired biological information, during the reproduction of the same content, identifies a similar user who shows a biological reaction similar to the biological reaction shown by the user who has requested the recommendation of the content. Thereafter, the information processing device identifies, as recommended content, content that is highly evaluated for the identified similar user. Then, the information processing device transmits information on the identified recommended content to an information processing terminal used by the user who has requested the recommendation of the content.

[0005] International Publication No. 2019 / 102535, Japanese Patent Application Laid-Open No. 2009-134669

[0006] As described in Patent Document 1, when acquiring biometric information such as pulse rate using image data, the user whose biometric information is being acquired must remain still for a certain period of time. As a result, some users may experience unpleasant feelings such as annoyance or inconvenience when acquiring biometric information. Thus, a problem has arisen in that it can be difficult to suppress the annoyance that arises when acquiring biometric information using image data.

[0007] Therefore, one of the objectives of this disclosure is to provide an processing apparatus, a processing method, and a recording medium that can solve the above-mentioned problems.

[0008] To achieve this objective, the processing apparatus in this disclosure has a content display unit that displays predetermined content on the screen that is different from information about the estimation of the biological state when estimating the biological state of a subject using image data in which the subject's face is at least the subject, and the content display unit is configured to terminate the display of the content according to the progress of the biological state estimation.

[0009] Furthermore, the processing method in this disclosure is configured such that when an information processing device estimates the biological state of a subject using image data in which the subject's face is at least the subject, it displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of the content according to the progress of the estimation of the biological state.

[0010] Furthermore, the recording medium in this disclosure is a computer-readable recording medium that stores a program for an information processing device that, when estimating the biological state of a subject using image data in which the subject's face is at least the subject, displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of the content according to the progress of the estimation of the biological state.

[0011] According to the configurations described above, it is possible to suppress the inconveniences that may arise when acquiring biometric information using image data.

[0012] This figure shows an example of the configuration of the estimation system in this disclosure. This is a block diagram showing an example of the configuration of the estimation device. This figure shows an example of content display. This figure shows an example of content display. This figure shows an example of content display. This figure shows an example of information to be output. This is a flowchart showing an example of the operation of the estimation device. This figure shows another example of the configuration of the estimation system. This figure shows an example of information to be output. This figure shows an example of the hardware configuration of the second estimation device in this disclosure. This is a block diagram showing an example of the configuration of the estimation device.

[0013] [First Embodiment] An example of the configuration of the estimation system 100 in this disclosure will be described with reference to Figures 1 to 9. Figure 1 is a diagram showing an example of the configuration of the estimation system 100. Figure 2 is a block diagram showing an example of the configuration of the estimation device 200. Figures 3 to 5 are diagrams showing examples of content display. Figure 6 is a diagram showing an example of information to be output. Figure 7 is a flowchart showing an example of the operation of the estimation device 200. Figure 8 is a diagram showing another example of the configuration of the estimation system 100. Figure 9 is a diagram showing an example of information to be output. Note that in this disclosure, the drawings may be associated with one or more embodiments.

[0014] This disclosure describes an estimation system 100 that estimates the biological state of a subject using image data in which at least the subject's face is the subject. As described later, the estimation system 100 estimates the biological state using the acquired image data. When estimating the biological state, the estimation system 100 displays predetermined content on the screen that is different from the information about the estimation of the biological state. The estimation system 100 then determines whether or not to end the display of the content depending on the status of the biological state estimation. As an example, the estimation system 100 displays interactive content that requires screen operation, such as a questionnaire requesting answers to questions. The estimation system 100 also determines whether or not to display the next question depending on the answer to a certain question, depending on the status of the biological state estimation. For example, the estimation system 100 ends the display of content when the biological state estimation is complete, such as not displaying new questions after the biological state estimation is complete. For example, by performing such content display control and content display termination control, the estimation system 100 guides the subject to keep their face and other features as still as possible during the biological state estimation without making the subject aware of it.

[0015] The estimation system 100 may obtain arbitrary biometric information by performing arbitrary processing on the acquired image data. For example, the estimation system 100 can obtain information indicating pulse waves, pulse rate, respiration, blood oxygen levels, and other arbitrary vital signs as biometric information. The estimation system 100 may obtain information indicating the emotions of the person being estimated, or other information corresponding to arbitrary states, instead of, or together with, the above information. Furthermore, the estimation system 100 can obtain the above-mentioned various types of information by performing arbitrary processing using the image data. For example, the estimation system 100 can obtain biometric information such as pulse waves by using a method that utilizes luminance data acquired from time-series image data, as described in Patent Document 1. The estimation system 100 may also obtain biometric information in response to pupillary response or eye movement acquired from image data, or by inputting image data or arbitrary information extracted from image data into a pre-trained model. The estimation system 100 may also obtain biometric information from image data using any other method.

[0016] Figure 1 shows an example configuration of the estimation system 100. Referring to Figure 1, the estimation system 100 includes an estimation device 200. For example, the estimation device 200 is a mobile device such as a smartphone or tablet. The estimation device 200 may be any information processing device other than those exemplified above.

[0017] Figure 2 is a block diagram showing an example configuration of the estimation device 200. Referring to Figure 2, the estimation device 200 has the following main components: an operation input unit 210, a screen display unit 220, a communication interface unit 230, an imaging unit 240, a storage unit 250, and a calculation processing unit 260.

[0018] Figure 2 illustrates a case where the functions of the estimation device 200 are realized using a single information processing device. However, at least a part of the functions of the estimation device 200 may be realized using multiple information processing devices, for example, by being implemented on the cloud. For example, the estimation device 200 may consist of an estimation device that estimates biometric information and a processing device that displays content or recommends products when estimating biometric information. Furthermore, the estimation device 200 may not include some of the configurations exemplified above, such as not having an operation input unit 210 or a screen display unit 220, and may have configurations other than those exemplified above.

[0019] The operation input unit 210 consists of an operation input device such as a touch panel, keyboard, or mouse. The operation input unit 210 detects the operation of the operator operating the estimation device 200 and outputs it to the calculation processing unit 260.

[0020] The screen display unit 220 consists of a screen display device such as a liquid crystal display or an organic electroluminescent (EL) display. The screen display unit 220 can display various information stored in the storage unit 250 on the screen in response to instructions from the arithmetic processing unit 260.

[0021] The communication interface unit 230 consists of a data communication circuit and the like. The communication interface unit 230 performs data communication with an external device connected via a communication line.

[0022] The imaging unit 240 consists of an imaging device such as a camera. The imaging unit 240 acquires image data in response to operations on the operation input unit 210. For example, the imaging unit 240 can acquire image data in which at least the face of the estimated subject is the subject. The imaging unit 240 may also acquire time-series image data such as video.

[0023] The storage unit 250 is a storage device such as a hard disk or memory. The storage unit 250 stores processing information and programs 254 necessary for various processes in the arithmetic processing unit 260. The programs 254 are read into the arithmetic processing unit 260 and executed to realize various processing processes. The programs 254 are pre-read from external devices or recording media via data input / output functions such as the communication interface unit 230 and stored in the storage unit 250. The main information stored in the storage unit 250 includes, for example, image data information 251, content information 252, and estimation result information 253.

[0024] Image data information 251 includes image data acquired by the imaging unit 240. As described above, image data information 251 may include image data in which the face of the estimated subject is the subject. Image data information 251 may also include time-series image data. Image data information 251 may associate image data with the time the image data was acquired. Image data information 251 can be updated in response to image data acquisition by the image acquisition unit 261, etc.

[0025] The content information 252 includes information about content to be displayed on the screen display unit 220, such as questionnaires. The content information 252 is acquired in advance by methods such as acquiring it in response to operations on the operation input unit 210, or acquiring it from an external device via the communication interface unit 230, and is stored in the storage unit 250. The content information 252 may be updated in response to content acquisition by the content acquisition unit 263 (described later) from an external device, etc.

[0026] As described above, in this disclosure, content is acquired during the estimation of the biological state. Therefore, it is desirable that the content included in content information 252 has as little impact as possible on the biological state of the subject being estimated. For example, questionnaires are expected to have less impact on the biological state, such as causing irritability or an increase in heart rate, compared to content such as games or videos. Therefore, it is preferable that the content included in content information 252 be questionnaires rather than games or videos. Thus, it is desirable that content information 252 includes content that has little impact on the biological state, or content that takes into consideration the impact it has. However, content information 252 may also include games, videos, and certain advertisements that are expected to have little impact on the biological state.

[0027] Furthermore, in this disclosure, content is displayed in such a way that the subject does not move their face significantly away from the imaging unit 240 during the estimation of biometric information. For this reason, it is desirable that the content included in the content information 252 be interactive content that requires action, such as a questionnaire, rather than static content such as text or non-interactive content that is only displayed. For example, with static or non-interactive content, there is a risk that the subject may move their face away from the screen if they are not interested in the content. By making the content included in the content information 252 interactive content that requires action, such as a questionnaire, the above risk can be suppressed. However, the content information 252 may also include static content such as news articles on current events.

[0028] Furthermore, in order to regularly track the health status of the estimated subject, it is desirable to periodically estimate their biological state. For this reason, it is desirable that content information 252 includes content that will not become tiresome even if displayed regularly. For example, content information 252 may include questionnaires (content) that are appropriate to the time, date, day of the week, season, etc. Content information 252 may also include questionnaires that are appropriate to current events. For example, content information 252 may include a questionnaire that includes news articles that reflect current events, and questions asking whether or not the person is interested in the article.

[0029] For example, as described above, content information 252 includes content that does not affect the biological state of the estimated subject as much as possible. Furthermore, as will be described later, in this disclosure, the estimated results of the biological information can be used when making product recommendations, etc. Therefore, content information 252 may include questionnaires (content) that can be used to determine the user's lifestyle, preferences, purchasing tendencies, and other information that can be used when deciding which products to recommend.

[0030] The estimation result information 253 includes information corresponding to the estimation result by the biological state estimation unit 262. The estimation result information 253 may also include information indicating the date and time the estimation was performed, and information corresponding to the estimation result. Furthermore, the estimation result information 253 may include information corresponding to product recommendations determined based on the estimation result. The estimation result information 253 can be updated in response to the biological state estimation unit 262 performing an estimation, the judgment unit 266 determining which products to recommend, and so on.

[0031] The arithmetic processing unit 260 includes a arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 260 reads and executes a program 254 from the storage unit 250, thereby realizing various processing functions by coordinating the hardware and the program 254. Examples of the main processing functions realized by the arithmetic processing unit 260 include an image acquisition unit 261, a biological state estimation unit 262, a content acquisition unit 263, a content display unit 264, a display end determination unit 265, a judgment unit 266, and an output unit 267. Note that the estimation of the biological state may be performed by an external device.

[0032] The arithmetic processing unit 260 may have a GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof, instead of the CPU described above.

[0033] The image acquisition unit 261 acquires image data acquired by the imaging unit 240 from the imaging unit 240. For example, the image acquisition unit 261 acquires image data from the imaging unit 240 that at least includes the face of the estimated subject as the subject. The imaging unit 240 may also acquire time-series image data. The image acquisition unit 261 also stores the acquired image data as image data information 251 in the storage unit 250. The image acquisition unit 261 may also store in the storage unit 250 the time at which the imaging unit 240 acquired the image data, etc., in association with the image data.

[0034] The biological state estimation unit 262 estimates the biological state using the image data contained in the image data information 251. As described above, the biological state estimation unit 262 may obtain arbitrary biological information by performing arbitrary processing on the image data. For example, the biological state estimation unit 262 can start estimating the biological state in response to an operation on the operation input unit 210. The biological state estimation unit 262 also stores the estimated result of the biological state as estimation result information 253 in the storage unit 250.

[0035] For example, the biological state estimation unit 262 can estimate pulse waves and pulse rate using luminance data acquired from time-series image data. Furthermore, the biological state estimation unit 262 may estimate information indicating arbitrary vital states or emotions by inputting time-series image data into a trained model. In addition to the methods exemplified above, the biological state estimation unit 262 may acquire biological information using any other method.

[0036] The content acquisition unit 263 acquires content. For example, the content acquisition unit 263 acquires content by referring to content information 252. In this case, the content acquisition unit 263 can acquire content corresponding to the date and time (current time) when the biological state is estimated by referring to the date and time referred to. The content acquisition unit 263 may also be configured to acquire content corresponding to the external environment such as temperature and weather around the estimation device 200.

[0037] The content acquisition unit 263 may acquire content from an external device by accessing the external device via the communication interface unit 230. For example, the content acquisition unit 263 can acquire content such as news articles or current events corresponding to the date and time (current time) for estimating the biological state. The content acquisition unit 263 may also acquire any other content, such as advertisements. Furthermore, the content acquisition unit 263 can update the content information 252 according to the acquisition results.

[0038] Furthermore, the content acquisition unit 263 may acquire content at any time, not just at the start timing of biological state estimation by the biological state estimation unit 262. For example, the content acquisition unit 263 can acquire content to be displayed in response to the start of biological state estimation by the biological state estimation unit 262. In addition, the content acquisition unit 263 may acquire content from external devices or the like at predetermined times or intervals.

[0039] The content display unit 264 displays the content to be displayed. For example, the content display unit 264 can start displaying the content at any time after the biological state estimation unit 262 has started estimating the biological state.

[0040] Figure 3 shows an example of display by the content display unit 264. Referring to Figure 3, the content display unit 264 displays content such as a questionnaire 221 on the screen display unit 220. The content display unit 264 can also terminate the display of content depending on the result of a determination by the display termination determination unit 265. For example, the content display unit 264 may terminate the display of content by not displaying any new questions, depending on the display termination determination unit 265's determination to terminate the display. Alternatively, the content display unit 264 may be configured to immediately terminate the display of content such as a questionnaire by stopping the display of content depending on the display termination determination unit 265's determination to terminate the display. In other words, the content display unit 264 may terminate the display of content after waiting for answers to the currently displayed questionnaire (questions), depending on the display termination determination unit 265's determination to terminate the display, or it may terminate the display of content without waiting for answers. Furthermore, the content display unit 264 may be configured to terminate the display of content not only when the display termination determination unit 265 determines when the display has ended, but also when the status of the content, such as the status of the survey responses, meets predetermined conditions, or when any other conditions are met in addition to the display termination determination.

[0041] As will be described later, the display termination determination unit 265 determines the end of the display depending on the progress of the biological state estimation by the biological state estimation unit 262, such as when the estimation of the biological state has been completed. Therefore, as shown in Figure 4, the content display unit 264 can also determine whether or not to display the next questionnaire, or otherwise determine whether or not to display the content, depending on the progress of the biological state estimation.

[0042] Furthermore, as shown in Figure 5, the content display unit 264 may be configured to display content such as the questionnaire 221, as well as an avatar 222 indicating the orientation of the estimated subject's face, etc., relative to the imaging unit 240, on the screen display unit 220. By displaying an avatar 222, etc., instead of simply displaying an image of the estimated subject's face, etc., it is possible to determine whether or not the estimated subject's face, etc., is within the frame for estimating their biological state without making the estimated subject strongly aware that their image is being acquired. In addition, by displaying an avatar 222, etc., the privacy of the estimated subject can be protected more appropriately. The content display unit 264 may be configured to terminate the display of the avatar 222, etc., at the same time as the termination of the display of the content described above.

[0043] The display termination determination unit 265 determines whether or not to terminate the display of the content. For example, the display termination determination unit 265 may determine whether or not to terminate the display of the content depending on the progress of the biological state estimation by the biological state estimation unit 262. As an example, the display termination determination unit 265 may make a display termination determination indicating that the display of the content will be terminated when the biological state estimation by the biological state estimation unit 262 is completed.

[0044] The determination unit 266 determines products, experiences, etc. to be recommended (referred) to the person to be estimated by using at least one of the result of the biological state estimation by the biological state estimation unit 262 and the result of content display such as the answer to the questionnaire displayed by the content display unit 264. By making a recommendation based on the biological state estimated in a state where stress is suppressed by the display of content or the like and the result of content display, the determination unit 266 can recommend products and experiences more suitable for the person to be estimated.

[0045] As an example, after identifying candidates such as products and experiences according to the result of the biological state estimation, the determination unit 266 can select the products and experiences actually recommended from the identified candidates according to the answer to the questionnaire or the like. For example, the determination unit 266 identifies products, candidates, etc. according to the result of the biological state estimation itself or any information such as the presence or absence of stress that can be determined according to the estimation result. The determination unit 266 may identify, as candidates, foods and beverages for relieving stress such as chocolate according to the result of the biological state estimation, and experiences for reducing stress such as walking, jogging, and hot spring travel. Thereafter, the determination unit 266 selects, as the products and experiences actually recommended, the products and experiences that can be determined to be more suitable for the person to be estimated from the identified candidates according to the answer to the questionnaire or the like. For example, the determination unit 266 may select a product such as chocolate when the answer corresponding to the questionnaire or the like indicates that the person likes sweet things, and select products and experiences according to the questionnaire or the like. Note that the determination unit 266 may obtain any information other than the presence or absence of stress, such as at least one or a combination thereof of fatigue, lack of sleep, skin condition, and any other condition, by using the result of the biological state estimation. Further, the determination unit 266 may be configured to select products and the like by processes other than those exemplified above, such as selecting the products to be recommended according to the result of the biological state estimation after identifying products and the like according to the result of the questionnaire or the like. Further, the determination unit 266 may make a recommendation on content other than those exemplified above, such as the type of facility to be visited and the type of experience to be performed.

[0046] For example, as described above, in addition to the result of the biological state estimation by the biological state estimation unit 262, the determination unit 266 makes recommendations using the result of content display and the like. By making such recommendations, the determination unit 266 can make more suitable recommendations for the person to be estimated than simply making recommendations using the estimation result of the biological state. Note that the types of products and experiences to be recommended by the determination unit 266 may be determined according to, for example, the location, time, and any other environment where the estimation device 200 is used. In other words, the determination unit 266 can recommend products and the like according to the location where the estimation device 200 is used or the installation location.

[0047] The output unit 267 outputs the processing results of each processing unit and the like. FIG. 6 shows an example of the information output by the output unit 267. Referring to FIG. 6, the output unit 267 displays at least a part of the biological state estimation result 223, the content display result 224 such as the questionnaire result (answer to the questionnaire), the information 225 about the product to be recommended, etc. on the screen display unit 220. Note that the output unit 267 may output at least a part of the above various information to an external device via the communication interface unit 230.

[0048] The above is a configuration example of the estimation device 200. Subsequently, referring to FIG. 7, an operation example of the estimation device 200 will be described.

[0049] FIG. 7 is a flowchart showing an operation example of the estimation device 200. Referring to FIG. 7, the biological state estimation unit 262 starts the estimation of the biological state using the image data included in the image data information 251 (step S101).

[0050] The content display unit 264 displays the content to be displayed (step S102). For example, the content display unit 264 can start displaying the content at an arbitrary timing after the processing of step S101.

[0051] The display termination determination unit 265 determines whether or not to terminate the display of the content according to the progress of the biological state estimation by the biological state estimation unit 262 (step S103). If the display is not terminated (step S103, NO), the content display unit 264 continues to display the content, such as by displaying a new questionnaire. On the other hand, if the display is terminated in accordance with the completion of the estimation, etc. (step S103, YES), the content display unit 264 terminates the display of the content by not displaying new questions corresponding to the answers (step S104).

[0052] The determination unit 266 uses at least one of the results of the biological state estimation by the biological state estimation unit 262 and the results of the content display, such as responses to questionnaires displayed by the content display unit 264, to determine which products or experiences to recommend to the estimated target person (step S105).

[0053] The output unit 267 outputs the processing results of each processing unit (step S106). The output unit 267 may output at least a portion of the following: the estimated biological state, the content display results such as questionnaire results, and information about recommended products.

[0054] The above is an example of the operation of the estimation device 200.

[0055] Thus, the estimation device 200 includes a biological state estimation unit 262, a content display unit 264, and a display termination determination unit 265. With this configuration, the content display unit 264 can display content in response to the start of biological state estimation by the biological state estimation unit 262. The content display unit 264 can also terminate the display of content based on the display termination determination unit 265's determination of the end of display according to the progress of the biological state estimation. As a result, the content display unit 264 can display content while the biological state estimation unit 262 is performing the estimation. This makes it possible to guide the subject to be estimated to keep their face and other features as still as possible during biological state estimation without making them aware of it, thereby reducing the risk of causing discomfort to the subject during biological state estimation.

[0056] Furthermore, the estimation device 200 includes a judgment unit 266. With this configuration, the judgment unit 266 can select products and experiences to recommend using the estimated biological state results as well as the content display results. As a result, the content display results can be effectively utilized, enabling recommendations that are more suitable for the estimated target person.

[0057] Note that the configuration of the estimation system 100 is not limited to the example shown in Figure 1. For example, Figure 8 shows another example of the configuration of the estimation system 100. Referring to Figure 8, the estimation device 200 included in the estimation system 100 may be a fixed information processing device or the like that is pre-installed in a retail store such as a convenience store or supermarket, or any other store.

[0058] In the example shown in Figure 8, the estimation device 200 can display content appropriate to the location where it is installed. For example, if the estimation device 200 is installed in a retail store, the content information 252 may include a questionnaire to narrow down the products to recommend from among the products sold in the store. As an example, the content information 252 may include a questionnaire with questions to narrow down the products, such as "Are you thirsty?", "Are you hungry?", or "Do you want something sweet?".

[0059] Furthermore, in the example shown in Figure 8, the judgment unit 266 may be configured to identify and select a recommended product from among the products currently on sale at the installation location, such as a retail store, where the estimation device 200 is installed. For example, the judgment unit 266 can select a recommended product from among food and beverage products such as sweets, based on the estimated biological state and the content display results. In addition, the output unit 267 may be configured to output related information 226 related to the recommended product, as shown in Figure 9. For example, the output unit 267 can display the location of the shelf where the recommended product is placed, the price of the product, and any other arbitrary information related to the product as related information 226.

[0060] [Second Embodiment] Next, a processing device 300, which is a modified version of the estimation device 200, will be described with reference to Figures 10 and 11. Figure 10 is a diagram showing an example of the hardware configuration of the processing device 300. Figure 11 is a block diagram showing an example of the configuration of the processing device 300.

[0061] The processing unit 300 is an information processing device that has a display device capable of displaying predetermined content. Figure 10 shows an example of the hardware configuration of the processing unit 300. Referring to Figure 10, the processing unit 300 has, as an example, the following hardware configuration: ・CPU (Central Processing Unit) 301 (arithmetic unit) ・ROM (Read Only Memory) 302 (storage device) ・RAM (Random Access Memory) 303 (storage device) ・Program group 304 loaded into RAM 303 ・Storage device 305 that stores the program group 304 ・Drive device 306 that reads and writes to a recording medium 310 outside the information processing device ・Communication interface 307 that connects to a communication network 311 outside the information processing device ・Input / output interface 308 that performs data input and output ・Bus 309 that connects each component

[0062] Furthermore, the processing unit 300 can realize the function of the content display unit 321 shown in Figure 11 by having the CPU 301 acquire the program group 304 and execute it. The program group 304 is, for example, stored in advance in the storage device 305 or ROM 302, and the CPU 301 loads it into the RAM 303 or the like and executes it as needed. Alternatively, the program group 304 may be supplied to the CPU 301 via the communication network 311, or it may be stored in advance in the recording medium 310, and the drive device 306 may read the program and supply it to the CPU 301.

[0063] Figure 10 shows an example of the hardware configuration of the processing unit 300. The hardware configuration of the processing unit 300 is not limited to the case described above. For example, the processing unit 300 may consist of only a part of the configuration described above, such as not having a drive device 306. Also, the CPU 301 may be a GPU as exemplified in the first embodiment.

[0064] The content display unit 321 displays predetermined content on the screen that is different from the information about the estimation of the biological state when estimating the biological state of a subject using image data that at least includes the subject's face. The content display unit 321 also terminates the display of content according to the progress of the biological state estimation.

[0065] Thus, the processing unit 300 has a content display unit 321. With this configuration, the content display unit 321 can display content and terminate the display of content according to the progress of the biological state estimation. This makes it possible to guide the subject to be estimated to keep their face and other features as still as possible during the biological state estimation without making them aware of it. As a result, the processing unit 300 can reduce the risk of causing discomfort to the subject during the biological state estimation.

[0066] The above-described processing device 300 can be realized by incorporating a predetermined program into the information processing device such as the processing device 300. Specifically, another form of the program described herein is a program for implementing a process in the information processing device such as the processing device 300 that, when estimating the biological state of a subject using image data in which the subject's face is at least the subject, displays predetermined content on the screen that is different from the information regarding the estimation of the biological state, and terminates the display of the content according to the progress of the biological state estimation.

[0067] Furthermore, the processing method performed by the information processing device such as the processing device 300 described above is a method in which, when the information processing device such as the processing device 300 estimates the biological state of the subject using image data in which the subject's face is at least the subject, it displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of the content according to the progress of the estimation of the biological state.

[0068] Even a program having the above-described configuration, or a recording medium readable by a computer containing the program, or an estimation method, can achieve the same effects and benefits as the processing device 300 described above, and thus the objectives of this disclosure described above can be achieved.

[0069] <Note> Some or all of the above embodiments may also be described as follows. The general outline of the processing apparatus, etc., in this disclosure is described below. However, this disclosure is not limited to the following configuration.

[0070] (Note 1) A processing device having a content display unit that displays predetermined content on the screen that is different from the information about the estimation of the biological state when estimating the biological state of a person using image data in which the face of the person to be estimated is the subject, wherein the content display unit terminates the display of the content according to the progress of the estimation of the biological state. (Note 2) The processing device according to Note 1, wherein the content display unit is configured to display a questionnaire requesting answers to questions as the content, and terminates the display of the content by not displaying new questions according to the progress of the estimation of the biological state. (Note 3) The processing device according to Note 2, having a judgment unit that recommends a predetermined product or experience using the estimation result of the biological state and the answers to the questionnaire displayed as content, and an output unit that outputs at least a part of the estimation result of the biological state, the answers to the questionnaire and the product or experience recommended by the judgment unit. (Note 4) The processing device according to any one of Notes 1 to 3, having a determination unit that determines whether or not to terminate the display of content according to the progress of biological state estimation, wherein the determination unit is configured to make a determination to terminate the display of content when the estimation of biological state is completed, and the content display unit terminates the display of content in accordance with the determination of termination of display by the determination unit. (Note 5) The processing device according to Note 4, wherein the content display unit terminates the display of content in accordance with predetermined conditions in addition to the determination of termination of display by the determination unit. (Note 6) The processing device according to any one of Notes 1 to 5, wherein the content display unit displays an avatar indicating the direction of the face of the person to be estimated along with the content on the screen. (Note 7) The processing device according to any one of Notes 1 to 6, wherein the content display unit displays content that takes into account the effect on the biological state after the estimation of the biological state of the person to be estimated has started. (Note 8) The processing device according to any one of Notes 1 to 7, wherein the content display unit displays interactive content that requires screen operation as the content.(Note 9) The processing device according to Note 3, wherein the determination unit identifies candidate products or experiences using the results of the estimation of the biological state, and then selects a recommended product or experience from among the identified products or experiences using the answers to the questionnaire. (Note 10) The processing device according to any one of Notes 1 to 9, wherein the estimation unit has an estimation unit that estimates the biological state of the person to be estimated using image data in which the face of the person to be estimated is at least the subject, and the content display unit terminates the display of content according to the progress of the biological state estimation by the estimation unit. (Note 11) A processing method in which an information processing device estimates the biological state of the person to be estimated using image data in which the face of the person to be estimated is at least the subject, displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of content according to the progress of the biological state estimation. (Note 12) A computer-readable recording medium that records a program for implementing the process of an information processing device estimating the biological state of the person to be estimated using image data in which the face of the person to be estimated is at least the subject, displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of content according to the progress of the biological state estimation.

[0071] Furthermore, some or all of the configurations described in Appendices 2 to 10, which are dependent on the processing device described in Appendice 1, may also be dependent on the processing method described in Appendice 11, the recording medium described in Appendice 12, etc., through a similar dependency relationship. Moreover, not limited to Appendices 11 and 12, some or all of the configurations described in the appendices may also be dependent on various hardware, software, various recording means, methods, programs, or systems for recording software, without departing from the embodiments described above.

[0072] The programs described in each of the above embodiments and appendices can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0073] Although the present disclosure has been described above with reference to the embodiments described above, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0074] 100 Estimation system 200 Estimation device 210 Operation input unit 220 Screen display unit 230 Communication interface unit 240 Imaging unit 250 Storage unit 251 Image data information 252 Content information 253 Estimation result information 254 Program 260 Calculation processing unit 261 Image acquisition unit 262 Biological state estimation unit 263 Content acquisition unit 264 Content display unit 265 Display end determination unit 266 Judgment unit 267 Output unit 300 Processing unit 301 CPU 302 ROM 303 RAM 304 Program group 305 Storage device 306 Drive device 307 Communication interface 308 Input / output interface 309 Bus 310 Recording medium 311 Communication network 321 Content display unit

Claims

1. A processing device having a content display unit that displays predetermined content on the screen that is different from information about the estimation of the biological state when estimating the biological state of a subject using image data in which the subject's face is at least the subject, wherein the content display unit terminates the display of the content according to the progress of the biological state estimation.

2. The processing apparatus according to claim 1, wherein the content display unit is configured to display a questionnaire requesting answers to questions as the content, and the display of content is terminated by not displaying new questions according to the progress of the biological state estimation.

3. The apparatus according to claim 2, comprising: a judgment unit that recommends a predetermined product or experience using the estimated results of the biological state and the answers to a questionnaire displayed as content; and an output unit that outputs at least a portion of the estimated results of the biological state, the answers to the questionnaire, and the product or experience recommended by the judgment unit.

4. The processing apparatus according to claim 1, comprising a determination unit that determines whether or not to terminate the display of content according to the progress of biological state estimation, wherein the determination unit is configured to make a determination to terminate the display of content when the estimation of biological state is completed, and the content display unit terminates the display of content in accordance with the determination of termination by the determination unit.

5. The processing apparatus according to claim 4, wherein the content display unit terminates the display of the content in accordance with the determination of termination by the determination unit, or in accordance with the status of the content meeting predetermined conditions.

6. The processing apparatus according to claim 1, wherein the content display unit displays an avatar indicating the orientation of the face of the estimated subject along with the content.

7. The processing apparatus according to claim 1, wherein the content display unit displays content that takes into account the effect on the biological state of the person to be estimated after the estimation of the biological state of the person to be estimated has started.

8. The processing apparatus according to claim 1, wherein the content display unit displays interactive content that requires screen operation as the content.

9. The processing apparatus according to claim 3, wherein the determination unit identifies candidate products or experiences using the estimated results of the biological state, and then selects a recommended product or experience from among the identified products or experiences using the responses to the questionnaire.

10. The apparatus according to claim 1, comprising an estimation unit that estimates the biological state of a subject using image data in which at least the face of the subject is the subject, wherein the content display unit terminates the display of content according to the progress of the biological state estimation by the estimation unit.

11. A processing method in which an information processing device estimates the biological state of a person using image data in which the face of the person to be estimated is at least one subject, displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of the content according to the progress of the estimation of the biological state.

12. A computer-readable recording medium containing a program for an information processing device that, when estimating the biological state of a subject using image data in which the subject's face is at least the subject, displays predetermined content on the screen that is different from the information about the estimation of the biological state, and terminates the display of the content according to the progress of the estimation of the biological state.

Images