Predictors of long-term affective responses to sensory stimuli

Abstract

This invention discloses a method for predicting an individual's long-term emotional response to at least one predetermined sensory stimulus. The method includes the steps of (a) exposing the individual to at least one sensory stimulus a number of times within an initial exposure period according to an exposure pattern; (b) for each exposure, obtaining data representing the individual's emotional response to the at least one sensory stimulus; (c) providing the data representing the individual's emotional response and the exposure pattern to a machine learning algorithm; and (d) at a predetermined prediction period following the initial exposure period, using the machine learning algorithm to predict the individual's long-term emotional response to the sensory stimulus.

Description

Technical Field

[0001] The present invention generally relates to a method of predicting a long-term affective response, such as a long-term liking, of an individual or audience to at least one predetermined sensory stimulus, such as an olfactory stimulus. Background Art

[0002] An individual's first exposure to a given sensory stimulus may elicit a certain affective or emotional response. However, the same sensory stimulus may not necessarily produce the same affective response in subsequent exposures, such that the initial affective response to a given sensory stimulus is generally not a reliable indicator of the affective response to the same sensory stimulus at a later time. Rather, affective responses to a given sensory stimulus vary over time and with repeated exposure. However, the evolution of affective responses over time is generally unknown in advance.

[0003] The purpose of the present invention is to estimate long term affective responses so that product development can be focused on manufacturing consumer products with sensory stimulation that have affective responses that will improve or at least not decline over time. In this way, time and energy spent on the development and eventual manufacture of consumer products that have a considerable risk of not being put into use and ultimately failing can be reduced.

[0004] It is therefore an object of the present invention to provide a method allowing streamlining the development of consumer products comprising at least one sensory stimulus, thereby avoiding waste and saving resources. Summary of the invention

[0005] The present invention relates to a computer-implemented method for predicting an individual's long-term emotional response to at least one predetermined sensory stimulus, comprising (a) exposing the individual to at least one predetermined sensory stimulus several times within an initial time period of exposure according to an exposure pattern, (b) obtaining data representing the individual's emotional response to the at least one predetermined sensory stimulus for each exposure, (c) providing the data representing the individual's emotional response and the exposure pattern to a machine learning algorithm, and (d) predicting the individual's long-term emotional response to the sensory stimulus by means of a machine learning algorithm at a time point within a predetermined prediction time period after the initial time period of exposure.

[0006] Thus, the method allows the use of machine learning algorithms to predict an individual's affective response to a given sensory stimulus at a specific time after the first exposure to the given sensory stimulus. In a professional setting, an individual will be exposed to a certain sensory stimulus not once, but multiple times during an initial period of exposure. The number of exposures, the duration of each exposure, and the time interval between consecutive exposures during the initial period of exposure determine the exposure pattern.

[0007] In certain embodiments, the individual is exposed to sensory stimulation at least twice, preferably three times or more, within a relatively short initial period of exposure. For example, the individual can be exposed to sensory stimulation twice or three times within a week. In other embodiments, the individual can be exposed to a series of sensory stimulations within a short period of time, such as one hour, such as three times, and this can be repeated multiple times, such as twice or three times, within the initial period of exposure, such as within a week. The initial period of exposure is preferably one week or shorter. The predetermined prediction period is preferably three months or longer.

[0008] When an individual is exposed to a sensory stimulus during an initial period of exposure, the individual's long-term affective response to the same sensory stimulus is the individual's affective response to the sensory stimulus when a predicted period of time has passed as measured from the end of the initial period of exposure. In one embodiment, the individual's affective response to the sensory stimulus is the individual's liking for the sensory stimulus. However, other measures of long-term affective response are possible.

[0009] The exposure pattern can have an impact on an individual's long-term affective response to a predetermined sensory stimulus. Therefore, when training a machine learning algorithm, training data with different numbers of exposures, intervals between subsequent exposures, and initial time periods of exposure should be provided.

[0010] At least one predetermined sensory stimulation can be selected from the group including smell, hearing, touch, taste and visual stimulation. Individual can be exposed to only a single sensory stimulation, or be exposed to two or more sensory stimulations that can be for the same sensory organ or different sensory organs. For example, an individual may be exposed to olfactory stimulation without any further sensory stimulation. If the emotional response of an individual is to be tested for two or more sensory stimulations, two or more sensory stimulations are presented to each individual simultaneously, so that each exposure can be associated not only with a sensory stimulation, but also with the simultaneous occurrence of two or more sensory stimulations. For example, an individual can be exposed to olfactory stimulation and visual stimulation simultaneously. Another combination of sensory stimulations that an individual may be exposed to is taste stimulation and visual stimulation. Other combinations of sensory stimulations are also possible. When referring to sensory stimulations, this should be understood as also relating to a combination of multiple sensory stimulations, unless it is inappropriate.

[0011] The emotional response of the individual may be determined by obtaining the individual's psychological response to at least one predetermined sensory stimulus.Preferably, the individual's psychological response is combined with data of the individual's psychometric response to the sensory stimulus.

[0012] Data representing the individual's affective response to at least one predetermined sensory stimulus is fed into a machine learning algorithm. Preferably, the machine learning algorithm has been previously trained, cross-validated, and tested using a large dataset previously collected on a large number of individuals and instances of sensory stimuli. The machine learning algorithm can generate predictions on an individual level for a given sensory stimulus.

[0013] The present invention also relates to a computer-implemented method for predicting a long-term emotional response of an audience to at least one predetermined sensory stimulus. The above method performed at the individual level is repeated for a plurality of different individuals representing a large audience, such as a consumer group. The predictions of the long-term emotional response for each individual are then combined to predict the long-term emotional response of the audience.

[0014] The invention also relates to a data processing device comprising means for executing the above method, a computer program comprising instructions, which, when a computer executes the program, cause the computer to execute the above method, and a computer-readable medium comprising instructions, which, when a computer executes the instructions, cause the computer to execute the above method.

[0015] The present invention also relates to a method for designing a consumer product, the method comprising the steps of performing the above-mentioned method implemented by a computer, the method predicting the long-term emotional response of an individual or audience to at least one predetermined sensory stimulus from a plurality of sensory stimuli, and determining the sensory stimulus with the maximum long-term emotional response from the plurality of sensory stimuli, the method also comprising the step of designing a consumer product containing the determined sensory stimulus. Statistical methods, such as standard significance tests, can be used to determine or confirm the sensory stimulus with the maximum long-term emotional response from the plurality of sensory stimuli. The maximum long-term emotional response can be the long-term emotional response with the highest value, which represents the strongest positive emotional response to a sensory stimulus.

[0016] The present invention also relates to a system configured to predict an individual's long-term emotional response to at least one predetermined sensory stimulus. The system includes a measuring unit configured to measure data representing the individual's emotional response to the at least one predetermined sensory stimulus for each exposure in which the individual is exposed to the at least one predetermined sensory stimulus several times within an initial time period of exposure according to an exposure pattern; and a control unit configured to predict the individual's long-term emotional response to the sensory stimulus by a machine learning algorithm for a time point of a predetermined prediction time period after the initial time period of exposure based on the data representing the individual's emotional response measured by the measuring unit and the exposure pattern. The measuring unit may include at least one of an EEG (electroencephalogram) measuring device, a GSR (galvanic skin response) measuring device, an eye tracker, and a video recording device. The EEG signal measuring device is generally configured to preferably non-invasively measure the electrical activity of the individual's brain. Other measuring devices for measuring brain activity may also be used, an example of which is fMRI (functional magnetic resonance imaging). The GSR signal measuring device is generally configured to measure the skin conductance of the individual. The eye tracker is generally configured to measure at least one of the individual's eye position, eye movement, gaze point, and pupil dilation and contraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart illustrating the method according to the invention is shown.

[0018] Figure 2 A flow chart illustrating a method for training, cross-validating and testing a machine learning algorithm used in the method according to the present invention is shown.

[0019] Figure 3 A system for predicting the long-term affective response of an individual or audience to at least one predetermined sensory stimulus is shown.

[0020] Figure 4 The raw EEG signal recorded from one electrode within 0.5 seconds is shown.

[0021] Figure 5 The power spectrum of the EEG data of one electrode calculated using a fast Fourier transform over a 2 second time window using a Hanning window function is shown.

[0022] Figure 6 A 15 s GSR recording is shown. DETAILED DESCRIPTION

[0023] A computer-implemented method for predicting a long-term emotional response to at least one predetermined sensory stimulus is described. Specifically, the method described herein includes predicting a long-term emotional response of an individual to at least one predetermined sensory stimulus, and predicting a long-term emotional response of an audience to at least one predetermined sensory stimulus. The audience includes a large number of individuals.

[0024] Figure 1 The flowchart in shows a method for predicting an individual's long-term affective response to at least one predetermined sensory stimulus. The method includes exposing an individual to a predetermined sensory stimulus for the first time during an initial period of exposure (110). The individual may be exposed to the sensory stimulus only once, so that the initial period of exposure has the length of a single exposure. In order to achieve a more reliable prediction, preferably, the individual is repeatedly exposed to the sensory stimulus during the initial period of exposure. For example, the individual is exposed to the sensory stimulus three times within a week. During each exposure, the individual's physiological response is measured, such as from EEG, GSR, eye tracking and / or facial video (120). Additionally or alternatively, the individual's psychological response is measured during each exposure, such as from a self-report and / or reaction time test (140). After removing noise from the physiological data (130) and calibrating and cleaning the psychometric data (150), significant features are extracted from the data (160). The extracted salient features are then used as input to an artificial neural network (170), which, when appropriately trained, cross-validated, and tested, produces predictions regarding the individual's long-term affective response to the predetermined sensory stimulus (180). The predicted long-term affective response is typically represented as a value ranging from negative infinity to positive infinity, where values ​​below 0 represent a negative affective response, 0 represents no difference, and values ​​above 0 represent a positive affective response to the particular sensory stimulus.

[0025] Figure 2 shows a method for training, cross validation and testing Figure 1 Flowchart of a method for an artificial neural network used in the method. The present invention is not limited to neural networks, and other machine learning algorithms may be used. Training data (210) is obtained by exposing an individual to a predetermined sensory stimulus during an initial period of exposure. The physiological and / or psychological responses of the individual are measured during each exposure (220, 240). The psychometric data is free of noise (230), so that significant features (260) can be extracted. Similarly, after calibration and data cleaning (250), significant features are extracted from the psychometric data (260). These steps are repeated for a large number of individuals, with different initial period lengths of exposure and exposure patterns, and for various sensory stimuli. The significant features extracted from each initial period of exposure (involving a specific individual and a specific sensory stimulus or combination of sensory stimuli) are input into the artificial neural network together with the corresponding exposure pattern for training purposes.

[0026] After a predetermined prediction period, preferably no less than 3 months after the initial period of exposure, further data is collected for each previously tested individual (280). In this way, data on long-term emotional responses are obtained, which can be used to train the machine learning algorithm. For example, for each individual, the actual purchase or repurchase behavior of consumer products with sensory stimuli or sensory stimuli combinations used during the initial period of exposure is observed (290). Each individual can also be exposed again to the same sensory stimuli or sensory stimuli combination encountered during the initial period of exposure, and data indicating emotional responses are obtained. Additionally or alternatively, self-reports of preferences for consumer products and / or self-reports of purchase intentions are performed, and the consumer products are combined with sensory stimuli / stimuli factors and / or selection experiments that include such consumer products (300).

[0027] When data on long-term affective responses have not been collected in a uniform manner, i.e., different explicit variables are measured while attempting to assess the inherent latent variables related to long-term affective responses, in order to still be able to utilize these different types of measures of long-term affective responses for training purposes, they need to be reconciled. This is accomplished by forming latent variable models from the research in which multiple modes of assessing long-term affective responses are used together. These models provide a way to estimate latent variable scores from the measurements available in the training data.

[0028] Alternatively, to avoid the need for coordination, data on long-term affective responses can be collected in a uniform manner across subjects, and these measurements used directly to train the machine learning algorithm.

[0029] During the training process, both input (the extracted features obtained from step 260) and the desired output (the long-term emotional response obtained in step 310) are provided, and the artificial neural network processes the input and compares the resulting output with the desired output. After the training process is considered complete, some of the data obtained during steps 220, 240, 290 and 300 are used to test the trained artificial neural network. As known to those skilled in the art, the artificial neural network is also cross-validated.

[0030] The following explains in more detail how to use Figure 3 System 400 is shown for collecting psychological data and extracting features from such data.

[0031] The raw EEG signals are collected using an EEG signal measuring device 410. In one embodiment, the EEG signal measuring device 410 has 14 to 20 electrodes or channels and a sampling rate between 128 Hz and 500 Hz. Figure 4An exemplary raw EEG signal recorded from three electrodes at positions F7, Fp1, and Fp2 within 0.5 seconds is shown in FIG.

[0032] Additionally or alternatively, raw EDA (electrodermal activity) signals are collected using an EDA signal measurement device, such as the GSR signal measurement device 420. In one embodiment, the GSR signal measurement device 420 has one electrode or channel and a sampling rate between 5 Hz and 128 Hz. Figure 6 An exemplary raw GSR signal recorded over 15 seconds is shown in FIG.

[0033] Additionally or alternatively, an eye tracker 430 is used to collect raw eye tracking signals. In one embodiment, the eye tracking signals are 2D Cartesian coordinates relative to the screen. The sampling rate may be 30 Hz.

[0034] Additionally or alternatively, a video recording device 440, such as a webcam, is used to capture video signals of individuals, preferably faces. The video signals may have various resolutions and sampling rates.

[0035] When measuring the feelings or emotional reactions to sensory stimulation, it is important to know exactly when the individual begins to experience sensory stimulation. Therefore, in a preferred embodiment, once the raw data signals have been collected, they are synchronized. Although visual stimulation can be easily controlled or timed when it is displayed on the screen, for olfactory, taste or tactile stimulation, the present invention contemplates synchronizing the video signal of the individual with other data signals such as EEG, GSR or eye tracking signals. Synchronize the data signals by regularly checking the clock or timestamp of each data signal relative to the system clock of the control unit 450 that collects data. When any drift occurs between the data signals, adjustments can be made to eliminate the drift.

[0036] In one embodiment, during EEG signal processing, which may be performed by the control unit 450, low-pass and high-pass filters are applied to the raw EEG signal to remove signal components having frequencies above 50 Hz and below 0.5 Hz. Thereafter, an independent component analysis (ICA) is performed, and a machine learning algorithm matches the independent components to non-brain signals (e.g., eye blinks, head movements) and removes independent components identified as non-brain signals. For each channel, the EEG signal is then divided into overlapping time windows. Each time window may be 2 seconds long, with a new time window starting every 0.5 seconds. A Higuchi fractal dimension (HFD) is calculated for each time window. Further, for each time window, a Hanning window is applied and a fast Fourier transform (FFT) is calculated to obtain an EEG power spectrum. Figure 5An exemplary EEG power spectrum at one electrode during a two-second time window is shown in FIG.

[0037] In one embodiment, during the eye tracking post-processing that can be performed by the control unit 450, the raw data is divided into time windows representing the time that the slide image is continuously displayed on the screen. For each window, the Cartesian coordinates are used to generate a heat map (a matrix with the same dimensions as the image on the screen) using Gaussian kernel estimation. The heat map can be plotted on top of the image as the final output.

[0038] In one embodiment, during GSR post-processing, which may be performed by the control unit 450, a continuous decomposition analysis is applied to decompose the raw GSR signal into a phasic component (fast moving) and a tonic (slow moving) component. A peak detection algorithm may be used to determine a binary outcome at each time point in time from the raw GSR data: whether the individual is in an emotionally aroused state.

[0039] The measurement data thus processed together with the exposure pattern, preferably represented as a time vector (step function), are then used as input to a machine learning algorithm on the control unit 450, which outputs the estimated long-term affective response associated with said measurement data.

[0040] An exemplary table of predicted results for Fragrance A tested on a group of individuals is shown in Table 1. The average predicted long-term sentiment for the 16 individuals was 0.12, meaning that Fragrance A was predicted to be positively received long-term. However, on average, the sentiment response was not very significant, making it possible to determine that Fragrance A still needed to be improved in order to achieve a better long-term sentiment response.

[0041] individual Predicted long-term emotional response 1 0.27 2 0.18 3 -0.10 4 -0.23 5 0.41 6 0.21 7 0.83 8 0.08 9 0.99 10 -0.29 11 0.44 12 -0.36 13 -0.50 14 -0.65 15 0.21 16 0.47

Claims

1. A method for predicting an individual's long-term affective response to at least one predetermined sensory stimulus, comprising: (a) exposing the individual to the at least one predetermined sensory stimulus a number of times during an initial period of exposure according to an exposure pattern; (b) for each exposure, obtaining data representing the individual's affective response to the at least one predetermined sensory stimulus; (c) providing data representing the individual's emotional response and the exposure pattern to a machine learning algorithm; and (d) predicting, by the machine learning algorithm, the individual's long-term affective response to the sensory stimulus at a predetermined prediction time point after the initial time period of exposure, wherein step (a) comprises exposing the individual to the at least one predetermined sensory stimulus at least three times, Wherein, the predetermined predicted time period is at least 10 times longer than the initial time period of exposure, the initial time period of exposure is one week or shorter, and the predetermined predicted time period is three months or longer.

2. The method according to claim 1, wherein: The individual's long-term affective response is the individual's long-term liking for the at least one predetermined sensory stimulus.

3. The method according to claim 1, wherein: The machine learning algorithm is based on artificial neural networks.

4. The method according to claim 1, wherein: Said data representing said individual's affective response to said at least one predetermined sensory stimulus is obtained by measuring a psychological and / or psychometric response of said individual.

5. The method according to claim 1, wherein: The data representing the individual's affective response to the at least one predetermined sensory stimulus comprises data relating to brain activity of the individual.

6. The method according to claim 4, wherein: The psychological and / or psychometric responses of the individuals are measured simultaneously.

7. The method according to claim 4, wherein: Measuring the psychological response of the individual includes measuring data obtained from a panel including EEG, GSR, eye tracking and facial video.

8. The method according to any one of claims 1 to 6, wherein: The at least one predetermined sensory stimulation is selected from the group consisting of olfactory stimulation, auditory stimulation, tactile stimulation, taste stimulation and visual stimulation.

9. A method for predicting a long-term emotional response of an audience to at least one predetermined sensory stimulus using the method of any one of claims 1 to 8, comprising: Repeating steps (a) through (d) of claim 1 for each individual in a group of individuals, wherein the number of individuals in the group of individuals is less than the number of individuals in the audience; and The long-term emotional response of the individual group to the at least one predetermined sensory stimulus is derived based on the long-term emotional response of each individual to predict the long-term emotional response of the audience.

10. A method of designing a consumer product, the method comprising: Performing the method according to any one of claims 1 to 9 for multiple sensory stimuli; determining a sensory stimulus from the plurality of sensory stimuli that has the greatest long-term emotional response; and Design consumer products that include stimulation of the identified sensory organs.

11. A system (400) configured to predict a long-term affective response of an individual to at least one predetermined sensory stimulus, the system comprising: a measuring unit (410-440) configured to measure data representing an affective response of the individual to at least one predetermined sensory stimulus for each exposure in which the individual is exposed to the at least one predetermined sensory stimulus a number of times within an initial period of exposure according to an exposure pattern; as well as a control unit (450) configured to predict the individual's long-term emotional response to the sensory stimulus at a predetermined prediction time point after the initial time period of the exposure by a machine learning algorithm based on the data representing the individual's emotional response measured by the measuring units (410-440) and the exposure pattern, wherein the individual is exposed to the at least one predetermined sensory stimulus at least three times, Wherein, the predetermined predicted time period is at least 10 times longer than the initial time period of exposure, the initial time period of exposure is one week or shorter, and the predetermined predicted time period is three months or longer.

12. The system according to claim 11, wherein: The control unit (450) is further configured to derive the long-term emotional response of the individual group to the at least one predetermined sensory stimulus based on the long-term emotional response of each individual to predict the long-term emotional response of the audience, wherein the number of individuals in the individual group is smaller than the number of individuals in the audience.

13. The system according to claim 11 or 12, wherein: The measuring unit (410-440) includes at least one of the group consisting of an EEG signal measuring device (410), a GSR signal measuring device (420), an eye tracker (430), and a video recording device (440).

Images