Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program for promoting synchronization between a user's subjective bodily sensation and an external change.SOLUTION: An information processing method in an information processing system includes acquiring a target image captured by an imaging device that moves with movement of a user, and a biological signal representing a state of a body of the user and characterized by a specific frequency indicating the state of the body, and generating acoustic information audible to the user and a display image visible to the user on the basis of a complexity parameter representing complexity of a change around the user and the specific frequency of the biological signal when the biological signal is acquired. The tempo of the music provided by the acoustic information is defined according to a specific tempo defined by a specific frequency, and the display image is displayed in a different mode according to the complexity parameter calculated from the target image and is generated so as to temporarily change at a specific timing synchronized with the specific tempo.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology that can eliminate time dependency such as playback time by outputting a sound with a completely different structure from the original sound, interacting with the listener and the listening environment, etc., to enable sound reproduction that is rich in chance, surprise, and interactivity. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 06-102877 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even if such audio is output, if a mismatch occurs between the user's bodily sensations and external changes, the synchronization of the user's subjective bodily sensations may be lost. Therefore, there is still room for improvement in technology for promoting synchronization between the user's subjective bodily sensations and external changes. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system comprising at least one processor configured to execute a program to perform the following steps: in the acquisition step, a target image captured by an imaging device that moves in accordance with the movement of a user and a biosignal representing the user's physical state are acquired, the biosignal being characterized by a specific frequency indicating the physical state; in the generation step, when the biosignal is acquired, acoustic information that can be heard by the user and a display image that can be seen by the user are generated based on a complexity parameter that represents the complexity of changes in the user's surroundings and the specific frequency of the biosignal, the tempo of the music provided by the acoustic information is determined according to a specific tempo determined by the specific frequency; and the display image is displayed in a different manner according to the complexity parameter calculated from the target image and is generated to temporarily change at a specific timing synchronized with the specific tempo.

[0006] This configuration can encourage synchronization between the user's subjective bodily sensations and external changes. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. FIG. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of an information processing device 8. [Figure 3] 1 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. FIG. [Figure 4] 10 is a diagram showing an example of a relationship between a specific tempo and a specific frequency of acoustic information when a biological signal is detected. FIG. [Figure 5] FIG. 10 is a diagram showing another example of the relationship between a specific tempo and a specific frequency of acoustic information when a biological signal is detected. [Figure 6] 10A and 10B are conceptual diagrams showing phonemes indicating a mora included in acoustic information, and modulation modes and timings of a display image. [Figure 7] 10A and 10B are diagrams illustrating an example of changes and modulation patterns of a display image. [Figure 8] This is a more specific example of the change and modulation of the display image shown in FIG. [Figure 9] 10A and 10B are diagrams illustrating other examples of changes and modulation modes of a display image. [Figure 10] This is a more specific example of the example shown in FIG. [Figure 11] 10A and 10B are diagrams illustrating an example of changes and modulation patterns of a display image. [Figure 12] This is a more specific example of the change and modulation of the display image shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0009] Incidentally, the program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] Furthermore, various information processing according to an embodiment may realize input and output corresponding to the input. Here, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited as long as an output is obtained as a result of the input. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression formula constructed using a statistical method), a trained model that has previously trained the correlation between input and output, or a large-scale language model that can output a desired result by inputting a prompt.

[0011] In one embodiment, a "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on a circuit in the broad sense.

[0012] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, processor, memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section explains the hardware configuration.

[0014] <Information Processing System 1> 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes, for example, a vehicle 2 as a moving object.

[0015] The vehicle 2 is configured to be operable while a passenger U, an example of a user, is aboard the vehicle 2. The vehicle 2 may be a regular passenger car or an industrial vehicle such as a forklift or a towing tractor. The vehicle 2 includes a seat 3, an on-board camera 4, an example of an imaging device, a display unit 5, a car audio 6, an example of an audio output unit, a sensor 7, and an information processing device 8.

[0016] The seat 3 is configured to accommodate a passenger U. The passenger U may be, for example, a driver who drives the vehicle 2, or a person who sits in the seat 3 corresponding to a passenger seat to assist the driver.

[0017] The on-board camera 4 is configured to move in accordance with the movement of the passenger U, and is configured to generate an on-board image by capturing an image of at least a portion of the surroundings of the vehicle 2. The on-board camera 4 of this embodiment is a so-called front camera configured to capture an image in front of the vehicle 2. Note that the on-board camera 4 may be a so-called back camera configured to capture an image behind the vehicle 2 when the vehicle 2 is backing up, or may be a combination of these.

[0018] The on-board image may change over time. The change over time of the on-board image may be caused by the relative movement of an object included in the imaging range of the on-board camera 4 with respect to the vehicle 2. For example, the change over time of the on-board image tends to increase as the speed of the vehicle 2 increases. Furthermore, the change over time of the on-board image tends to increase according to the amount of change in the direction of travel of the vehicle 2 due to a sharp curve, etc. Furthermore, the change over time of the on-board image tends to increase in the area corresponding to an object around the vehicle 2, such as a wall, the closer the object, for example, a wall, is to the vehicle 2. Therefore, the change over time of the on-board image tends to increase when the occupant U is required to pay relatively more attention when operating the vehicle 2.

[0019] The display unit 5 is configured to display a graphical user interface (GUI) screen that can be operated by a user. The display unit 5 can be implemented using a display device such as a CRT display, a liquid crystal display, an organic light-emitting diode (OLED) display, or a plasma display. The display unit 5 may be integrated with the vehicle 2, such as a front display provided in front of the seat 3, an instrument panel, or side monitors provided on the left and right sides of the seat 3. The display unit 5 may also be externally attached, such as a car navigation display or a smartphone that can be attached to the vehicle 2. The display unit 5 may be configured to display information visible to the passenger U, for example. For example, the display unit 5 is configured to display a display image, which will be described later. The display unit 5 may also be configured to perform so-called projection mapping, which projects an image onto the interior of the vehicle 2. In this case, a projector that projects an image onto the interior can function as the display unit 5. The display unit 5 of this embodiment may include a front display located in front of the passenger U and a side display located on either the left or right side of the passenger U near the door for boarding the seat 3.

[0020] The display unit 5 is configured to display a display image. The display image may include a plurality of movable objects. The movable objects are objects that can move within the area in which the display image is displayed on the display unit 5, for example, by moving according to a predetermined rule or by changing when a predetermined condition is met.

[0021] The car audio 6 is configured to output acoustic information to the passenger U in a manner that can be perceived through hearing. The acoustic information is information for outputting a piece of music defined by a temporal change in at least one phoneme, and may be defined by, for example, MIDI data. Specific aspects of the acoustic information in this embodiment will be described later.

[0022] The sensor 7 is configured to detect a biosignal of the occupant U. The biosignal represents the physical state of the occupant U. The biosignal may be characterized by a specific frequency indicating the physical state. Examples of such biosignals include pulse waves, pulse rates, and breathing patterns, which may indicate the occupant U's unconscious movements. For convenience of explanation, the biosignal will be assumed to represent the occupant U's pulse wave. In this case, the specific frequency may be determined by, for example, the pulse rate. The sensor 7 detects. The sensor 7 is preferably a so-called wearable device configured to detect biosignals by being attached to the body of the occupant U. This configuration enables more accurate detection of weaker biosignals. However, the sensor 7 is not limited to this and may be, for example, a portable terminal such as a smartphone on which an application for detecting biosignals is installed, or a camera that estimates biosignals by photographing the user's body.

[0023] The information processing device 8 is configured to be able to communicate with the in-vehicle camera 4, the display unit 5, the car audio 6, and the sensor 7 via electrical communication lines. In one embodiment, the information processing system 1 is made up of one or more devices or components. For example, if the information processing system 1 is made up of only the information processing device 8, the information processing system 1 can be the information processing device 8. These components will be described below.

[0024] <Information processing device 8> 2 is a block diagram showing the hardware configuration of the information processing device 8. The information processing device 8 includes a communication unit 81, a storage unit 82, at least one processor 83, and an HMI device 84, and these components are electrically connected via a communication bus 80 inside the information processing device 8.

[0025] The communication unit 81 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), or wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, or BLUETOOTH (registered trademark) communication as needed. That is, it is more preferable to implement it as a collection of multiple communication means. That is, the information processing device 8 may communicate various information from the outside via the communication unit 81 and the network.

[0026] The storage unit 82 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 8 executed by the processor 83, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program operations. The storage unit 82 stores various programs, variables, etc. related to the information processing device 8 executed by the processor 83.

[0027] The processor 83 processes and controls the overall operations related to the information processing device 8. The processor 83 is, for example, a central processing unit (CPU) not shown. The processor 83 realizes various functions related to the information processing device 8 by reading out predetermined programs stored in the storage unit 82. In other words, information processing by software stored in the storage unit 82 is specifically realized by the processor 83, which is an example of hardware, and can be executed as each functional unit included in the processor 83. These will be described in further detail in the next section. Note that the processor 83 is not limited to being single, and multiple processors 83 may be provided for each function. A combination of these may also be used.

[0028] The processor 83 is configured to be able to acquire information from the in-vehicle camera 4 or other devices. The processor 83 is configured to be able to acquire various pieces of information by reading out various pieces of information stored in a storage area that is at least a part of the memory unit 82 and writing the read out information in a working area that is at least a part of the memory unit 82. The storage area is, for example, an area of ​​the memory unit 82 that is implemented as a storage device such as an SSD. The working area is, for example, an area that is implemented as a memory such as a RAM. Note that acquisition by the processor 83 includes acquiring output results from each functional unit included in the processor 83.

[0029] The processor 83 is configured as a display processing unit to be able to display various types of information. The information can be presented to a user via the display unit 5 or another device. In such a case, for example, the processor 83 controls the display unit 5 to display visual information such as a screen, an image including a still image or a video, an icon, or a message. The processor 83 may generate only rendering information for displaying the visual information on the display unit 5. Note that the processor 83 may present the output information to a user without going through the display unit 5 or another device user.

[0030] The HMI device 84 is a human-machine interface device. The HMI device 84 may be included in the housing of the vehicle 2 or may be externally attached. For example, the HMI device 84 may be integrated with the display unit 5 as a touch panel. The touch panel allows the user to input operations such as tapping and swiping. Of course, instead of a touch panel, a switch button, a mouse, a QWERTY keyboard, a voice recognition device, a gesture detection device, a gaze detection device, a biosignal detection device, an imaging device, or the like may be used. That is, the HMI device 84 accepts an operation input made by the user. In response, the HMI device 84 transfers a signal corresponding to the operation input to the processor 83 via the communication bus 80. The processor 83 may perform predetermined control or calculation as necessary. The HMI device 84 can also be said to include an input unit configured to accept input from the user.

[0031] 3. Examples of information processing In this section, the information processing executed in the information processing system 1 described above will be described.

[0032] 3.1. Information processing flow FIG. 3 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. Note that the information processing may include any exception processing not shown. Exception processing includes interruption of the information processing or omission of each process. Selection or input performed in the information processing may be based on a user operation or may be performed automatically without relying on a user operation. For convenience of explanation, the following description will be given assuming that the information processing is performed while the vehicle 2 is traveling, but of course, this is not limited to this.

[0033] [Activity A1] First, in activity A1, processor 83 acquires an on-board image captured by on-board camera 4. As vehicle 2 travels, the on-board image changes over time in different ways depending on the width of the road on which vehicle 2 is traveling, the traveling speed of vehicle 2, and the like. In general, the narrower the road on which vehicle 2 is traveling or the faster the traveling speed, the more drastic the change in the on-board image over time tends to be. In one embodiment, the on-board image is a video including multiple images that are consecutive in time, and processor 83 can acquire from the on-board images the most recent image included in the on-board image and the image in the on-board image immediately preceding the most recent image. The difference between these images can suggest the change in the on-board image over time.

[0034] [Activity A2] Next, in activity A2, processor 83 calculates a complexity parameter based on the in-vehicle image acquired in activity A1. The complexity parameter indicates the complexity of changes in the user's surroundings, for example, the complexity of changes over time in the in-vehicle image. Processor 83 calculates the complexity parameter based on the difference between the latest in-vehicle image and an image of the in-vehicle image taken a predetermined period before a certain image (e.g., the current image). This configuration allows the user to visually grasp the difference between an image taken a predetermined period before and the current image. When a time element is expressed using frames, the predetermined period may be, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 frames, or may be within a range between any two of the values ​​exemplified here. In this embodiment, processor 83 calculates the complexity parameter based on the difference between the latest in-vehicle image and the image immediately preceding the latest in-vehicle image (in other words, the image one frame before). Specifically, processor 83 identifies the number of pixels that differ between the latest and previous images, and calculates the complexity parameter from a relational expression that relates the number of pixels to the complexity parameter. According to this configuration, the time required to calculate the complexity parameter can be reduced. The complexity parameter in this embodiment is a scalar value. The relational expression is defined so that the complexity parameter increases as the number of pixels increases. When the vehicle-mounted image does not change, the number of pixels is approximately 0, and as the changes in the vehicle-mounted image become more complex, the number of pixels increases. Therefore, the number of approximately different pixels correlates with the complexity of the changes in the vehicle-mounted image. Note that the manner of calculating the complexity parameter is not limited to this and is arbitrary. For example, the complexity parameter may be calculated using a pre-trained model that inputs multiple (e.g., two) images and outputs the complexity parameters for the multiple images.

[0035] In one embodiment, the processor 83 calculates a plurality of complexity parameters based on the in-vehicle image. For example, in addition to the complexity parameter of the entire in-vehicle image (hereinafter, for convenience of explanation, sometimes referred to as the overall complexity parameter), the processor 83 may divide the in-vehicle image acquired in activity A1 into a plurality of regions and calculate a partial complexity parameter for each of the divided regions (hereinafter, for convenience of explanation, sometimes referred to as the partial complexity parameter). The partial complexity parameter is an example of a complexity parameter that represents the complexity of changes in different regions of the in-vehicle image.

[0036] [Activity A3] Next, in activity A3, processor 83 generates a display image based on the complexity parameter. That is, the display image is displayed in a different manner depending on the complexity parameter calculated from the in-vehicle image. For example, processor 83 sets the state of movable objects in the display image (e.g., any value related to the display manner of movable objects, such as the position, orientation, color, and number of movable objects) based on the complexity parameter. Processor 83 may set the state of the movable objects based on the overall complexity parameter and the partial complexity parameter.

[0037] [Activity A4] Meanwhile, in activity A4, processor 83 generates acoustic information based on the complexity parameter. For example, processor 83 generates acoustic information composed of at least one phoneme by determining the pitch of the phoneme based on the complexity parameter. When the complexity parameter is small, processor 83 generates acoustic information using consonant sounds composed of one or two intervals, such as a perfect first, a perfect octave, a perfect fourth, or a perfect fifth. As the complexity parameter increases, processor 83 generates acoustic information using consonant sounds composed of three or more intervals, such as a major triad or a minor triad. As the complexity parameter increases further, processor 83 generates acoustic information using dissonant sounds composed of four or more intervals, such as a seventh chord. With this configuration, the chords become more complex as the complexity parameter increases, thereby encouraging occupant U to auditorily notice the complexity of changes in the surrounding situation.

[0038] In one embodiment, the acoustic information may include information about a plurality of phonemes corresponding to sounds output from a musical instrument. Here, a phoneme is information indicating a pitch output by a certain musical instrument. In this case, the processor 83 may generate the acoustic information by determining the output mode of each of the plurality of phonemes based on the complexity parameter. This configuration makes it possible to provide a variety of music based on the complexity of changes in the surrounding environment. For example, when a plurality of complexity parameters (e.g., an overall complexity parameter and a partial complexity parameter) are calculated, the processor 83 assigns each of the plurality of complexity parameters to each of the plurality of phonemes and outputs a phoneme having a pitch corresponding to the value of each complexity parameter. The processor 83 may also generate the acoustic information by determining the output mode of each of the plurality of phonemes based on the partial complexity parameter. This configuration makes it possible to provide a variety of music based on the complexity of changes in the surrounding environment. For example, the processor 83 generates the acoustic information by assigning a different musical instrument to each partial complexity parameter and determining the pitch of each instrument based on the complexity parameter.

[0039] In this way, the processor 83 serves as a generator and generates acoustic information that can be heard by the user and a display image that can be seen by the user based on the complexity parameter and a specific frequency of the biosignal. Note that the display image and acoustic information generated in activities A3 and A4 are generated automatically, without relying on the biosignal detected by sensor 7, which will be described later. The display image and acoustic information are to be modulated based on the biosignal by processing, which will be described later. Hereinafter, for convenience of explanation, the display image and acoustic information to be modulated based on the biosignal will be referred to as the standard display image and standard acoustic information, respectively.

[0040] [Activity A5] After activities A3 and A4, the processor 83 acquires a biological signal from the sensor 7 in activity A5.

[0041] [Activity A6] Next, in activity A6, the processor 83 extracts a specific frequency from the biosignal and sets a specific tempo defined by the specific frequency. For example, the processor 83 extracts the frequency of the maximum component included in the spectrum obtained by Fourier transforming the biosignal as the specific frequency. Here, the processor 83 extracts a frequency corresponding to the pulse rate based on the pulse wave as the biosignal as the specific frequency. Note that the manner of extracting the specific frequency is not limited to this and is arbitrary. For example, the processor 83 may extract one of the frequency peaks included in a frequency band set according to the type of biosignal from the frequencies of the spectrum of the biosignal as the specific frequency. In this embodiment, if the biosignal cannot be acquired from the sensor 7 (for example, the biosignal is transmitted from the sensor 7), the processor 83 sets the specific frequency to zero (or "None").

[0042] The specific tempo is a parameter that defines the bpm of the audio information and represents, for example, the duration of one beat of the audio information. Specifically, for example, if the audio information represents a piece of music defined in 4 / 4 time, the specific tempo corresponds to the duration of one quarter note. The specific tempo is defined to correspond to the breathing interval of the occupant U, whose biosignal has a specific frequency. For example, if the heart rate obtained from the pulse wave is extracted as the specific frequency, the processor 83 sets the specific tempo to a bpm value that matches the heart rate. The heart rate tends to be approximately 1 / 4 to 1 / 2 times the breathing frequency (in other words, the pulse wave period is 2 to 4 times the breathing interval). Therefore, the breathing interval of the occupant U is approximately 2 to 4 beats of the music (i.e., 0.5 to 1 bar), and the biosignal and the breathing interval can be synchronized through the specific tempo. In this embodiment, the processor 83 may set the time moving average of the specific frequency as the specific tempo. With this configuration, it is possible to prevent the occupant U from losing the sense of unity between his / her subjective bodily sensations and the changes in his / her surroundings due to the sudden modulation of the acoustic information described below.

[0043] [Activity A7] Next, in activity A7, processor 83 determines whether a modulation condition is satisfied based on the extracted specific frequency. The modulation condition is a condition for determining whether to introduce modulation according to a specific tempo into the standard display image generated in activity A3 and the standard audio information generated in activity A4. The modulation condition is configured to be satisfied, for example, when the set specific tempo is equal to or greater than a preset value. The preset value is the minimum value of the specific tempo obtained when the biosignal is normally detected. Alternatively, the modulation condition may be configured to be satisfied when a determination result indicating that the biosignal is normally acquired from sensor 7 is acquired. Such a determination result can be obtained, for example, based on the operating status of sensor 7, the positional relationship and contact status between sensor 7 and occupant U, etc.

[0044] [Activity A8] If it is determined in activity A7 that the modulation condition is satisfied, in activity A8, processor 83 modulates the display image generated in activity A3 (i.e., the standard display image) and the audio information generated in activity A4 (i.e., the standard audio information) based on the specific tempo set in activity A6, thereby generating the modulated display image and audio information as the latest display image and audio information.

[0045] First, the processor 83 modulates the standard audio information into audio information having a specific tempo. For example, the processor 83 changes the tempo of the music provided by the standard audio information to the specific tempo while maintaining the musical pitch of the music. At this time, the processor 83 may modulate the standard audio information so that the downbeat of the music provided by the audio information is synchronized with the specific timing. This configuration makes it easier for the user to synchronize the biosignal with the specific tempo of the audio information. More specifically, the processor 83 may generate audio information to output a phoneme of a bass instrument or percussion instrument on the downbeat as a downbeat. This configuration makes it easier for the user to synchronize the biosignal with the specific tempo of the audio information. Examples of bass instruments include low string instruments such as a double bass, and low wind instruments such as a tuba, baritone saxophone, and bass clarinet. The percussion instrument is preferably a skin-type percussion instrument such as a bass drum, timpani, or snare drum to indicate the downbeat. If the standard audio information does not include such strong beat phonemes or has a lower volume than the other phonemes, the processor 83 may add the strong beat phonemes to the standard audio information or modulate the standard audio information by amplifying the volume of the strong beat phonemes with a lower volume, thereby generating audio information having a specific tempo.

[0046] Next, processor 83 modulates the standard display image so that it temporarily changes at a specific timing synchronized with the specific tempo. As a result, the display image is generated so that it temporarily changes at a specific timing synchronized with the specific tempo. For example, processor 83 modulates the standard display image so as to change the state of a movable object included in the standard display image. At this time, it is preferable that the variable indicating the state of the movable object that can be changed by the complexity parameter is different from the variable indicating the state of the movable object to which modulation is introduced. With this configuration, it becomes easier to determine whether a change in the display image visually recognized by occupant U is due to a change in the external environment (complexity parameter) or a biological signal.

[0047] [Activity A9] Thereafter, in activity A8, processor 83 outputs the generated latest display image and audio information to occupant U via display unit 5 and car audio 6. If activity A8 is used, processor 83 outputs the modulated display image and audio information via display unit 5 and car audio 6. This allows occupant U to perceive the display image and audio information.

[0048] If it is determined in activity A7 that the modulation conditions are not satisfied, processor 83 skips the processing of activity A8 and executes the processing of activity A9. In this case, processor 83 outputs the standard display image and standard audio information as the latest display image and audio information.

[0049] After activity A9, processor 83 determines whether to continue this information processing, and if it determines to continue the processing, returns to activity A1. As a result, the latest display image and audio information are updated sequentially while this information processing is continuing, and continuous music and display images are output. On the other hand, if it determines to end the processing, processor 83 ends this information processing. Note that information processing may be ended in response to various operations or conditions, such as receiving an end operation from passenger U or stopping the power supply to the electrical components of vehicle 2.

[0050] With the above configuration, the synchronization between the changes in the external environment reflected in the complexity parameters and the user's physical sensations reflected in the biosignals is encouraged through both visual and auditory senses, thereby assisting the user in moving comfortably.

[0051] 3.2. Example of the relationship between specific tempo and specific frequency of acoustic information when detecting biological signals Next, an example of the relationship between the specific tempo of the acoustic information and the specific frequency when a biosignal is detected will be described. FIG. 4 is a diagram showing an example of the relationship between the specific tempo of the acoustic information and the specific frequency when a biosignal is detected. For convenience of explanation, the units of the specific frequency indicating the periodicity of the biosignal and the specific tempo indicating the periodicity of the music provided by the acoustic information are both assumed to be bpm (beats per minute). In the embodiment shown in FIG. 4, the biosignal is detected at time t1, and the biosignal is no longer detected at time t4. In addition, in this embodiment, the modulation condition is that the specific tempo is a modulation tempo f m The modulation tempo f m may correspond to the lower limit of a particular tempo that indicates that the sensor 7 is continuously detecting the biological signal.

[0052] As shown in Figure 4, before time t1, no biological signal is detected, so the specific frequency is zero, and the specific tempo, which is the time moving average of the specific frequency, is also zero. In this case, the modulation condition is not met, so modulation of the standard acoustic information is not performed, and the standard tempo f in The music is output while maintaining the frequency f. Then, at time t1, a biosignal is detected and the specific frequency becomes a finite value (for example, 100 bpm, which is a standard heart rate). Meanwhile, the specific tempo, which is a moving average of the specific frequency over time, gradually increases from a value of zero at time t1, and reaches the modulated tempo f at time t2. m As a result, the modulation condition is satisfied at time t2, and the processing of activity A8 is performed, so that the tempo of the audio information matches the specific tempo. At this time, the specific tempo is the standard tempo f in Modulated tempo f m Then, as the biosignal continues to be detected, the specific tempo is reduced to the modulated tempo f m and at time t3, the steady-state value f s The tempo reaches f and takes a value (here, a nearly constant value) that is linked to the specific frequency of the biosignal. During this time, modulation of the acoustic information occurs, and the tempo of the acoustic information matches the specific tempo. After that, at time t4, the biosignal is no longer detected, and the specific tempo becomes f s and at time t5, the modulation tempo f m Accordingly, the tempo of the audio information decreases from time t4 along with the specific tempo, and from time t5 onwards, it becomes the standard tempo f in Return to.

[0053] In this way, when a biological signal is not acquired, the processor 83 as a generating unit generates a predetermined standard tempo f in When the biosignal is acquired, the processor 83 generates standard acoustic information having a standard tempo f in Slower tempo (modulated tempo f m ) to a specific tempo (f s) to approach the normal tempo of the user's body. This configuration allows the user to gradually synchronize the changes in the external environment with their bodily sensations while being aware of the timing at which the biosignals are detected. Furthermore, information processing relating to changes in the tempo of such acoustic information can constitute a technical concept independent of this information processing. For example, the information processing system includes at least one processor 83 capable of executing a program to perform the following functions. The processor 83 as an acquisition unit acquires in-vehicle images captured by an imaging device (in-vehicle camera 4) that moves as the user moves, and biosignals representing the user's physical state. The biosignals are characterized by a specific frequency that indicates the physical state. The processor 83 as a generation unit generates acoustic information that the user can hear based on a complexity parameter that represents the complexity of changes in the user's surroundings calculated from the in-vehicle images and the specific frequency of the biosignals. When no biosignals are acquired, the processor 83 generates audio information that the user can hear based on a predetermined standard tempo f in When the biosignal is acquired, the processor 83 generates standard acoustic information having a standard tempo f in Slower tempo (modulated tempo f m ) to a specific tempo f defined by a specific frequency in the steady state s With this configuration, it is possible to provide auditory feedback to the user so that the specific frequency of the biosignal falls within a specified range, while being aware of the detection timing of the biosignal.

[0054] The relationship between the specific tempo of the acoustic information and the specific frequency when a biosignal is detected is not limited to the above. Fig. 5 is a diagram showing another example of the relationship between the specific tempo of the acoustic information and the specific frequency when a biosignal is detected. Note that, among the relationships shown in Fig. 5, parts that are common to the relationship shown in Fig. 4 are given the same numbers and description thereof will be omitted. The relationship shown in Fig. 5 may indicate a situation in which the passenger U is in a tense state and therefore has a higher heart rate than in the case of the relationship shown in Fig. 4. As shown in Fig. 5, when the specific tempo is f mAfter the modulation condition is satisfied, the specific tempo and the tempo of the acoustic information rise together, and at time t31, the maximum tempo f max Reach the maximum tempo f max is a parameter that indicates the upper limit of the specified range. Then, when the biosignal has a specific frequency f max By showing a relatively high value above and falling outside the specified range, the specific tempo is limited to the maximum tempo f max On the other hand, the processor 83 adjusts the tempo of the audio information to a value closer to a specified range than a specific frequency (here, the maximum tempo f max ) for example, set the specified range (i.e., the maximum tempo f max ) may be set to a value corresponding to the heart rate of the occupant U when he / she is calm (in other words, when he / she is relaxed). In this case, the occupant U is motivated to synchronize his / her movements linked to his / her biosignals, such as breathing, with the tempo of the music in order to eliminate the discomfort caused by the difference between the tempo of the music and the tempo of his / her biosignals. As a result, the occupant U's state of tension is gradually released, and for example, at time t32, the tempo of the music and the tempo of his / her biosignals gradually become consistent. Thereafter, the specific tempo is further reduced, and at time t33, the specific tempo f in the steady state is s At this time, the specific tempo f s takes a value corresponding to the heart rate of the occupant U when he / she is calm. After that, the relationship between the specific tempo and the tempo of the acoustic information from time t4 onwards is the same as that shown in FIG. 4. In this way, when the specific frequency is outside a predetermined range, the processor 83 may generate acoustic information so that the specific tempo is closer to the predetermined range than the specific frequency. With this configuration, auditory feedback can be provided to the user so that the specific frequency of the biological signal falls within the predetermined range, while keeping in mind the detection timing of the biological signal.

[0055] 3.3. An example of modulation of audio information and displayed images Next, a specific example of the above-mentioned acoustic information will be described. Fig. 6 is a conceptual diagram showing the phonemes indicating the main beats included in the acoustic information, and the modulation mode and timing of the displayed image. The acoustic information in this embodiment is in 4 / 4 time, with the sixteenth note being the smallest unit of output timing for the phonemes.

[0056] As shown in FIG. 4, the acoustic information is defined by four downbeats b1 to b4. In this embodiment, the acoustic information is modulated at a specific timing that matches, for example, a specific tempo. That is, the interval between adjacent downbeats b1 to b4 matches the bpm of the specific tempo. In this case, a phoneme that is a strong beat is placed on each of the four downbeats b1 to b4. In this case, the phonemes on each downbeat b1 to b4 may be different. For example, the modulated phoneme S1 on the first beat b1 and the third beat b3 may be configured to be more pronounced and perceptible than the modulated phoneme S2 on the second beat and the fourth beat. Each modulated phoneme S1, S2 may be shorter than the duration of one beat of the music provided by the acoustic information. For example, the length of each modulated phoneme S1, S2 may be equivalent to a sixteenth note. In addition to / instead of the short phonemes described above, each modulated phoneme S1 in the first beat b1 may include a phoneme that decays toward the next measure (for example, so-called environmental sounds such as the sound of waves or the rustling of trees). In this case, the length of the phoneme is preferably configured to correspond to a series of periods during which the occupant U inhales or exhales. This configuration can encourage the occupant U to regulate his / her breathing and thereby feel a greater sense of unity between the sound and his / her own interval.

[0057] Meanwhile, the display image is modulated to synchronize with the modulation timing of the acoustic information. V1 in FIG. 6 indicates the modulation timing of the display image (here, the standard display image). In this embodiment, the display image is modulated at the timing of the first beat b1 and the third beat b3 (in other words, odd-numbered beats). That is, the processor 83 can generate a display image so as to temporarily change the display mode of a specific object included in the display image during a specific period from a specific timing until a predetermined period has elapsed. This configuration makes it easier to synchronize the user's consciousness with the tempo of the biosignal through vision and hearing, while taking into account the complexity of changes in the user's surroundings.

[0058] The period during which the display image is modulated (i.e., the specific period) may be shorter than one beat of the music provided by the acoustic information. With this configuration, instantaneous visual changes based on the tempo of the music make it easier to synchronize changes in the external environment with bodily sensations in a sharper manner. For example, the specific period may be included in the period during which the modulated phoneme S1 is output. In this embodiment, the period during which the display image is modulated is the length of a sixteenth note.

[0059] 3.4. Examples of modulation modes of displayed images Next, a specific example of the modulation of the display image described above will be described. Fig. 7 is a diagram showing an example of a change in the display image and a modulation mode. Fig. 8 is a more specific example of the change in the display image and a modulation mode shown in Fig. 7.

[0060] As shown in FIGS. 7 and 8, each of the standard display images IM11 to IM13 includes a first movable object Ob1 and a second movable object Ob2 within a display area of ​​the image. In this embodiment, the first movable object Ob1 is an object that resembles a plant such as a flower. The second movable object Ob2 is an object made of colors that resemble light. The standard display images transition from IM11 to IM12 to IM13 as the complexity parameter increases. Specifically, as the complexity parameter increases, the first movable object Ob1 moves from the center of the image to the left and right, and the second movable object Ob2 appears between the first movable object Ob1. The second movable object Ob2 enlarges, changes color, moves, etc. as the complexity parameter increases. This makes the change in the complexity parameter visible as a display image. The change in the display image based on the change in the complexity parameter is performed regardless of modulation based on a specific tempo.

[0061] As an example, when modulation is performed at the timing when the standard display image IM12 is displayed, the standard display image IM12 becomes a modulated display image IM14. In this embodiment, the display mode of the first movable object Ob1 changes due to the modulation. Here, as an example, the color of the stem of a plant-type object serving as the first movable object Ob1 is modulated so as to temporarily change from a light green to a dark purple. Such visual modulation of the display image is performed in synchronization with the auditory modulation of the acoustic information on the main beat.

[0062] The modulation mode of the display image is not limited to color. FIG. 9 is a diagram showing another example of changes and modulation modes of the display image. FIG. 10 is a more specific example of the example shown in FIG. 9. As shown in FIGS. 9 and 10, in this embodiment, the standard display images IM21 to IM23 include a first movable object Ob1 and a second movable object Ob2 within the image area, as in the above-mentioned example. In this embodiment, the first movable object Ob1 is a regular hexagonal frame. The second movable object Ob2 is a light object superimposed on the first movable object Ob1, etc., and exhibits a shape corresponding to the complexity parameter or the passage of time (for example, a shape that changes to simulate human growth). The first movable object Ob1 and the second movable object Ob2 are configured to move or change within a background image where parallelogram-shaped tiles are laid out. In this embodiment, the number of first movable objects Ob1 increases as the complexity parameter increases. This allows the occupant U to visually grasp the change in the complexity parameter.

[0063] As an example, when modulation is performed at the timing when the standard display image IM22 is displayed, the standard display image IM22 becomes a modulated display image IM24. In this embodiment, the standard display image IM22 is modulated by temporarily changing the first movable object Ob1 from a regular hexagonal frame to a regular triangular frame through modulation. Such visual modulation of the display image is performed so as to synchronize with the auditory modulation of the acoustic information on the right beat.

[0064] The display image may also be modulated for a single object such as a specific character. Fig. 11 shows an example of the change and modulation of the display image. Fig. 12 shows a more specific example of the change and modulation of the display image shown in Fig. 11.

[0065] As shown in FIGS. 11 and 12, the standard display images IM31 to IM33 include a first movable object Ob1 and a third movable object Ob3. The first movable object Ob1 deforms so that the surface curved shape becomes more complex as the complexity parameter increases. The first movable object Ob1 may also rotate so that its orientation changes depending on the turning direction of the vehicle 2. The third movable object Ob3 is configured to move within the image regardless of the complexity parameter. For example, the third movable object Ob3 is configured to move according to the positional relationship (e.g., distance) between the current location and the destination on the planned travel route of the vehicle 2, which is planned by a car navigation system or the like. In this case, the third movable object Ob3 moves from right to left within the standard display images IM31 to IM33 according to the ratio of the distance from the current location to the destination to the distance from the departure location to the destination. This allows the first movable object Ob1, which changes depending on the modulation, and the third movable object Ob3, which shows information about the travel route, to be viewed at a glance, thereby synchronizing the changes in the external environment with the passenger U's physical sensations and allowing the passenger U to visually grasp the distance to the destination, thereby further improving driving comfort.

[0066] As an example, when modulation is performed at the timing when the standard display image IM32 is displayed, the standard display image IM32 becomes a modulated display image IM34. In this embodiment, modulation is performed so that the first movable object Ob1 is temporarily enlarged. At this time, modulation such as enlargement may also be performed on the third movable object Ob3 at the same time. Furthermore, the third movable object Ob3 may change without being dependent on a biological signal.

[0067] 4.Other The above embodiment is merely an example. For example, the above embodiment can be configured as follows.

[0068] The time signature of the above music is not limited to 4 / 4 time, but may be any time signature such as 2 / 4 time, 3 / 4 time, 6 / 8 time, etc. Furthermore, the music provided by the standard acoustic information may be composed of a single phoneme.

[0069] The above-described display images may be displayed on different portions of the display unit 5. For example, the display images IM11 to IM14 may be displayed on a front display of the display unit 5, and the display images IM31 to IM34 may be displayed on a side display of the display unit 5. With this configuration, a greater sense of unity with the vehicle 2 can be achieved.

[0070] The imaging device is not limited to an optical device that captures the surroundings of the vehicle 2, such as the in-vehicle camera 4. For example, the imaging device may be configured to capture images in all directions as a point cloud, such as a LiDAR (Light Detection and Ranging) device, or may be configured to capture images by acoustically measuring information about the surroundings of the vehicle 2, such as a sonar, and visually reconstructing the surrounding situation based on the measurement results. The imaging device may also be a video camera, smartphone, smart glasses, or the like that is carried or worn by the user.

[0071] The user is not limited to the passenger U. For example, if the vehicle 2 is configured to be remotely operable, the user may be a remote operator of the vehicle 2, rather than the passenger U. In this case, the display unit 5 may be displayed on a display panel visible to the remote operator.

[0072] The vehicle 2 is not limited to an automobile, but may include vehicles used as public transportation such as trains, bullet trains, and linear motor cars, as well as vehicles intended to run off public roads, such as go-karts and industrial vehicles. The mobile body may also include equipment capable of moving off the ground, such as flying cars, airplanes, and ships.

[0073] The target image is not limited to an in-vehicle image, but may include any image that shows temporal changes in the environment around the user in real time, obtained using a video camera or the like carried by the user.

[0074] The information processing device 8 is not limited to being integrally mounted on the vehicle 2, but may be, for example, a portable terminal such as a smartphone of the passenger U, or an information processing device provided outside the vehicle 2, such as a cloud server.

[0075] When the information processing device 8 is implemented as a separate unit from the vehicle 2, the information processing device 8 may include an in-vehicle camera 4, a display unit 5, a car audio system 6, and the like.

[0076] The information processing device 8 may be an on-premise type or a cloud type. As the information processing device 8 in the cloud type, the above-mentioned functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0077] In the above embodiment, the information processing device 8 performs various storage and control operations, but multiple external devices may be used instead of the information processing device 8. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0078] The above embodiment is not limited to the information processing system 1, and may be an information processing method or an information processing program. The information processing method includes each step of the information processing system 1. The information processing program causes at least one computer to execute each step of the information processing system 1.

[0079] The information processing system 1 and the like may be provided in the following aspects.

[0080] (1) An information processing system comprising at least one processor configured to execute a program to perform the following steps: in the acquisition step, a target image captured by an imaging device that moves in accordance with the movement of a user and a biosignal representing the user's physical state are acquired, wherein the biosignal is characterized by a specific frequency that indicates the physical state; in the generation step, when the biosignal is acquired, acoustic information that can be heard by the user and a display image that can be seen by the user are generated based on a complexity parameter that represents the complexity of changes in the user's surroundings and the specific frequency of the biosignal, wherein the tempo of the music provided by the acoustic information is determined according to a specific tempo determined by the specific frequency; and the display image is displayed in a different manner according to the complexity parameter calculated from the target image and is generated so as to temporarily change at a specific timing synchronized with the specific tempo.

[0081] With this configuration, the synchronization between the changes in the external environment reflected in the complexity parameters and the user's physical sensations reflected in the biosignals is encouraged through both visual and auditory senses, thereby assisting the user in moving comfortably.

[0082] (2) In the information processing system described in (1) above, in the generating step, the acoustic information is generated so that the downbeat of the music provided by the acoustic information is synchronized with the specific timing.

[0083] With this configuration, the user can easily synchronize the biological signal with a specific tempo of the acoustic information.

[0084] (3) In the information processing system described in (2) above, in the generating step, the acoustic information is generated so that a phoneme of a bass instrument or percussion instrument is output as a strong beat on the downbeat.

[0085] With this configuration, the user can more easily synchronize the biological signal with the specific tempo of the acoustic information.

[0086] (4) In the information processing system described in any one of (1) to (3) above, the acoustic information includes information about a plurality of phonemes corresponding to sounds output from a musical instrument, and in the generation step, the acoustic information is generated by determining the output mode of each of the plurality of phonemes based on the complexity parameter.

[0087] With this configuration, it is possible to provide a variety of music depending on the complexity of changes in the surrounding situation.

[0088] (5) In the information processing system described in (4) above, in the generation step, the acoustic information is generated by determining the output mode of each of the plurality of phonemes based on a partial complexity parameter representing the complexity of changes in different regions of the target image.

[0089] With this configuration, a variety of music can be provided based on the complexity of changes in the surrounding situation, making it easier for the user to synchronize the biosignal with a specific tempo of the acoustic information.

[0090] (6) In the information processing system described in any one of (1) to (5) above, in the generation step, the display image is generated so as to temporarily change the display mode of a specific object included in the display image during a specific period from the specific timing until a predetermined period has elapsed.

[0091] With this configuration, it is possible to easily synchronize the user's own consciousness with the tempo of the biological signal through vision and hearing, while taking into account the complexity of changes in the situation around the user.

[0092] (7) In the information processing system described in (6) above, the specific period is shorter than the period of one beat of the music provided by the acoustic information.

[0093] With this configuration, it becomes easier to synchronize changes in the external environment with bodily sensations in a sharper way, thanks to instantaneous visual changes that reflect the tempo of the music.

[0094] (8) In the information processing system described in any one of (1) to (7) above, in the generation step, when the biological signal is not acquired, standard acoustic information having a predetermined standard tempo is generated, and when the biological signal is acquired, the acoustic information having the specific tempo is generated so as to approach the specific tempo in a steady state from a tempo slower than the standard tempo.

[0095] With this configuration, the user can be gradually encouraged to synchronize changes in the external environment with bodily sensations while being aware of the timing of detection of the biological signal.

[0096] (9) In the information processing system described in any one of (1) to (8) above, in the generation step, when the specific frequency is outside a predetermined specified range, the system generates the acoustic information having a tempo closer to the specified range than the specific frequency.

[0097] With this configuration, auditory feedback can be given to the user so that a specific frequency of the biological signal falls within a specified range, while being aware of the timing of detection of the biological signal.

[0098] (10) An information processing system according to any one of (1) to (9) above, further comprising a moving body, the moving body being configured to be movable with the user on board, the moving body being configured to have an imaging device, a display unit, and an acoustic output unit, the imaging device being configured to generate the target image by imaging at least a portion of the surroundings of the moving body, the display unit being configured to display the display image in a manner that is visible to the user, and the acoustic output unit being configured to output the acoustic information to the user in a manner that is perceptible through hearing.

[0099] (11) An information processing system comprising at least one processor capable of executing a program to perform the following steps: in the acquisition step, a target image captured by an imaging device that moves in accordance with the movement of a user and a biosignal representing the user's physical state are acquired, wherein the biosignal is characterized by a specific frequency that indicates the physical state; in the generation step, acoustic information that can be heard by the user is generated based on a complexity parameter that represents the complexity of changes in the user's surroundings calculated from the target image and the specific frequency of the biosignal; in the generation step, when the biosignal is not acquired, standard acoustic information having a predetermined standard tempo is generated as the acoustic information; when the biosignal is acquired, the acoustic information having the specific tempo is generated so as to approach the specific tempo defined by the specific frequency in a steady state from a tempo slower than the standard tempo; and the tempo of a piece of music provided by the acoustic information is defined according to the specific tempo.

[0100] With this configuration, auditory feedback can be given to the user so that a specific frequency of the biological signal falls within a specified range, while being aware of the timing of detection of the biological signal.

[0101] (12) An information processing method, comprising the steps of the information processing system according to any one of (1) to (11) above.

[0102] (13) A program that causes at least one computer to execute each step of the information processing system according to any one of (1) to (11) above. Of course, this is not the case.

[0103] Finally, while various embodiments of the present disclosure have been described, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0104] 1: Information processing system 2: Vehicle 3: Sheet 4: In-car camera 5:Display section 6: Car audio 7: Sensor 8: Information processing equipment 80: Communication bus 81: Communications Department 82: Storage section 83: Processor 84: HMI device IM11: Standard display image IM12: Standard display image IM13: Standard display image IM14: Display image IM21: Standard display image IM22: Standard display image IM23: Standard display image IM24: Display image IM31: Standard display image IM32: Standard display image IM33: Standard display image IM34: Display image Ob1: First movable object Ob2: Second movable object Ob3: Third movable object S1: Phoneme S2: Phoneme U: Passenger b1~b4: top beat

Claims

1. An information processing system, at least one processor configured to execute a program to perform the following steps: In the acquisition step, a target image captured by an imaging device that moves in accordance with the movement of the user and a biosignal representing a physical state of the user are acquired, wherein the biosignal is characterized by a specific frequency that represents the physical state; In the generation step, when the biological signal is acquired, acoustic information that can be heard by the user and a display image that can be seen by the user are generated based on a complexity parameter that indicates the complexity of changes in the user's surroundings and a specific frequency of the biological signal, wherein: a tempo of the music piece provided by the acoustic information is determined in accordance with a specific tempo determined by the specific frequency; The display image is displayed in different manners depending on the complexity parameter calculated from the target image, and is generated so as to temporarily change at specific timings synchronized with the specific tempo.

2. 2. The information processing system according to claim 1, In the generating step, the system generates the acoustic information so that the downbeat of the music provided by the acoustic information is synchronized with the specific timing.

3. 3. The information processing system according to claim 2, In the generating step, the acoustic information is generated so that a phoneme of a bass instrument or a percussion instrument is output as a strong beat on the downbeat.

4. 2. The information processing system according to claim 1, the acoustic information includes information about a plurality of phonemes corresponding to sounds output from a musical instrument; In the generating step, the system generates the acoustic information by determining an output mode for each of the plurality of phonemes based on the complexity parameter.

5. 5. The information processing system according to claim 4, In the generating step, the system generates the acoustic information by determining the output manner of each of the plurality of phonemes based on a partial complexity parameter that represents the complexity of changes in different regions of the target image.

6. 2. The information processing system according to claim 1, In the generating step, the display image is generated so as to temporarily change the display mode of a specific object included in the display image during a specific period from the specific timing until a predetermined period has elapsed.

7. 7. The information processing system according to claim 6, The system, wherein the specific period is shorter than a period of one beat of the music piece provided by the acoustic information.

8. 2. The information processing system according to claim 1, In the generating step, when the biological signal is not acquired, standard acoustic information having a predetermined standard tempo is generated; A system that, when the biological signal is acquired, generates the acoustic information having the specific tempo so as to approach the specific tempo in a steady state from a tempo slower than the standard tempo.

9. 2. The information processing system according to claim 1, In the generating step, when the specific frequency is outside a predetermined range, the system generates the acoustic information having a tempo closer to the predetermined range than the specific frequency.

10. 2. The information processing system according to claim 1, Furthermore, it is equipped with a mobile body, the moving body is configured to be movable with the user on board, and includes an imaging device, a display unit, and an audio output unit; the imaging device is configured to capture an image of at least a portion of a periphery of the moving object to generate the target image; the display unit is configured to display the display image in a manner that is visible to the user; The information processing system is configured so that the acoustic output unit outputs the acoustic information to the user in a manner that can be perceived through hearing.

11. An information processing system, at least one processor capable of executing a program to perform the following steps; In the acquisition step, a target image captured by an imaging device that moves in accordance with the movement of the user and a biosignal representing a physical state of the user are acquired, wherein the biosignal is characterized by a specific frequency that represents the physical state; In the generating step, acoustic information that can be heard by the user is generated based on a complexity parameter that represents the complexity of changes in the user's surroundings calculated from the target image and a specific frequency of the biological signal, wherein: In the generating step, when the biological signal is not acquired, standard acoustic information having a predetermined standard tempo is generated as the acoustic information; When the biological signal is acquired, the system generates the acoustic information having the specific tempo so as to approach a specific tempo defined by the specific frequency in a steady state from a tempo slower than the standard tempo, and the tempo of the music provided by the acoustic information is defined according to the specific tempo.

12. An information processing method, comprising: A method comprising the steps of the information processing system according to any one of claims 1 to 11.

13. A program, A program that causes at least one computer to execute each step of the information processing system according to any one of claims 1 to 11.

Images