Sound output control device, sound output system, sound output control method, and computer-readable storage medium

Abstract

The sound output control device (20) includes: a direction detection unit (21) for detecting the state of the user's face direction; a surrounding sound acquisition unit (24) for acquiring surrounding sounds; a surrounding sound reduction processing unit (25) for performing processing to reduce surrounding sounds based on the surrounding sounds acquired by the surrounding sound acquisition unit (24); and a sound output control unit (26) for outputting a sound in which the surrounding sounds are reduced by the surrounding sound reduction processing unit (25) when the direction detection unit (21) detects that the state of the user's face direction is a first state, and for setting a state in which the surrounding sounds are easier to hear than in a state in which the surrounding sounds are reduced by the surrounding sound reduction processing unit (25) when the state of the user's face direction changes from the first state to the second state.

Description

Technical Field

[0001] The present invention relates to a sound output control device, a sound output system, a sound output control method and a program. Background Art

[0002] There is known a technology for outputting sound in addition to displaying the details of the content on a display screen, such as an electronic book or a game that can output sound (for example, refer to Patent Document 1).

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent document 1: Japanese Patent Application Publication No. 2016-192211. Summary of the invention

[0006] Problem that the invention aims to solve

[0007] However, when such a device is used while riding in public transportation, for example, it may be difficult to hear surrounding sounds such as in-vehicle broadcasts.

[0008] The present invention has been made in view of the above circumstances, and an object of the present invention is to appropriately confirm surrounding sounds.

[0009] Means used to solve problems

[0010] In order to solve the above-mentioned problems and achieve the purpose, the sound output control device involved in the present invention includes: an orientation detection unit, which detects the state of the user's face orientation; a surrounding sound acquisition unit, which acquires the surrounding sound of the user; a surrounding sound reduction processing unit, which performs processing to reduce the surrounding sound based on the surrounding sound acquired by the surrounding sound acquisition unit; and a sound output control unit, which, when the orientation detection unit detects that the state of the user's face orientation is a first state, outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit, and, when the state of the user's face orientation changes from the first state to the second state, sets the state in which the surrounding sound is easier to hear than the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit.

[0011] The sound output control device involved in the present invention includes: a line of sight detection unit, which detects the line of sight direction of a user; a peripheral sound acquisition unit, which acquires peripheral sounds of the user; a gaze judgment unit, which judges whether the user is gazing at a display screen displaying content based on the detection result of the line of sight detection unit; and a sound output control unit, which changes the output state of peripheral sounds acquired by the peripheral sound acquisition unit and outputs sounds when the gaze judgment unit judges that the user is gazing at the display screen displaying content or judges that the user is not gazing at the display screen.

[0012] The sound output system according to the present invention comprises: the above-mentioned sound output control device; a sound pickup unit for picking up surrounding sounds; and a sound output unit for outputting sounds.

[0013] The sound output control method involved in the present invention includes the following steps: detecting the state of the face orientation of a user using a sound output device; acquiring the surrounding sounds of the user; and when the state of the face orientation of the user is a first state, outputting a sound with reduced surrounding sounds based on the acquired surrounding sounds, and when the state of the face orientation of the user changes from the first state to a second state, setting the state in which the surrounding sounds are easier to hear than in the state in which the surrounding sounds are reduced.

[0014] The program involved in the present invention enables a computer to execute the following steps: detecting the state of the face orientation of a user using a sound output device; acquiring the surrounding sounds of the user; and when the state of the face orientation of the user is a first state, outputting a sound with reduced surrounding sounds based on the acquired surrounding sounds, and when the state of the face orientation of the user changes from the first state to a second state, setting the state in which the surrounding sounds are easier to hear than in the state in which the surrounding sounds are reduced.

[0015] The sound output control method involved in the present invention includes the following steps: detecting the line of sight direction of a user using a sound output device; acquiring the peripheral sounds of the user; judging whether the user is looking at a display screen showing content based on the line of sight direction detection result; and changing the output state of the peripheral sounds and outputting the sounds when it is judged that the user is not looking at the display screen showing content or when it is judged that the user is not looking at the display screen showing content.

[0016] The program involved in the present invention enables a computer to execute the following steps: detecting the line of sight direction of a user who is using a sound output device; acquiring the peripheral sounds of the user; judging whether the user is looking at a display screen showing content based on the line of sight direction detection result; and changing the output state of the peripheral sounds and outputting the sounds when it is judged that the user is not looking at the display screen showing content or when it is judged that the user is not looking at the display screen showing content.

[0017] Effects of the Invention

[0018] According to the present invention, it is possible to appropriately check surrounding sounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a block diagram showing a sound output device as a sound output system according to a first embodiment;

[0020] Figure 2is a schematic diagram illustrating the orientation of the sound output device;

[0021] Figure 3 is a schematic diagram illustrating the orientation of the sound output device;

[0022] Figure 4 is a flowchart showing an example of a process flow in the sound output system according to the first embodiment;

[0023] Figure 5 is a flowchart showing another example of the processing flow in the sound output system according to the first embodiment;

[0024] Figure 6 is a schematic diagram of a sound output system according to a second embodiment;

[0025] Figure 7 is a block diagram of a sound output system according to a second embodiment;

[0026] Figure 8 is a block diagram of a sound output system according to a third embodiment;

[0027] Fig. 9 is a flowchart showing an example of a process flow in the sound output system according to the third embodiment;

[0028] Fig.10 is a block diagram showing a content output system according to a fourth embodiment;

[0029] Fig.11 is a flowchart showing an example of a process flow in a content output system according to a fourth embodiment;

[0030] Fig.12 is a flowchart showing another example of the processing flow in the content output system according to the fourth embodiment;

[0031] Fig.13 is a block diagram of a content output system according to a fifth embodiment;

[0032] Fig.14 is a flowchart showing an example of a process flow in a content output system according to a fifth embodiment;

[0033] Fig.15 is a block diagram of a content output system according to a sixth embodiment;

[0034] Fig.16 is a flowchart showing an example of a process flow in the content output system according to the sixth embodiment;

[0035] Fig.17 is a block diagram showing a content output system according to a seventh embodiment;

[0036] Fig.18 is a flowchart showing an example of a process flow in the content output system according to the seventh embodiment;

[0037] Fig.19 is a flowchart showing another example of the processing flow in the content output system according to the seventh embodiment;

[0038] Fig. 20 is a block diagram of a content output system according to an eighth embodiment;

[0039] Fig.21 is a flowchart showing an example of a process flow in the content output system according to the eighth embodiment;

[0040] Fig. 22 is a block diagram of a content output system according to a ninth embodiment;

[0041] Fig.23 This is a flowchart showing an example of a process flow in the content output system according to the ninth embodiment. DETAILED DESCRIPTION

[0042] Hereinafter, an embodiment of the sound output system 1 according to the present invention will be described in detail with reference to the drawings. In addition, the present invention is not limited to the following embodiment.

[0043] [First embodiment]

[0044] <Sound Output System>

[0045] Figure 1 1 is a block diagram showing a sound output device 10 as a sound output system 1 according to the first embodiment. The sound output device is, for example, a headphone 10. In the present embodiment, the sound output system 1 is constituted by the headphone 10 alone.

[0046] <Headphones>

[0047] The headset 10 outputs sound or eliminates the noise of surrounding sound. The headset 10 is, for example, a top-mounted type worn on the user's head. The headset 10 outputs sound based on sound data output from the sound output control device 20. The headset 10 can eliminate surrounding noise based on a control signal output from the sound output control device 20. The headset 10 has a built-in sound output control device 20. The headset 10 has a left sound output unit 11, a right sound output unit 12, a sound input unit 13, a left microphone 14, a right microphone 15, a sensor 16, and a sound output control device 20.

[0048] The left sound output unit 11 is a sound output unit for the left ear. The left sound output unit 11 has a frame that covers the left ear. The left sound output unit 11 outputs the sound heard in the left ear. The left sound output unit 11 obtains sound data from the sound output control device 20. The left sound output unit 11 outputs the left channel data of the sound data. The left sound output unit 11 converts the electrical signal obtained by D / A (Digital to Analog) conversion of the left channel data of the sound data into sound and outputs it.

[0049] The right sound output unit 12 is a sound output unit for the right ear. The right sound output unit 12 outputs the sound to be heard in the right ear. The right sound output unit 12 obtains sound data from the sound output control device 20. The right sound output unit 12 outputs the right channel data of the sound data. The right sound output unit 12 converts the electrical signal obtained by D / A conversion of the right channel data of the sound data into sound and outputs it.

[0050] The sound input unit 13 may be input with sound data of content such as music or images, for example. The sound input unit 13 may also be input with sound data stored in a storage unit (not shown), for example. The sound input unit 13 may also be input with sound data from an electronic device such as an e-book terminal (not shown), a smart phone, a tablet terminal, a portable music playback device, and a portable game console, for example, via wired or wireless communication.

[0051] The left microphone 14 is disposed in the housing of the left sound output unit 11 . The left microphone 14 acquires surrounding sounds. The surrounding sounds are, for example, environmental sounds including voices of third parties and noises of vehicles. The left microphone 14 outputs the acquired sounds to the surrounding sound acquisition unit 24 .

[0052] The right microphone 15 is disposed in the housing of the right sound output unit 12 . The right microphone 15 acquires surrounding sounds and outputs the acquired sounds to the surrounding sound acquisition unit 24 .

[0053] The sensor 16 detects the posture of the headset 10, in other words, detects the direction of the headset 10. In other words, the sensor 16 detects the direction of the face of the user wearing the headset 10. The sensor 16 detects the posture of the headset 10 using various sensors such as a 3-axis acceleration sensor. The method of calculating the posture from the various sensors can use any known method. The sensor 16 outputs the detection result to the direction detection unit 21. The sensor 16 is preferably configured in the frame of the left sound output unit 11 and the frame of the right sound output unit 12, respectively. Thereby, the posture of the headset 10 can be detected with high accuracy.

[0054] <Sound Output Control Device>

[0055] The sound output control device 20 outputs sound from the headphone 10 or eliminates surrounding noise. The sound output control device 20 outputs sound with reduced surrounding sound, or sets the state in which the surrounding sound is easier to hear than in the state in which the surrounding sound is reduced, according to the user's face orientation. The sound output control device 20 is, for example, an operation processing device (control unit) composed of a CPU (Central Processing Unit) or a processor for sound processing. The sound output control device 20 loads a program stored in a storage unit not shown in the figure into a memory and executes the commands contained in the program. The sound output control device 20 includes a direction detection unit 21, a sound processing unit 22, and a storage unit as an internal memory. The sound processing unit 22 has a sound acquisition unit 23, a surrounding sound acquisition unit 24, a surrounding sound reduction processing unit 25, and a sound output control unit 26. The sound output control device 20 can be composed of one or more devices.

[0056] The orientation detection unit 21 detects the posture of the headphones 10, in other words, the orientation of the headphones 10, based on the detection result of the sensor 16. Thus, the orientation detection unit 21 detects the orientation of the face of the user wearing the headphones 10. In more detail, the orientation detection unit 21 detects that the user's face orientation is in a first state and that the user's face orientation is in a second state different from the first state. For example, the orientation detection unit 21 detects a state in which the user's face orientation is facing downward and a state in which the user's face orientation is not facing downward. The orientation detection unit 21 can, for example, detect a case in which the acceleration generated by the change in the user's face orientation is less than a threshold value and a case in which the acceleration generated by the change in the user's face orientation is greater than a threshold value.

[0057] When the user's face orientation changes from the first state to the second state, there is a high possibility that the user's attention is directed toward the surroundings, such as the user looking around. When the user's face orientation changes from a downward state to a state not facing downward, there is a high possibility that the user's attention is directed toward the surroundings, such as the user raising his face to check the surroundings. When the acceleration generated by the change in the user's face orientation is above the threshold, it means that the user has changed the orientation of his face drastically, and there is a high possibility that the user's attention is directed toward the surroundings. In this way, when there is a high possibility that the user's attention is directed toward the surroundings, it is preferable to set the noise cancellation function that reduces the surrounding sounds to OFF so that the surrounding sounds are easily heard.

[0058] Figure 2 and Figure 3 1 is a schematic diagram showing the orientation of the headphones 10 when the user wears the headband 101 of the headphones 10 on the top of the user's head. Figure 2The figure shows the posture when the headphone 10 is viewed from the horizontal direction, specifically, the figure shows the posture when the headphone 10 is viewed from the right sound output unit 12 side on the horizontal right side of the state in which the user wears the headphone 10. In this case, the face of the user wearing the headphone 10 is facing the +Y direction, not facing downward but facing forward. Figure 3 1 is a diagram showing a posture when the headphones 10 are viewed from the lateral direction, specifically, a diagram showing a posture when the headphones 10 are viewed from the lateral right side of the state in which the user wears the headphones 10. In this case, the face of the user wearing the headphones 10 is facing downward, which is the +Y direction.

[0059] like Figure 2 and Figure 3 As shown in FIG. 1 , the face orientation of the user wearing the headphones 10 is defined by the inclination of the Y-axis direction and the Z-axis direction of the headphones 10 relative to the direction in which the gravitational acceleration G is detected. When the user wearing the headphones 10 changes from facing straight ahead to facing downward, the posture of the headphones 10 changes from Figure 2 The status shown becomes Figure 3 Status shown.

[0060] For example, when the headset 10 is worn on the user's head, it is determined whether the user's face is facing downward based on the angle of the headset 10. Figure 2 As shown in the state, when the direction of the gravitational acceleration G is detected to be within ±20 degrees from the -Z direction, it is determined that the user's face is facing forward rather than downward. Figure 3 As shown in the state, when the direction of the gravitational acceleration G is detected to be more than ±20 degrees from -Z, it is determined that the user's face is facing downward. In addition, it is not limited to the Z axis, and the Y axis or a composite vector of the Z axis and the Y axis may also be used.

[0061] Whether the headphones 10 are worn on the user's head can be determined based on the detection result of the sensor 16. The detection result of the sensor 16 in the X-axis direction indicates that the headphone 10 is substantially horizontal, and when the gravitational acceleration is not detected within ±10 degrees from the +Y-axis direction or the -Y-axis direction, for example, it is determined that the headphones 10 are worn on the user's head. In addition, other sensors that detect, for example, the degree of opening of the headband 101 or the pressure on the ear pads may be used.

[0062] The sound acquisition unit 23 acquires, from the sound input unit 13 , sound data to be output from the left sound output unit 11 and the right sound output unit 12 .

[0063] The surrounding sound acquisition unit 24 acquires the surrounding sound of the user from the left microphone 14 and the right microphone 15 .

[0064] The surrounding sound reduction processing unit 25 performs noise elimination of the surrounding sound and reduces the surrounding sound based on the surrounding sound acquired by the surrounding sound acquisition unit 24. The noise elimination method can be applied by a known method, and a signal with a phase opposite to that of the surrounding sound is synthesized with the sound data acquired by the sound input unit 13 and outputted to the sound output control unit 26.

[0065] The sound output control unit 26 controls the sound data to be output as sound from the headphone 10. More specifically, the sound output control unit 26 causes the left sound output unit 11 to output a signal obtained by performing D / A conversion on the left channel data of the sound data and amplifying the signal. The sound output control unit 26 causes the right sound output unit 12 to output a signal obtained by performing D / A conversion on the right channel data of the sound data and amplifying the signal.

[0066] When the orientation detection unit 21 detects that the user's face orientation is in the first state, the sound output control unit 26 outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25, and when the user's face orientation changes from the first state to the second state, the state is set to a state in which the surrounding sound is easier to hear than the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25.

[0067] When the user's face orientation is in a downward state, the sound output control unit 26 outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25. When the user's face orientation changes from a downward state to a not downward state, the sound output control unit 26 is set to a state in which the surrounding sound is easier to hear than in a state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25.

[0068] The sound output control unit 26 outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25 when the acceleration generated by the change in the user's face orientation is smaller than a threshold value, and sets the state in which the surrounding sound is easier to hear than in the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25 when the acceleration change generated by the change in the user's face orientation is greater than a threshold value.

[0069] As a state in which the surrounding sounds are easier to hear than the state in which the surrounding sounds are reduced by the surrounding sound reduction processing unit 25, the sound output control unit 26 may be set not to perform the surrounding sound reduction processing by the surrounding sound reduction processing unit 25. In this case, the sound output control unit 26 turns off the noise canceling. Thus, the user wearing the headphones 10 can hear the surrounding sounds more easily.

[0070] As a state in which the surrounding sound is easier to hear than the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 25, the sound output control unit 26 may output the surrounding sound acquired by the surrounding sound acquisition unit 24 from the left sound output unit 11 and the right sound output unit 12 without performing the surrounding sound reduction processing by the surrounding sound reduction processing unit 25. Thus, the user wearing the headphones 10 can hear the surrounding sound from the left sound output unit 11 and the right sound output unit 12.

[0071] Next, use Figure 4 , Figure 5 , information processing in the headset 10 is described. Figure 4 This is a flowchart showing an example of a process flow in the sound output system 1 according to the first embodiment. Figure 5 1 is a flowchart showing another example of the processing flow in the sound output system 1 according to the first embodiment. For example, when the power of the headphones 10 is turned on, when the headphones 10 are worn on the head of the user, when the headphones 10 have a function of outputting the sound of the content, when the sound output starts from the headphones 10, or when the start operation of the sound output control process is performed, the processing is executed. Figure 4 , Figure 5 In addition, Figure 4 , Figure 5 When the process of the flowchart shown is started, it is assumed that the noise elimination process is performed.

[0072] First of all, Figure 4. The processing shown is described. The sound output control device 20 determines whether the state of the user's face orientation has changed (step S101). In more detail, the orientation detection unit 21 detects the posture of the headphone 10, in other words, detects the orientation of the headphone 10 based on the detection result of the sensor 16, and determines whether the orientation of the headphone 10 has changed. When the orientation detection unit 21 determines that the state of the user's face orientation has changed ("Yes" in step S101), the process proceeds to step S102. When the orientation detection unit 21 does not determine that the state of the user's face orientation has changed ("No" in step S101), the process proceeds to step S104. Whether the state of the user's face orientation has changed in step S101 means that when the user's face orientation has changed by, for example, more than 10 degrees in at least any one of the X-axis direction, the Y-axis direction, or the Z-axis direction, it is determined that the state of the user's face orientation has changed. Alternatively, whether the state of the user's face orientation has changed in step S101 means that when the acceleration when the user's face orientation has changed is greater than a threshold, it is determined that the state of the user's face orientation has changed. The threshold in this case is a threshold that detects the acceleration generated when the user's face suddenly faces a different direction.

[0073] When it is determined that the state of the user's face orientation has changed ("Yes" in step S101), the sound output control device 20 determines whether the change in the state of the user's face orientation detected in step S101 is a change from a first state to a second state (step S102). More specifically, the sound output control unit 26 determines whether the change in the state of the user's face orientation detected in step S101 is a change from a first state in which the user's face is facing downward to a second state in which the user's face is not facing downward. Alternatively, the sound output control unit 26 determines whether the change in the state of the user's face orientation detected in step S101 is a change from a first state in which the acceleration generated by the change in the user's face orientation is smaller than a threshold value to a second state in which the acceleration generated by the change in the user's face orientation is larger than a threshold value.

[0074] If it is determined that the change in the state of the user's face orientation is a change from the first state to the second state ("Yes" in step S102), the sound output control device 20 proceeds to step S103. If it is not determined that the change in the state of the user's face orientation is a change from the first state to the second state ("No" in step S102), the sound output control device 20 proceeds to step S104.

[0075] When it is determined that the change in the state of the user's face orientation is a change from the first state to the second state ("Yes" in step S102), the sound output control device 20 turns off the noise cancellation (step S103). More specifically, the sound output control unit 26 turns off the noise cancellation of the surrounding sound reduction processing unit 25. In addition, the sound output control unit 26 may output the sound of the content from the left sound output unit 11 and the right sound output unit 12. In this state, the sound of the content is output from the left sound output unit 11 and the right sound output unit 12 in a state where the surrounding sounds are easy to hear. The sound output control device 20 proceeds to step S104.

[0076] The sound output control device 20 determines whether to end the use of the headphones 10 (step S104). For example, when the power of the headphones 10 is turned off, when the headphones 10 are removed from the user's head, when the sound output from the headphones 10 is stopped, or when the sound output control process is terminated, the sound output control device 20 determines that the use is terminated ("Yes" in step S104), and the process is terminated. If the above conditions are not met, the sound output control device 20 does not determine that the use is terminated ("No" in step S104), and proceeds to step S105.

[0077] If the sound output control device 20 is not determined to be finished ("No" in step S104), it determines whether a predetermined period has passed (step S105). In more detail, after the noise cancellation is turned off in step S103, the sound output control unit 26 determines whether a predetermined period of about 1 minute from 30 seconds has passed. The predetermined period in this case is a period that is appropriate for the user to confirm the surrounding sounds. If the sound output control unit 26 does not determine that the predetermined period has passed ("No" in step S105), the determination of step S105 is performed again. If it is determined that the predetermined period has passed ("Yes" in step S105), the process proceeds to step S106.

[0078] In step S105 , when it is determined that the predetermined period has elapsed (“YES” in step S105 ), the sound output control device 20 turns the noise canceling on (ON) (step S106 ), and the process proceeds to step S101 .

[0079] Next, Figure 5 The processing shown is described below. Figure 5 Steps S111, S112, S114, and S115 of the flowchart shown in FIG. Figure 4 Steps S101, S102, S104, and S105 of the flowchart shown are the same processes.

[0080] When it is determined that the state of the user's face orientation is a change from the first state to the second state ("Yes" in step S112), the sound output control device 20 turns off the noise cancellation and outputs the surrounding sound (step S113). In more detail, the sound output control unit 26 turns off the noise cancellation of the surrounding sound reduction processing unit 25, and outputs the surrounding sound acquired by the surrounding sound acquisition unit 24 from the left sound output unit 11 and the right sound output unit 12. In addition, the sound output control unit 26 may also output the sound of the content from the left sound output unit 11 and the right sound output unit 12. In this state, the sound of the content and the surrounding sound are output from the left sound output unit 11 and the right sound output unit 12 in a state where the surrounding sound is easy to hear. The sound output control device 20 proceeds to step S114.

[0081] In step S115 , when it is determined that the predetermined period has elapsed (YES in step S115 ), the sound output control device 20 turns on the noise canceling and stops the output of the surrounding sound (step S116 ), and the process proceeds to step S111 .

[0082] In this way, when the face orientation of the user wearing the headphone 10 changes from the first state to the second state, when the face orientation of the user changes from the state facing downward to the state not facing downward, or when the acceleration generated by the change in the face orientation of the user is greater than a threshold value, it becomes a state in which the surrounding sounds are easier to hear than in a state in which the surrounding sounds are reduced.

[0083] <Effect>

[0084] As described above, in the present embodiment, when the state of the user's face orientation changes from the first state to the second state, when the user's face orientation changes from the state facing downward to the state not facing downward, or when the acceleration generated by the change in the user's face orientation is greater than a threshold value, the surrounding sounds can be easily heard compared to a state where the surrounding sounds are reduced. According to the present embodiment, when it is estimated that the user is paying attention to the surroundings, the surrounding sounds can be appropriately confirmed compared to a state where the surrounding sounds are noise-canceled. In this way, according to the present embodiment, when the user wants to hear the surrounding sounds, the surrounding sounds can be appropriately confirmed.

[0085] In this embodiment, when the state of the user's face orientation changes from the first state to the second state, when the user's face orientation changes from the state of facing downward to the state of not facing downward, or when the acceleration generated by the change in the user's face orientation is greater than a threshold, it is easier to hear the surrounding sounds than in a state where the surrounding sounds are reduced. In these cases, the possibility that the user's attention is directed to the surroundings is high. According to this embodiment, when the possibility that the user's attention is directed to the surroundings is high, the noise cancellation function is turned off, and the surrounding sounds can be easily heard.

[0086] In the present embodiment, as a state in which the surrounding sound is easier to hear than in a state in which the surrounding sound is reduced, the surrounding sound reduction processing by the surrounding sound reduction processing unit 25 is not performed, or the surrounding sound reduction processing is not performed, but the surrounding sound acquired by the surrounding sound acquisition unit 24 is output from the left sound output unit 11 and the right sound output unit 12. According to the present embodiment, the surrounding sound can be easily heard.

[0087] When the state of the user's face orientation changes from the first state to the second state, for example, when the user is sitting on a seat of a vehicle with the face facing downward while listening to music or the like using the headphones 10, the user wants to know information such as a stop station through the announcement in the vehicle, and thus performs an action of raising the face. By performing such an action, the state of the user's face orientation changes from the first state to the second state.

[0088] [Second embodiment]

[0089] Figure 6 It is a schematic diagram of a sound output system 1A according to a second embodiment. Figure 7 1 is a block diagram of a sound output system 1A according to a second embodiment. The sound output system 1A includes a headset 10A as a sound output device, and an electronic device 30A such as an e-book terminal, a smartphone, a tablet terminal, a portable music playback device, and a portable game console. In other words, the sound output system 1A is a combination of the headset 10A as a sound output device and the electronic device 30A.

[0090] The headphone 10A is connected to the electronic device 30A so as to be able to transmit and receive data by wire or wirelessly. The headphone 10A outputs a sound signal output from the electronic device 30A.

[0091] The electronic device 30A includes a sound input unit 31A having the same function as the sound input unit 13 , and a sound output control device 40A having the same function as the sound output control device 20 .

[0092] Figure 7The structure of the headset 10A is Figure 1 The structure included in the structure of the headphone 10 shown is the same. Figure 7 The structure of the electronic device 30A is relative to Figure 1 The structure of the headphone 10 shown in the figure includes the following structures: the sound output control device 40A corresponds to the sound output control device 20, and the sound processing unit 42A corresponds to the sound processing unit 22. Similarly, the sound acquisition unit 43A corresponds to the sound acquisition unit 23, the surrounding sound acquisition unit 44A corresponds to the surrounding sound acquisition unit 24, the surrounding sound reduction processing unit 45A corresponds to the surrounding sound reduction processing unit 25, the sound output control unit 46A corresponds to the sound output control unit 26, and the direction detection unit 41A corresponds to the direction detection unit 21, and they are the same structures.

[0093] As described above, even when the headphones 10A as the sound output device and the electronic device 30A are combined as in the present embodiment, it is possible to appropriately check the surrounding sound as in the first embodiment.

[0094] [Third Embodiment]

[0095] Reference Figure 8 , Fig. 9 , the sound output system 1B involved in this embodiment is described. Figure 8 It is a block diagram of a sound output system 1B according to the third embodiment. Fig. 9 1 is a flowchart showing an example of a processing flow in the sound output system 1B according to the third embodiment. The basic structure of the sound output system 1B is the same as that of the sound output system 1A according to the second embodiment. In this embodiment, the electronic device 30B has a GNSS (Global Navigation Satellite System) receiving unit 32B, a position information calculating unit 47B, and a determining unit 48B.

[0096] The GNSS receiving unit 32B is constituted by a GNSS receiver that receives a GNSS signal from a GNSS satellite, etc. The GNSS receiving unit 32B outputs the received GNSS signal to the position information calculating unit 47B.

[0097] The position information calculation unit 47B receives the GNSS signal from the GNSS receiving unit 32B. The position information calculation unit 47B calculates the current position information based on the GNSS signal. The position information calculation unit 47B and the GNSS receiving unit 32B are not limited to the GNSS signal, and may also correspond to other types of positioning satellite systems.

[0098] The determination unit 48B determines whether the user has used a means of transportation. For example, the determination unit 48B may determine that a user has used a means of transportation based on the position information calculated by the position information calculation unit 47B and based on whether the current position of the user wearing the headphones 10B matches the position information of a means of transportation in the map information (not shown), or the movement history or the movement speed. The determination unit 48B does not limit the method of determining whether a means of transportation has been used, and may determine by other methods using, for example, the surrounding noise or vibration of the headphones 10B.

[0099] Next, use Fig. 9 , the information processing in the sound output system 1B is explained. Fig. 9 Steps S122 to S124, step S126 and step S127 of the flowchart shown in FIG. Figure 4 The processes of steps S101 to S103, step S105, and step S106 of the flowchart shown are the same.

[0100] The sound output control device 40B determines whether to use a means of transportation (step S121). More specifically, the determination unit 48B determines that a means of transportation is used based on the position information calculated by the position information calculation unit 47B and the determination result that the user's current position matches the position information of the means of transportation in the map information. If the determination unit 48B determines that a means of transportation is used ("Yes" in step S121), the process proceeds to step S122. If the determination unit 48B does not determine that a means of transportation is used ("No" in step S121), the process ends.

[0101] In the judgment of the end of use in step S125, in addition to the judgment of the end of use of the headphones 10B or the electronic device 30B, the sound output control device 40B also judges whether to end the use of the vehicle (step S125). In more detail, the judgment unit 48B, based on the position information calculated by the position information calculation unit 47B, determines that the use of the vehicle is ended by detecting that the use of the vehicle is ended when the user's current position deviates from the position information of the vehicle in the map information. When the judgment unit 48B judges that the use of the vehicle is ended ("Yes" in step S125), the process ends. When the judgment unit 48B does not judge that the use of the vehicle is ended ("No" in step S125), the process proceeds to step S126.

[0102] <Effect>

[0103] As described above, in the present embodiment, when the user wearing the headphones 10B uses transportation, when the user's face orientation changes from the first state to the second state, when the user's face orientation changes from a downwardly-oriented state to a non-downwardly-oriented state, or when the acceleration generated by the change in the user's face orientation is greater than a threshold value, it is possible to easily hear surrounding sounds compared to a state where surrounding sounds are reduced. According to the present embodiment, when the user wearing the headphones 10B raises his face in order to hear a broadcast or the like while using transportation, it is possible to easily hear surrounding sounds such as a broadcast or the like. According to the present embodiment, it is possible to suppress the noise cancellation from being accidentally turned off.

[0104] [Fourth Embodiment]

[0105] <Content Output System>

[0106] Fig.10 : is a block diagram showing a content output system 1T involved in the fourth embodiment. In this embodiment, a content output control device 40T as an embodiment of a sound output control device is described. The content output system 1T is an embodiment of a sound output system. The same is true in the following embodiments. The content output control device 40T corresponds to the sound output control device, and the content output system 1T corresponds to the sound output system. In the following description, the same symbols or corresponding symbols are marked for the same components as the sound output system and the sound output control device of the first to third embodiments, or the corresponding relationship is recorded, and the detailed description thereof is omitted. The content output system 1T includes a headset 10T as a sound output device, and an electronic device 30T such as an electronic book terminal, a smart phone, a tablet terminal, a portable music playback device, and a portable game console that reproduces and displays content composed of images and sounds. In other words, the content output system 1T is a combination of the headset 10T as a sound output device and the electronic device 30T. The content output system 1T outputs sound from the headset 10T or eliminates the noise of surrounding sounds according to the user's line of sight.

[0107] <Headphones>

[0108] The headset 10T outputs sound related to the content displayed on the display screen 36 based on the sound data output from the content output control device 40T of the electronic device 30T. The headset 10T can eliminate the surrounding noise based on the control signal output from the content output control device 40T. The headset 10T is connected to the electronic device 30T in a manner that can transmit and receive data by wire or wirelessly. The headset 10T has a left sound output unit 11, a right sound output unit 12, a left microphone 14, a right microphone 15, and a content output control device 40T. The headset 10T is configured in the same manner as the headset 10A of the second embodiment, but is different in that it does not have a sensor.

[0109] <Electronic equipment>

[0110] The electronic device 30T includes a content input unit 31T and a display unit 32T (see Figure 6 ), line of sight sensor 33T and content output control device 40T.

[0111] The content input unit 31T corresponds to the sound input unit 31A of the second embodiment. The content data input to the content input unit 31T is content data output with sound, such as moving image content, game content, WEB content, etc.

[0112] The content data includes image data and sound data related to the content. The image data related to the content is the image data of the content displayed on the display screen 36. The sound data related to the content is the sound data of the sound output corresponding to the image data of the content displayed on the display screen 36. The sound data related to the content may be, for example, the sound of the text of the electronic book being read aloud, the sound of the text of the WEB page being read aloud, and the sound of the explanation.

[0113] The sight line sensor 33T is arranged so as to face the same direction as the display screen 36 of the electronic device 30T. The sight line sensor 33T is a sensor that detects the sight line of a person facing the display screen 36 of the electronic device 30T. The sight line sensor 33T is arranged at a position facing the user's face when the user visually recognizes the display screen 36. The sight line sensor 33T is arranged, for example, at the upper part of the display screen 36 of the electronic device 30T. The sight line sensor 33T outputs the captured image data to the sight line detection unit 43T of the content output control device 40T.

[0114] The sight line sensor 33T includes, for example, an infrared light emitting unit composed of an infrared LED (Light Emitting Diode) group, and a pair of infrared cameras. In the present embodiment, the sight line sensor 33T uses a pair of infrared light emitting units to irradiate infrared light in the direction of the user's face, and uses an infrared camera to take a picture. In this way, based on the image captured by the infrared camera, the sight line detection unit 43T described later determines whether the user's sight line is toward the display screen 36 based on the position of the user's pupil and corneal reflection. Furthermore, based on the position of the user's pupil and corneal reflection, the position of the user's sight line on the display screen 36 is determined. The sight line sensor 33T may also be another structure having the same function, such as a visible light camera.

[0115] The display unit 32T displays the image of the content input to the content input unit 31T. The display unit 32T is a display including a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) display. The display unit 32T displays the image of the content based on the display data output from the display control unit 42T. The display unit 32T includes a display screen 36 for displaying the image.

[0116] The headset 10T may include a sensor for detecting whether the headset 10T is worn on the user's head. Specifically, the headset 10T includes a three-axis acceleration sensor, and determines that the headset 10T is worn on the user's head based on the direction of the detected gravitational acceleration. In addition, other sensors that detect the degree of opening of the headband or the pressure on the ear pads, etc., may also be used. The sensor may also have the same structure as the sensor 16 of the second embodiment.

[0117] <Content output control device>

[0118] The content output control device 40T outputs sound from the headphone 10T or eliminates surrounding noise. The content output control device 40T outputs sound with reduced surrounding sound, or sets the state in which the surrounding sound is easier to hear than the state in which the surrounding sound is reduced, according to the user's line of sight. The content output control device 40T includes an image acquisition unit 41T, a display control unit 42T, a line of sight detection unit 43T, a gaze determination unit 44T, a sound processing unit 50T, and a storage unit as an internal memory. The sound processing unit 50T corresponds to the sound processing unit 22 of the first embodiment. The content output control device 40T may also be composed of one or more devices.

[0119] The image acquisition unit 41T acquires image data of content to be output to the display screen 36 of the display unit 32T from the content input unit 31T.

[0120] The display control unit 42T displays the image data of the content acquired by the image acquisition unit 41T on the display screen 36 of the display unit 32T.

[0121] The sight line detection unit 43T detects the sight line direction of the user based on the image data captured by the sight line sensor 33T. The method of detecting the sight line is not limited, but in this embodiment, the sight line is detected by corneal reflection.

[0122] The gaze determination unit 44T determines whether the user is gazing at the display screen 36 of the electronic device 30T displaying the content based on the detection result of the sight line detection unit 43T. Gazing at the display screen 36 means that the display screen 36 is located in the direction of the user's sight line, in other words, the state in which the user's sight line direction and the display screen 36 intersect each other for more than a first predetermined period. The first predetermined period is, for example, about 5 seconds. Not gazing at the display screen 36 means that the user's sight line direction is in a direction different from the display screen 36, in other words, the state in which the user's sight line direction and the display screen 36 do not intersect each other for more than a second predetermined period. The second predetermined period is, for example, about 5 seconds.

[0123] The surrounding sound reduction processing unit 54T performs a process of reducing the surrounding sound of the sound output by the left sound output unit 11 and the right sound output unit 12 that output the sound to the user based on the surrounding sound acquired by the surrounding sound acquisition unit 52. The surrounding sound reduction processing unit 54T performs noise elimination of the surrounding sound based on the surrounding sound acquired by the surrounding sound acquisition unit 52 to reduce the surrounding sound. The noise elimination method can apply a known method to output a signal with an opposite phase of the surrounding sound to the sound output control unit 26.

[0124] The sound output control unit 53T changes the state of output of the surrounding sound acquired by the surrounding sound acquisition unit 52 and outputs the sound when the attention determination unit 44T determines that the user is looking at the display screen 36 displaying the content and when the user is not looking at the display screen 36 displaying the content. In the present embodiment, the sound output control unit 53T outputs the sound with the surrounding sound reduced by the surrounding sound reduction processing unit 54T when the attention determination unit 44T determines that the user is looking at the display screen 36 displaying the content, and when the user is not looking at the display screen 36 displaying the content, the sound output control unit 53T outputs the sound with the surrounding sound reduced by the surrounding sound reduction processing unit 54T, and becomes a state in which the surrounding sound is easier to hear than the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 54T.

[0125] Next, use Fig.11 , Fig.12 , information processing in the headphone 10T is described. Fig.11 This is a flowchart showing an example of a process flow in the content output system 1T according to the fourth embodiment. Fig.121 is a flowchart showing another example of the processing flow in the content output system 1T according to the fourth embodiment. For example, when the power of the headphones 10T is turned on, when the headphones 10T are worn on the user's head, when the sound output is started from the headphones 10T, or when the start operation of the content output control process is performed, the content output control process is executed. Fig.11 , Fig.12 Processing of the flowchart shown.

[0126] First of all, Fig.11 The processing shown is explained. The content output control device 40T determines whether the content accompanied by sound output is displayed on the display unit 32T (step ST101). In more detail, it is determined whether the content composed of sound and image input to the content input unit 31T is reproduced, whether the image of the content is displayed on the display unit 32T, and whether the sound of the content is output to the headphone 10T. In other words, it is determined whether the content composed of sound and image is reproduced. The state in which the content accompanied by sound output is displayed on the display unit 32T means that the image of the content is displayed on the display unit 32T and the sound of the content is output to the headphone 10T, and these outputs continue until the reproduction of the content is completed.

[0127] If it is determined that the content with sound output is displayed on the display unit 32T ("Yes" in step ST101), the process proceeds to step ST102. If it is not determined that the content with sound output is displayed on the display unit 32T ("No" in step ST101), the process ends. If it is determined that the content with sound output is displayed on the display unit 32T ("Yes" in step ST101), it is determined whether the user is looking at the display screen 36 (step ST102). More specifically, the gaze determination unit 44T determines whether the user is looking at the display screen 36 of the electronic device 30T displaying the content based on the detection result of the line of sight detection unit 43T. If it is determined that the user is looking at the display screen 36 by the gaze determination unit 44T ("Yes" in step ST102), the process proceeds to step ST103. If it is not determined that the user is looking at the display screen 36 by the gaze determination unit 44T ("No" in step ST102), the process proceeds to step ST104.

[0128] When it is determined that the user is looking at the display screen 36 ("Yes" in step ST102), the content output control device 40T turns on the noise cancellation (step ST102). In more detail, the sound output control unit 53T performs the noise cancellation of the surrounding sound reduction processing unit 54T. In addition, the sound output control unit 53T may also output the sound of the content from the left sound output unit 11 and the right sound output unit 12. In this state, the sound of the content is output from the left sound output unit 11 and the right sound output unit 12. The content output control device 40T proceeds to step ST105.

[0129] The process of step ST103 keeps the noise cancellation on if the noise cancellation is already on when the answer is “YES” in step ST101 , and turns the noise cancellation on if the noise cancellation is off when the answer is “YES” in step ST101 .

[0130] If it is not determined that the user is looking at the display screen 36 ("No" in step ST102), the content output control device 40T turns off the noise cancellation (step ST104). In more detail, the sound output control unit 53T turns off the noise cancellation of the surrounding sound reduction processing unit 54T. In addition, the sound output control unit 53T can also output the sound of the content from the left sound output unit 11 and the right sound output unit 12. In this state, the sound of the content is output from the left sound output unit 11 and the right sound output unit 12 in a state where the surrounding sounds are easy to hear. The content output control device 40T enters step ST105.

[0131] The process of step ST104 turns off the noise cancelling if the noise cancelling is already on when the answer is “YES” in step ST101 , and keeps the noise cancelling off when the answer is “YES” in step ST101 .

[0132] The content output control device 40T determines whether to terminate the use of the content output system 1T such as the headphones 10T and the electronic device 30T (step ST105). For example, when the power of the headphones 10T and the electronic device 30T is turned off, when the headphones 10T is removed from the user's head, when the sound output to the headphones 10T is stopped, or when the end operation of the content output control process is performed, it is determined that the use is terminated ("Yes" in step ST105) and the process is terminated. If the above situation does not meet the requirements, the content output control device 40T will not determine that the use is terminated ("No" in step ST105), and the process of step ST101 will be executed again.

[0133] Next, Fig.12 The processing shown is described below. Fig.12Steps ST111, ST112, ST113, and ST115 of the flowchart shown in FIG. Fig.11 The steps ST101, ST102, ST103, and ST105 of the flowchart shown are the same processing.

[0134] When it is not determined that the user is looking at the display screen 36 ("No" in step ST112), the content output control device 40T turns off the noise cancellation and outputs the surrounding sound (step ST114). In more detail, the sound output control unit 53T turns off the noise cancellation of the surrounding sound reduction processing unit 54T, and outputs the surrounding sound acquired by the surrounding sound acquisition unit 52 from the left sound output unit 11 and the right sound output unit 12. In addition, the sound output control unit 53T can also output the sound of the content from the left sound output unit 11 and the right sound output unit 12. In this state, in a state where it is easy to hear the surrounding sounds, the sound of the content is output from the left sound output unit 11 and the right sound output unit 12 while also outputting the surrounding sounds. The content output control device 40T enters step ST115.

[0135] The processing of step ST113 is to keep the noise cancellation turned on when the noise cancellation is already turned on when the answer is "yes" in step ST111, and to turn on the noise cancellation and stop the output of the surrounding sound when the noise cancellation is turned off and the surrounding sound is output when the answer is "yes" in step ST111.

[0136] The processing of step ST114 is as follows: if the noise cancellation is already turned on when the answer is "yes" in step ST111, the noise cancellation is set to off and the surrounding sound is output; if the noise cancellation is turned off and the surrounding sound is output when the answer is "yes" in step ST111, the noise cancellation is kept turned off and the surrounding sound is output.

[0137] Thus, when the user wearing the headphones 10T is looking at the display screen 36 , the surrounding sounds are reduced, and when the user wearing the headphones 10T is not looking at the display screen 36 , the surrounding sounds are easier to hear than in the state in which the surrounding sounds are reduced.

[0138] <Effect>

[0139] As described above, in the present embodiment, when the user wearing the headphones 10T is looking at the display screen 36, the surrounding sounds are reduced, and when the user is not looking at the display screen 36, the surrounding sounds can be easily heard compared to the state where the surrounding sounds are reduced. According to the present embodiment, when the user stops looking at the display screen 36 and it is estimated that the user is paying attention to the surroundings, the surrounding sounds can be properly confirmed compared to the state where the surrounding sounds are noise-canceled. In this way, according to the present embodiment, when the user wants to hear the surrounding sounds, the surrounding sounds can be properly confirmed even if the user does not perform an operation.

[0140] In the present embodiment, as a state in which the surrounding sound is easier to hear than in a state in which the surrounding sound is reduced, the surrounding sound reduction processing by the surrounding sound reduction processing unit 54T is not performed, or the surrounding sound reduction processing is not performed, but the surrounding sound acquired by the surrounding sound acquisition unit 52 is output from the left sound output unit 11 and the right sound output unit 12. According to the present embodiment, the surrounding sound can be easily heard.

[0141] [Fifth Embodiment]

[0142] Reference Fig.13 , Fig.14 , the content output system 1TA involved in this embodiment is explained. Fig.13 It is a block diagram of a content output system 1TA according to the fifth embodiment. Fig.14 1 is a flowchart showing an example of a processing flow in the content output system 1TA involved in the fifth embodiment. The basic structure of the content output system 1TA is the same as that of the content output system 1T of the fourth embodiment. In the following description, the same reference numerals or corresponding reference numerals are marked on the same components as those of the content output system 1T, and their detailed description is omitted. In this embodiment, the difference from the fourth embodiment is that the electronic device 30TA has a shooting unit 33TA and a face detection unit 48TA of the content output control device 40TA, and the processing in the sound output control unit 53TA.

[0143] The photographing unit 33TA is a visible light camera that photographs a person facing the display screen 36 of the electronic device 30TA. The photographing unit 33TA photographs the face of the user. The photographing unit 33TA is arranged at a position where the face of the user can be photographed while the user visually confirms the display screen 36. The photographing unit 33TA is arranged, for example, at the upper part of the display screen 36 of the electronic device 30TA. The photographing unit 33TA outputs the photographed data to the face detection unit 48TA of the content output control device 40TA. The photographing unit 33TA and the sight line sensor 33T may be the same visible light camera.

[0144] The face detection unit 48TA recognizes the face of the user from the captured data captured by the capturing unit 33TA, and detects the orientation of the recognized face. More specifically, the face detection unit 48TA detects whether the orientation of the face detected by the face detection unit 48TA is an orientation opposite to the display screen 36. For example, when the user is looking at the display screen 36 displaying content, the user's face orientation is an orientation opposite to the display screen 36. For example, when the user pays attention to the surroundings and looks around, the user's face orientation is not an orientation opposite to the display screen 36.

[0145] The direction opposite to the display screen 36 refers to the direction in which the user can visually recognize the image of the content displayed on the display screen 36. The direction opposite to the display screen 36 may be defined as a direction in which the user can clearly see the display screen 36, such as a straight line passing through the center of both eyes of the user and extending in front of the user when viewed from the top and bottom directions and a straight line intersecting the display screen 36 in the range of about 90°±30° when viewed from the left and right directions.

[0146] The sound output control unit 53TA outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit 54T when the gaze determination unit 44T determines that the user is gazing at the display screen 36 displaying the content and the face orientation detected by the face detection unit 48TA is the orientation opposite to the display screen 36. The sound output control unit 53TA is in a state in which the surrounding sound is easier to hear than in a state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 54T when the gaze determination unit 44T determines that the user is not gazing at the display screen 36 displaying the content and the face orientation detected by the face detection unit 48TA is not the orientation opposite to the display screen 36. Even when the user is not gazing at the display screen 36 displaying the content, when the user's face orientation is opposite to the display screen 36, it is estimated that the user intends to continue viewing and listening to the content. In this case, it is preferable to continue to reduce the surrounding sound. In contrast, when the user is not gazing at the display screen 36 displaying the content and the user's face orientation is not opposite to the display screen 36, it is estimated that the user pays high attention to the surroundings. In this case, it is preferable to be able to confirm the surrounding sound.

[0147] Next, use Fig.14 , the information processing in the content output system 1TA is explained. Fig.14 Steps ST121, ST122, ST124 to ST126 of the flowchart shown in FIG. Fig.11 Steps ST101, ST102, and ST103 to ST105 of the flowchart shown are the same processes.

[0148] The content output control device 40TA determines whether the user's face is facing the display screen 36 (step ST123). More specifically, if the face orientation detected by the face detection unit 48TA is facing the display screen 36 ("Yes" in step ST123), the process proceeds to step ST124. If the face orientation detected by the face detection unit 48TA is not facing the display screen 36 ("No" in step ST123), the process proceeds to step ST125.

[0149] <Effect>

[0150] As described above, in the present embodiment, when the user is looking at the display screen 36 and the user's face is facing the display screen 36, the surrounding sounds are reduced, and when the user is not looking at the display screen 36 and the user's face is not facing the display screen 36, the surrounding sounds can be heard more easily than in the state where the surrounding sounds are reduced. According to the present embodiment, in the state where the user's face is not facing the display screen 36 and the user is paying attention to the surroundings, the surrounding sounds can be appropriately confirmed.

[0151] [Sixth Embodiment]

[0152] Reference Fig.15 , Fig.16 , the content output system 1TB involved in this embodiment is explained. Fig.15 It is a block diagram of a content output system 1TB according to the sixth embodiment. Fig.16 is a flowchart showing an example of a processing flow in the content output system 1TB according to the sixth embodiment. The basic structure of the content output system 1TB is Fig.13 The present embodiment is different from the fourth embodiment in that the electronic device 30TB has a GNSS receiving unit 34TB, a position information calculating unit 49TB and a determining unit 50TB of the content output control device 40TB, and processing in the sound output control unit 53TB.

[0153] The GNSS receiving unit 34TB is configured similarly to the GNSS receiving unit 32B of the third embodiment.

[0154] The position information calculation unit 49TB is configured similarly to the position information calculation unit 47B of the third embodiment.

[0155] The determination unit 50TB is configured similarly to the determination unit 48B of the third embodiment.

[0156] When the user is using a vehicle, the sound output control unit 53TB, when it is determined that the user is gazing at the display screen 36 displaying content, outputs the sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit 54T. When it is determined that the user is not gazing at the display screen 36 displaying content, the surrounding sound is in a state in which it is easier to hear than in a state in which the surrounding sound is reduced by the surrounding sound reduction processing unit 54T.

[0157] Next, use Fig.16 , explains the information processing in the content output system 1TB. Fig.16 Steps ST131 and ST136 of the flowchart shown in FIG. Fig. 9 The processing in steps S121 and S125 of the flowchart shown is the same. Fig.16 Steps ST132 to ST135 of the flowchart shown in FIG. Fig.11 The processing from step ST101 to step ST104 of the flowchart shown is the same as that of step ST101.

[0158] <Effect>

[0159] As described above, in the present embodiment, when the user wearing the headphones 10T uses transportation, when the user is looking at the display screen 36, the surrounding sounds are reduced, and when the user is not looking at the display screen 36, the surrounding sounds can be heard more easily than in a state where the surrounding sounds are reduced. According to the present embodiment, when the user wearing the headphones 10T changes the direction of the face from the direction facing the display screen 36 in order to hear the announcement or the like while using the transportation, the surrounding sounds such as the announcement can be heard more easily. According to the present embodiment, it is possible to suppress the noise canceling from being accidentally turned off.

[0160] [Seventh Embodiment]

[0161] <Content Output System>

[0162] Fig.17 1 is a block diagram showing a content output system 1U according to a seventh embodiment. The content output system 1U outputs sound related to the content displayed on the display screen 36 from a headphone 10U or outputs surrounding sound, depending on whether the user is looking at the display screen 36. The sound processing unit 50U of the content output control device 40U of the content output system 1U is different from that of the fourth embodiment.

[0163] <Content output control device>

[0164] The content output control device 40U outputs sound related to the content from the headphone 10U or outputs surrounding sound, depending on whether the user is looking at the display screen 36 .

[0165] The sound processing unit 50U includes a sound acquisition unit 51 , a surrounding sound acquisition unit 52 , and a sound output control unit 53U having the same configuration as those of the fourth embodiment.

[0166] The sound output control unit 53U changes the state of output of the surrounding sound acquired by the surrounding sound acquisition unit 52 and outputs the sound when the gaze determination unit 44U determines that the user is gazing at the display screen 36 displaying the content and when the gaze determination unit 44U determines that the user is not gazing at the display screen 36 displaying the content. In the present embodiment, the sound output control unit 53U outputs the sound related to the content acquired by the sound acquisition unit 51 when the gaze determination unit 44U determines that the user is gazing at the display screen 36 displaying the content, and outputs the surrounding sound acquired by the surrounding sound acquisition unit 52 when the gaze determination unit 44U determines that the user is gazing at the display screen 36 displaying the content.

[0167] When the attention determination unit 44U determines that the user is not looking at the display screen 36 displaying the content, the sound output control unit 53U may output the surrounding sound acquired by the surrounding sound acquisition unit 52 in addition to the sound related to the content acquired by the sound acquisition unit 51. In this case, the headphones 10U output the surrounding sound in addition to the sound related to the content.

[0168] Next, use Fig.18 , Fig.19 , information processing in the headset 10U is explained. Fig.18 This is a flowchart showing an example of a process flow in the content output system 1U according to the seventh embodiment. Fig.19 1 is a flowchart showing another example of the processing flow in the content output system 1U according to the seventh embodiment. For example, when the power of the headphones 10U is turned on, when the headphones 10U are worn on the user's head, when the output of sound starts from the headphones 10U, or when the start operation of the content output control process is performed, the content output control process is executed. Fig.18 , Fig.19 Processing of the flowchart shown. Fig.18 Steps SU101, SU102, and SU105 of the flowchart shown in FIG. Fig.11 Steps ST101, ST102, and ST105 of the flowchart shown are the same processes.

[0169] When it is determined that the user is looking at the display screen 36 ("Yes" in step SU102), the content output control device 40U outputs the sound of the content (step SU103). More specifically, the sound output control unit 53U outputs the sound related to the content acquired by the sound acquisition unit 51 from the headphone 10U. As a result, the sound of the content is output from the left sound output unit 11 and the right sound output unit 12 of the headphone 10U. The content output control device 40U proceeds to step SU105.

[0170] The process of step SU103 is to continue outputting content sound when the ambient sound is not output when the answer is "YES" in step SU101, and to continue outputting content sound and stop outputting ambient sound when the answer is "YES" in step SU101.

[0171] If it is not determined that the user is looking at the display screen 36 ("No" in step SU102), the content output control device 40U outputs the surrounding sound (step SU104). More specifically, the sound output control unit 53U outputs the surrounding sound acquired by the surrounding sound acquisition unit 52. As a result, the surrounding sound is output from the left sound output unit 11 and the right sound output unit 12 of the headphone 10U. The content output control device 40U proceeds to step SU105.

[0172] The process of step SU104 starts outputting the surrounding sounds when the surrounding sounds are not outputted when the answer is YES in step SU101 , and keeps outputting the surrounding sounds when the answer is YES in step SU101 .

[0173] Next, Fig.19 The processing shown is described below. Fig.19 Steps SU111, SU112, SU113, and SU115 of the flowchart shown in FIG. Fig.18 The steps SU101, SU102, SU103, and SU105 of the flowchart shown are the same processing.

[0174] If it is not determined that the user is looking at the display screen 36 ("No" in step SU112), the content output control device 40U outputs the sound related to the content and also outputs the surrounding sound (step SU114). More specifically, the surrounding sound acquired by the surrounding sound acquisition unit 52 is output from the left sound output unit 11 and the right sound output unit 12 of the headphone 10U together with the sound related to the content acquired by the sound acquisition unit 51. The content output control device 40U proceeds to step SU115.

[0175] The processing of step SU113 is to continue outputting the content sound when the ambient sound is not output when the answer is "yes" in step SU111, and to continue outputting the content sound and stop outputting the ambient sound when the answer is "yes" in step SU111.

[0176] The processing of step SU114 is as follows: when the answer is "yes" in step SU111, in a state where content sound is output but surrounding sound is not output, the output of surrounding sound starts while maintaining the output of content sound; when the answer is "yes" in step SU111, in a state where content sound and surrounding sound are output, the output of content sound and surrounding sound is maintained.

[0177] In step SU114 , when the ambient sound is output together with the content sound, the volume of the content sound may be at the volume set by the user, or the volume of the content sound during the output of the ambient sound may be lower than the volume set by the user.

[0178] In this way, when the user wearing the headphone 10U is looking at the display screen 36 , sound related to the content is output, and when the user is not looking at the display screen 36 , surrounding sound can be output.

[0179] <Effect>

[0180] As described above, in the present embodiment, when the user wearing the headphones 10U is looking at the display screen 36, the sound of the content is output, and when the user is not looking at the display screen 36, the surrounding sound can be output. According to the present embodiment, when the user stops looking at the display screen 36 and it is estimated that the user is paying attention to the surroundings, the surrounding sound can be appropriately confirmed. In this way, according to the present embodiment, when the user wants to hear the surrounding sound, the surrounding sound can be appropriately confirmed even if the user does not perform an operation.

[0181] In the present embodiment, the surrounding sound can be output together with the sound of the content when the user is not looking at the display screen 36. According to the present embodiment, the user can check the surrounding sound while continuing to view and listen to the content.

[0182] [Eighth Embodiment]

[0183] Reference Fig. 20 , Fig.21 , the content output system 1UA of this embodiment is described. Fig. 20 It is a block diagram of a content output system 1UA according to the eighth embodiment. Fig.211 is a flowchart showing an example of a processing flow in the content output system 1UA involved in the eighth embodiment. The basic structure of the content output system 1UA is the same as that of the content output system 1U of the seventh embodiment. In the following description, the same reference numerals or corresponding reference numerals are marked on the same components as those of the content output system 1U, and their detailed description is omitted. In this embodiment, the difference from the seventh embodiment is that the electronic device 30UA has a GNSS receiving unit 34UA, a position information calculation unit 45UA and a judgment unit 46UA of the content output control device 40UA, and the processing in the sound output control unit 53UA.

[0184] The GNSS receiving unit 34UA is configured similarly to the GNSS receiving unit 32B of the third embodiment.

[0185] The position information calculation unit 45UA is configured similarly to the position information calculation unit 47B of the third embodiment.

[0186] The determination unit 46UA is configured similarly to the determination unit 48B of the third embodiment.

[0187] When the user is using a vehicle, the sound output control unit 53UA outputs sounds related to the content acquired by the sound acquisition unit 51 if it is determined that the user is looking at the display screen 36 displaying the content, and outputs surrounding sounds acquired by the surrounding sound acquisition unit 52 if it is determined that the user is not looking at the display screen 36 displaying the content.

[0188] Next, use Fig.21 , the information processing in the content output system 1UA is explained. Fig.21 Steps SU121 and SU126 of the flowchart shown in FIG. Fig. 9 The processing in steps S121 and S125 of the flowchart shown is the same. Fig.21 Steps SU122 to SU125 of the flowchart shown in FIG. Fig.18 The processing from step SU101 to step SU104 of the flowchart shown is the same.

[0189] <Effect>

[0190] As described above, in the present embodiment, when the user wearing the headphones 10U uses transportation, the surrounding sounds are output when the user is not looking at the display screen 36. According to the present embodiment, when the user wearing the headphones 10U no longer looks at the display screen 36 of the content in order to listen to the announcement or the like while using transportation, the surrounding sounds such as the announcement can be heard using the headphones 10U. According to the present embodiment, it is possible to suppress the surrounding sounds from being output unintentionally.

[0191] [Ninth Embodiment]

[0192] Reference Fig. 22 , Fig.23 , the content output system 1UB involved in this embodiment is explained. Fig. 22 It is a block diagram of a content output system 1UB according to the ninth embodiment. Fig.23 1 is a flowchart showing an example of a processing flow in the content output system 1UB according to the ninth embodiment. The basic structure of the content output system 1UB is the same as that of the content output system 1U according to the seventh embodiment. In this embodiment, the difference from the seventh embodiment is that the electronic device 30UB has a camera 35UB and a face detection unit 47UB of the content output control device 40UB, and the processing in the sound output control unit 53UB.

[0193] The imaging unit 35UB is configured similarly to the imaging unit 33TA of the fifth embodiment.

[0194] The face detection unit 47UB is configured similarly to the face detection unit 48TA of the fifth embodiment.

[0195] The sound output control unit 53UB outputs the sound related to the content acquired by the sound acquisition unit 51 when the face orientation detected by the face detection unit 47UB is the orientation opposite to the display screen 36, except when the gaze determination unit 44U determines that the user is not gazing at the display screen 36 displaying the content. The sound output control unit 53UB outputs the surrounding sound acquired by the surrounding sound acquisition unit 52 when the face orientation detected by the face detection unit 47UB is not the orientation opposite to the display screen 36, except when the gaze determination unit 44U determines that the user is not gazing at the display screen 36 displaying the content. Even when the user is not gazing at the display screen 36 displaying the content, when the user's face orientation is opposite to the display screen 36, it is estimated that the user intends to continue viewing the content. In this case, it is preferred to continue viewing the content. On the other hand, when the user is not gazing at the display screen 36 displaying the content and the user's face orientation is not opposite to the display screen 36, it is estimated that the user is highly aware of the surroundings. In this case, it is preferred to be able to confirm the surrounding sounds.

[0196] Next, use Fig.23 , the information processing in the content output system 1UB is explained. Fig.23 Steps SU131, SU132, SU134 to SU136 of the flowchart shown in FIG. Fig.18 The processes in steps SU101, SU102, SU103 to SU105 of the flowchart shown are the same. Fig.23Step SU133 of the flowchart shown in FIG. Fig.14 The same processing as step ST123 of the flowchart shown in the figure.

[0197] <Effect>

[0198] As described above, in the present embodiment, when the user is looking at the display screen 36 and the user's face direction is opposite to the display screen 36, the sound of the content is output, and when the user is not looking at the display screen 36 and the user's face direction is not opposite to the display screen 36, the surrounding sound acquired by the surrounding sound acquisition unit 52 can be output. According to the present embodiment, when the user sets the face direction to a direction not opposite to the display screen 36 but to a state where the user pays attention to the surroundings, the surrounding sound can be appropriately confirmed.

[0199] The components of the sound output system 1 shown in the figure are conceptual functions and do not need to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the form shown in the figure, and all or part of them can be functionally or physically dispersed or integrated in arbitrary units according to the processing load of each device, usage status, etc.

[0200] The structure of the sound output system 1 is implemented, for example, by a program loaded into a memory as software. In the above embodiment, the functional blocks implemented by the cooperation of these hardware or software are described. That is, these functional blocks can be implemented in various forms only by hardware, only by software, or by a combination of these.

[0201] The constituent elements of the above-mentioned record include elements that can be easily thought of by those skilled in the art, and substantially the same elements. Furthermore, the structures of the above-mentioned record can be appropriately combined. In addition, various omissions, substitutions or changes of the structure can be performed within the scope of the gist of the present invention.

[0202] In the above, as examples of states in which it is easy to hear the surrounding sounds, an example of a state in which the surrounding sound reduction process is not performed, and an example in which the surrounding sounds are output without performing the surrounding sound reduction process are shown. As examples of states in which it is easy to hear the surrounding sounds, in addition to the above, the effect of reducing the surrounding sound reduction level, that is, reducing the noise cancellation level, may also be used. In addition, as another example, the surrounding sounds may be output while reducing the effect of the noise cancellation level. The process of reducing the surrounding sound reduction level is to make the phases of the surrounding sounds obtained from the left microphone 14 and the right microphone 15 opposite, synthesize the signal with the reduced sound pressure level with the sound data obtained by the sound input unit 13, and output it to the sound output control unit 26.

[0203] In the above description, the left sound output unit 11 and the right sound output unit 12 are processed to be in a state where the surrounding sounds are easy to hear, but the processing may be performed on either the left sound output unit 11 or the right sound output unit 12. For example, the sound output unit used by the user in his or her dominant ear may be processed to be in a state where the surrounding sounds are easy to hear.

[0204] In the above, the headphone 10 is described as an example of the sound output device, but the present invention is not limited thereto. The sound output device may be, for example, earphones and neck-hanging speakers. The sound output device may also be applied to an electronic earplug device that does not have a sound input unit 13 and only has a noise canceling function.

[0205] Explanation of symbols

[0206] 1: Sound output system;

[0207] 10: Headphones (sound output device);

[0208] 11: Left sound output unit;

[0209] 12: right sound output unit;

[0210] 13: Sound input unit;

[0211] 14: Left microphone;

[0212] 15: right microphone;

[0213] 16: Sensor;

[0214] 20: Sound output control device;

[0215] 21: Towards the detection unit;

[0216] 22: Sound processing unit;

[0217] 23: Sound acquisition unit;

[0218] 24: surrounding sound acquisition unit;

[0219] 25: surrounding sound reduction processing unit;

[0220] 26: Sound output control unit.

Claims

1. A sound output control device, comprising: a direction detection unit that detects a state of a face direction of a user wearing the headset using a sensor provided in the headset; A surrounding sound acquisition unit, which acquires the surrounding sound of the user from a microphone provided in the headset; a surrounding sound reduction processing unit, which performs a process of reducing the surrounding sound on the sound output of the headphone based on the surrounding sound acquired by the surrounding sound acquisition unit; A determination unit, configured to determine that the user is using a means of transportation; as well as A sound output control unit, when the user is using a vehicle, outputs a sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit when the orientation detection unit detects that the state of the user's face orientation is a first state, and when the state of the user's face orientation changes from the first state to a second state, sets the state in which the surrounding sound is easier to hear than in the state in which the surrounding sound is reduced by the surrounding sound reduction processing unit.

2. The sound output control device according to claim 1, wherein: The first state is a state in which the user's face is facing downward. The second state is a state in which the user's face is not facing downward.

3. The sound output control device according to claim 1, wherein: The first state is a state where the acceleration generated by the change in the face orientation of the user is less than a threshold value, The second state is a state in which the acceleration generated by the change in the face orientation of the user is greater than or equal to a threshold value.

4. A sound output control device, comprising: A sight line detection unit detects the sight line direction of the user by means of a sight line sensor arranged to face the same direction as the display screen; a surrounding sound acquisition unit, which acquires the surrounding sound of the user from a microphone provided in a headset worn by the user; a gaze determination unit that determines, based on the detection result of the gaze detection unit, whether the user is gazing at the display screen displaying the content; A determination unit, configured to determine that the user is using a means of transportation; as well as The sound output control unit sets the state in which the user can hear surrounding sounds more easily than when the gaze determination unit determines that the user is not gazing at the display screen displaying the content while the user is using transportation, compared with when the gaze determination unit determines that the user is gazing at the display screen displaying the content.

5. The sound output control device according to claim 4, wherein: The system further includes a surrounding sound reduction processing unit that performs a process of reducing surrounding sounds on the sound to be output by the sound output unit that outputs the sound to the user based on the surrounding sounds acquired by the surrounding sound acquisition unit. The sound output control unit outputs a sound in which the surrounding sound is reduced by the surrounding sound reduction processing unit when the gaze determination unit determines that the user is gazing at the display screen displaying the content, and sets the sound to a state in which the surrounding sound is easier to hear than in a state in which the surrounding sound is reduced by the surrounding sound reduction processing unit when the gaze determination unit determines that the user is not gazing at the display screen displaying the content.

6. The sound output control device according to any one of claims 1 to 3 and 5, wherein: The sound output control unit does not perform the surrounding sound reduction processing of the surrounding sound reduction processing unit, thereby setting a state in which the surrounding sound is more easily heard than in a state in which the surrounding sound is reduced by the surrounding sound reduction processing unit.

7. The sound output control device according to any one of claims 1 to 3 and 5, wherein: The sound output control unit outputs the ambient sound acquired by the ambient sound acquisition unit without performing the ambient sound reduction processing by the ambient sound reduction processing unit, so that the ambient sound is more easily heard than in a state in which the ambient sound is reduced by the ambient sound reduction processing unit.

8. The sound output control device according to claim 4, wherein: It also includes a sound acquisition unit, which acquires sound related to the content. The sound output control unit outputs a sound related to the content acquired by the sound acquisition unit when the gaze determination unit determines that the user is gazing at the display screen displaying the content, and outputs a surrounding sound acquired by the surrounding sound acquisition unit when the gaze determination unit determines that the user is not gazing at the display screen displaying the content.

9. The sound output control device according to claim 8, wherein: The sound output control unit outputs the surrounding sound acquired by the surrounding sound acquisition unit in addition to the sound related to the content acquired by the sound acquisition unit when it is determined that the user is not looking at the display screen displaying the content.

10. The sound output control device according to any one of claims 4, 5, 8 and 9, wherein: The device further includes a face detection unit that recognizes the face of the user and detects the orientation of the recognized face. When the sound output control unit determines that the user is not looking at the display screen displaying the content and the orientation of the face detected by the face detection unit is not an orientation opposite to the display screen, the sound output control unit is set to a state in which it is easier to hear surrounding sounds than when it determines that the user is looking at the display screen displaying the content and the orientation of the face detected by the face detection unit is an orientation opposite to the display screen.

11. A sound output system, comprising: The sound output control device according to any one of claims 1 to 10; The sound pickup part picks up surrounding sounds; as well as The sound output unit outputs sound.

12. A sound output control method, comprising the following steps: detecting a state of a face orientation of a user wearing the headset using a sensor provided in the headset; Acquiring the user's surrounding sound from a microphone provided in the headset; Determining that the user is using a means of transportation; as well as When the user is using a vehicle, when the state of the user's face orientation is a first state, a sound with reduced surrounding sounds compared to the sound output of the headphones is output based on the acquired surrounding sounds, and when the state of the user's face orientation changes from the first state to a second state, a state is set in which the surrounding sounds are easier to hear than in the state in which the surrounding sounds are reduced.

13. A computer-readable storage medium storing a program for causing a computer to execute the following steps: detecting a state of a face orientation of a user wearing the headset using a sensor provided in the headset; Acquiring the user's surrounding sound from a microphone provided in the headset; Determining that the user is using a means of transportation; as well as When the user is using a vehicle, when the state of the user's face orientation is a first state, a sound with reduced surrounding sounds compared to the sound output of the headphones is output based on the acquired surrounding sounds, and when the state of the user's face orientation changes from the first state to a second state, a state is set in which the surrounding sounds are easier to hear than in the state in which the surrounding sounds are reduced.

14. A sound output control method, comprising the following steps: Detecting the gaze direction of a user who is using the headset by means of a gaze sensor arranged to face the same direction as the display screen; Acquiring ambient sound of the user from a microphone provided in the headset worn by the user; Based on the detection result of the sight direction, determining whether the user is looking at a display screen displaying content; Determining that the user is using a means of transportation; as well as When the user is using transportation, if it is determined that the user is not looking at the display screen displaying content, the user is set to a state in which surrounding sounds are more easily heard than when it is determined that the user is looking at the display screen displaying content.

15. A computer-readable storage medium storing a program for causing a computer to execute the following steps: Detecting the gaze direction of a user who is using the headset by means of a gaze sensor arranged to face the same direction as the display screen; Acquiring ambient sound of the user from a microphone provided in the headset worn by the user; Based on the detection result of the sight direction, determining whether the user is looking at a display screen displaying content; Determining that the user is using a means of transportation; as well as When the user is using transportation, if it is determined that the user is not looking at the display screen displaying content, the sound is more easily heard than when it is determined that the user is looking at the display screen displaying content.

Images