Event notification system, program for the event notification system, and method for notifying events.

Abstract

This invention provides an event notification system, method, and program that can notify users of the occurrence of an event without causing them stress. [Solution] In the event notification system 10, a remote control computer 34 notifies an operator 22 working at a remote location 14 of the occurrence of an event where a visitor 18 reaches a reception counter 16 where a reception avatar 20 is located. The remote control computer 34 outputs ambient sound at a relatively high first volume (e.g., -10dB) from a speaker 38 installed at the remote location. A confidence calculation means calculates the confidence level that the visitor will reach the reception counter, and a volume control means reduces the volume of the ambient sound when the confidence level is continuously at a predetermined value. The operator senses the visitor's arrival at the reception counter when the ambient sound is reduced. [Effect] By reducing ambient noise, the system notifies users of the occurrence of an event, thus minimizing stress on the user.

Description

Technical Field

[0001] The present disclosure relates to an event notification system, a program for an event notification system, and an event notification method, and more particularly, for example, to an event notification system, a program for an event notification system, and an event notification method that notify a user of the occurrence of an event by sound.

Background Art

[0002] As ICT (Information and Communication Technology) develops, it has become common to work from a remote location. A cybernetic avatar (remote operation robot: hereinafter sometimes referred to as "CA") has attracted attention by enabling operations and work from a remote location as if an operator were physically present. By utilizing the semi-autonomous control of CA, simple tasks such as greeting people can be entrusted to CA, and the operator can execute multitasks and perform other activities. However, there are cases where the intervention of an operator is required for CA, and in such cases, it is necessary to notify the operator who is executing multitasks.

[0003] Non-Patent Document 1 discloses a system that notifies by "sound" in a work environment. In this system, specifically, a buzzer sound is emitted to notify a person who is working of an event.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

[0005] However, when notifications are made with a loud buzzer, as in Non-Patent Document 1, for example, such notifications can be intrusive and stressful. For instance, a loud buzzer demanding an immediate response can interrupt the operator's current work and cause frustration.

[0006] Therefore, this disclosure provides a novel event notification system, a program for the event notification system, and an event notification method.

[0007] This disclosure provides an event notification system, a program for the event notification system, and an event notification method that can notify users of the occurrence of an event without causing them stress. [Means for solving the problem]

[0008] This disclosure adopts the following configuration to solve the above-mentioned problems. The reference numerals and supplementary explanations in parentheses indicate the correspondence with embodiments described later to aid in understanding the present invention, and do not limit the present invention in any way.

[0009] The first embodiment is an event notification system for notifying the occurrence of an event, comprising an ambient sound output means for outputting ambient sound from a speaker, and a first volume control means for reducing the volume of the ambient sound output from the speaker to a level lower than the volume before the event occurred when the event occurs.

[0010] In the first embodiment, the event notification system (10: a non-limiting reference numeral illustrating a corresponding part in the embodiment; the same applies hereinafter) is a system for notifying a user (22) of the occurrence of an event. Ambient sound output means (52, 78c) output ambient sound (ES) from a speaker (38) at, for example, a remote location (14). At that remote location (14), a user, for example, an operator (22), will hear the ambient sound while, for example, doing other work. The first volume control means (52, 76e, S23) reduces the volume of the ambient sound (ES) output from the speaker to a lower level than before the event occurred when an event occurs.

[0011] According to the first embodiment, the volume of ambient sounds that the user was listening to without particularly noticing them is reduced when an event occurs, thereby making the user aware of the event without being intrusive or causing stress to the user.

[0012] The second embodiment is an event notification system dependent on the first embodiment, further comprising confidence calculation means for calculating the degree of confidence that an event will occur at predetermined time intervals, and the first volume control means reduces the volume of ambient sound when the degree of confidence is continuously at a predetermined value.

[0013] In the second embodiment, the confidence calculation means (52, 76d, S5) calculates the confidence (CF) that an event will occur at predetermined time intervals (for example, every 0.5 seconds), and the first volume control means (52, 76e, S23) reduces the volume of the ambient sound (ES) output from the speaker when the confidence is continuously at a predetermined value (for example, "1").

[0014] According to the second embodiment, the volume of ambient sound is reduced when the confidence level reaches a predetermined value, so that the user can be notified of the occurrence of a definite event.

[0015] A third embodiment is an event notification system dependent on the second embodiment, wherein the event is the arrival of a movable body at a specific location, and the confidence calculation means calculates the confidence that the movable body will arrive at the specific location.

[0016] In the third embodiment, the confidence calculation means (52, 76d, S5) calculates the confidence that the movable body (person or movable robot (avatar)) will reach a specific location, for example, every 0.5 seconds. The first volume control means then reduces the volume of ambient sound when the confidence level is continuously at a predetermined value.

[0017] According to the third embodiment, for example, the system calculates the degree of confidence that a visitor will come to the reception desk, so that the user can be appropriately notified when a visitor approaches the reception desk.

[0018] The fourth embodiment is an event notification system dependent on the second embodiment, wherein the first volume control means controls the volume of ambient sound from a first volume to a second volume lower than the first volume when the confidence level reaches a threshold lower than a predetermined value, and sets the volume of ambient sound to a third volume lower than the first volume when the confidence level remains at a predetermined value for a continuous period.

[0019] In the fourth embodiment, the first volume control means (52, 76e) controls the volume of ambient sound (ES) from a first volume (e.g., -10dB) to a second volume (e.g., -15dB) which is lower than the first volume when the confidence level (CF) reaches a threshold (TH) which is lower than a predetermined value (e.g., "1"). However, when the confidence level remains at the predetermined value for a continuous period, the volume of ambient sound is set to a third volume (e.g., -∞dB) which is lower than the volume.

[0020] According to the fourth embodiment, when the confidence level of event occurrence becomes large enough, by once reducing the volume (from the first volume of -10 dB to the second volume (-15 dB)), the occurrence of the event can be made expected by the user (the user can be made to be on guard for the event occurrence).

[0021] The fifth embodiment is an event notification system that depends on any one of the first to fourth embodiments, and further includes related sound output means for outputting an event-related sound from a speaker, and second volume control means for making the volume of the related sound output from the speaker larger than the volume before the event occurrence at the time of event occurrence.

[0022] In the fifth embodiment, the related sound output means (52, 78c, S2) outputs an event-related sound (EA), that is, a sound that reminds of the occurrence of an event, from the speaker (38). The second volume control means (52, 76e, S24) sets the volume of the related sound (EA) to be larger (for example, -15 dB) than the volume before the event occurrence (for example, -∞ dB) at the time of event occurrence. When the confidence level (CF) reaches a predetermined value, for example, "1", the volume of the related sound (EA) is controlled from the third volume (for example, -∞ dB) until then to the fifth volume (for example, -15 dB).

[0023] According to the fifth embodiment, the occurrence of an event (for example, a visitor coming to the reception counter) can be made more easily noticed by the user.

[0024] The sixth embodiment is a program executed by one or more computers of an event notification system, and the program causes one or more computers to function as environmental sound output means for outputting environmental sound from a speaker, and first volume control means for making the volume of the environmental sound output from the speaker smaller than the volume before the event occurrence at the time of event occurrence, which is a program of an event notification system.

[0025] According to the sixth embodiment, the same effect as the first embodiment can be expected.

[0026] The seventh embodiment is a program of an event notification system that depends on the sixth embodiment. The program further causes one or more computers to function as confidence calculation means for calculating the confidence that an event occurs. When the confidence is continuously a predetermined value, the first volume control means reduces the volume of the ambient sound.

[0027] According to the seventh embodiment, the same effect as the second embodiment can be expected.

[0028] The eighth embodiment is an event notification method for notifying the occurrence of an event. The method outputs ambient sound from a speaker and reduces the volume of the ambient sound output from the speaker to be lower than the volume before the occurrence of the event when the event occurs.

[0029] According to the eighth embodiment, the same effect as the first embodiment can be expected.

Advantages of the Invention

[0030] According to the present disclosure, by reducing the volume of the ambient sound when an event occurs, the user is made aware of the occurrence of the event, so it is not obtrusive and does not cause stress to the user.

[0031] The above objects, other objects, features, and advantages of this invention will become more apparent from the following detailed description of the embodiments made with reference to the drawings.

Brief Description of the Drawings

[0032] [Figure 1] FIG. 1 is a block diagram showing an event notification system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is an illustrative diagram showing an example of an entrance in the embodiment of FIG. 1. [Figure 3] FIG. 3 is an illustrative diagram showing the entrance in the embodiment of FIG. 2 in a planar view. [Figure 4] FIG. 4 is an illustrative diagram showing an example of a remote location in the embodiment of FIG. 1. [Figure 5]Figure 5 is a block diagram showing the electrical configuration of the remote-controlled computer in the embodiment shown in Figure 1. [Figure 6] Figure 6 is an illustrative diagram illustrating the visitor's movement route in the embodiment shown in Figure 1. [Figure 7] Figure 7 is an illustrative diagram illustrating an example of a method for calculating the degree of confidence that a visitor will reach the reception counter based on the travel path shown in Figure 6. [Figure 8] Figure 8 is an illustrative diagram showing an example of controlling the volume of ambient noise in accordance with changes in the confidence level of event occurrence in the embodiment shown in Figure 1. [Figure 9] Figure 9 is an illustrative diagram showing another example of controlling the volume of ambient noise in response to changes in the confidence level of event occurrence in the embodiment shown in Figure 1. [Figure 10] Figure 10 is an illustration showing the change in ambient noise volume before and after reception. [Figure 11] Figure 11 is an illustrative diagram showing an example of the memory map of the remote computer's memory in the embodiment shown in Figure 1. [Figure 12] Figure 12 is a flowchart showing an example of the operation of the remote computer in the embodiment shown in Figure 1. [Figure 13] Figure 13 is an illustrative diagram showing an example of the change in ambient noise volume and event-related noise volume before and after reception in the embodiment shown in Figure 1. [Figure 14] Figure 14 is an illustrative diagram showing another example of the change in ambient noise volume and event-related noise volume before and after reception in the embodiment shown in Figure 1. [Figure 15] Figure 15 is a flowchart showing an example of the operation of a remote computer in the embodiment of Figure 1, taking into account volume control of event-related sounds. [Figure 16] Figure 16 is a block diagram showing a modified example of the embodiment shown in Figure 1. [Best Mode for Carrying Out the Invention]

[0033] Referring to Figure 1, an event notification system 10 (hereinafter sometimes simply referred to as the "notification system") according to one embodiment of the present disclosure is applied to an entrance 12 and a remote location 14 located away from the entrance 12.

[0034] A reception counter 16 is installed in the entrance 12, and a reception avatar 20 that performs reception duties for visitors 18 is positioned inside the reception counter 16. The reception avatar 20 is a semi-autonomous cybernetic avatar (CA) that autonomously performs reception duties via an avatar control computer 40 in response to event notifications from an operator 22 located at a remote location 14.

[0035] However, the reception avatar 20 may be a 3D avatar such as the robot shown in Figure 1, or a 2D avatar such as computer graphics displayed on a screen.

[0036] In relation to the reception avatar 20, a camera 24 is provided above the reception counter 16 in front of it, and a microphone 26 and a speaker 28 are also provided.

[0037] Camera 24 can photograph the entrance 12, and when a visitor 18 approaches the reception counter 16, it can primarily photograph the visitor 18's face and other features.

[0038] Microphone 26 primarily captures the voices of visitors, while speaker 28 outputs the voice of the reception avatar 20 providing customer service.

[0039] Furthermore, one or more three-dimensional position measurement sensors 30 are provided at appropriate locations in the entrance 12. These three-dimensional position measurement sensors 30 may be laser rangefinders or similar laser distance meters, and are provided to determine the walking trajectory of a visitor 18 entering the entrance 12 and to calculate the probability that the visitor 18 will move in front of the reception avatar 20 based on that walking trajectory.

[0040] In this embodiment, an event is recognized when visitor 18 approaches the reception counter 16, i.e., the reception avatar 20. Therefore, the "possibility of visitor 18 approaching the reception avatar 20" here can be said to indicate the degree of confidence that the event will occur.

[0041] At the remote location 14, operator 22 performs their own work using the work computer 32, and simultaneously notifies the reception avatar 20 from the remote control computer 34 that a visitor 18 is approaching the reception counter 16, meaning that an event has occurred in which visitor 18 is coming to the reception desk. Triggered by this event notification from operator 22, the reception avatar 20 faces visitor 18 and performs reception duties. In other words, operator 22 performs multiple tasks.

[0042] In the inventors' experiments, operator 22 notified the avatar control computer 40 of the occurrence of the above-mentioned events from the remote control computer 34, and used the work computer 32 to perform a task, for example, checking for typos (checking for incorrect characters in the text).

[0043] A microphone 36 and a speaker 38 are provided in conjunction with the remote control computer 34. The microphone 36 is for the operator 22 to speak to the visitor 18 through the speaker 28 installed in the entrance 12, if necessary. The speaker 38 is for listening to the audio acquired by the microphone 26 installed in the entrance 12, for example, the conversation between the visitor 18 and the reception avatar 20.

[0044] However, the speech and actions (movements) of the reception avatar 20 are performed autonomously or semi-autonomously according to the speech script and action script pre-configured in the avatar control computer 40.

[0045] The aforementioned 3D position measurement sensor 30, remote control computer 34, and avatar control computer 40 are connected to a wireless or wired network 42 by wireless or wired means.

[0046] Here, with reference to Figures 2 and 3, the arrangement of the reception counter 16 and other elements within the entrance 12 in the embodiment shown in Figure 1 will be explained.

[0047] Entrance 12 is provided with a first entrance 44, a second entrance 46, a third entrance 48, and a fourth entrance 50. The first entrance 44, the second entrance 46, and the third entrance 48 all connect Entrance 12 to the outside. The fourth entrance 50 connects Entrance 12 to the inside of the building.

[0048] As shown in the figure, the entrance 12 is formed in a substantially rectangular shape, and the second entrance 46 is located on the shorter side of the rectangle corresponding to the front of the reception counter 16. The first entrance 44 and the third entrance 48 are located on the longer sides on either side of the shorter side, respectively.

[0049] Entrance 12 also contains chairs and sofas, which can be obstacles for visitors.

[0050] As mentioned above, an operator 22 is located at the remote location 14 shown in Figure 4. When the visitor 18 reaches the reception counter 16, the display 35 of the operator 22's remote control computer 34 shows images 18A and 20A of the visitor 18 and the reception avatar 20, respectively. These images are captured by a camera separate from the camera 24 (Figure 1) installed above the reception counter 16.

[0051] However, until visitor 18 arrives at reception counter 16, the video of visitor 18 captured by camera 24 is displayed on display 35.

[0052] As shown in Figure 5, the remote control computer 34 located at the remote location 14 (Figure 1) includes a CPU 52 that controls the overall system. This CPU 52 is connected to memory 56 and a communication unit 58 via a bus 54. Memory 56 includes RAM, ROM, HDD, etc. The CPU 52 sends commands to the avatar control computer 40 by executing a computer program described later. This program is pre-stored in ROM or HDD and is loaded into RAM and executed as needed. RAM is also used as the working area for the CPU 52. The communication unit 58 is connected to the network 42 via wired or wireless communication.

[0053] An input / output interface (I / F) 60 is further connected to bus 54. The microphone 36 and speaker 38 shown in Figure 1 are connected to the input / output interface 60.

[0054] A sensor interface 62 is also connected to bus 54, and a three-dimensional position measurement sensor 30 is connected to the sensor interface 62.

[0055] Figure 6 illustrates three typical paths 64A, 64B, and 64C that a visitor might take when entering the entrance 12, for example, through the first entrance 44. These paths are tracked and determined by a computer, such as a remote control computer 34, based on information from the 3D position measurement sensor 30.

[0056] Route 64A is a route in which visitors enter through the first entrance / exit 44, initially proceed towards the reception counter 16, but ultimately do not go to the reception counter 16 and continue straight to the third entrance / exit 48. In this route 64A, visitors follow the same route as route 64C, which goes to the reception counter 16, up to a certain point, but then follow a different route.

[0057] Route 64B is a route in which visitors enter through the first entrance 44, initially proceed towards the reception counter 16, but ultimately do not go to the reception counter 16 and continue straight to the fourth entrance 50. In this route 64B, visitors follow the same route as route 64C, which goes to the reception counter 16, up to a certain point, but then follow a different route.

[0058] Route 64C is the route taken by visitors entering through the first entrance 44 and proceeding to the reception counter 16.

[0059] In this embodiment, the remote control computer 34 tracks the visitor according to the information from the 3D position measurement sensor 30, and sequentially calculates the degree of confidence (CF: Certainty Factor or Confidence Factor) that the visitor will reach the reception counter 16. When the visitor reaches the reception counter 16, the computer confirms that an event has occurred.

[0060] Referring to Figure 7, an example of how to calculate the confidence level (CF) will be explained.

[0061] When a person (indicated by a circle in Figure 7) is located at a certain position P in the entrance 12 shown in Figure 2, the probability (confidence level) that this person will move towards one of the five candidate locations (indicated by a star in Figure 7) is calculated as P5 using the following equations (1) and (2). Here, θi represents the angle between the orientation of each candidate location as seen from the person and the predicted direction of the person's movement.

[0062]

number

[0063]

number

[0064] Let Pi = (1 ≤ i ≤ N) be the position vector of the person, vi = (1 ≤ i ≤ N) be the velocity vector, and ai = (1 ≤ i ≤ N) be the acceleration vector. Then, according to equation (3) below, the velocity and acceleration are updated each time the person's position data is acquired.

[0065]

number

[0066] Furthermore, the direction of movement d is estimated according to the following equations (4) and (5).

[0067]

number

[0068]

number

[0069] To explain using the specific example shown in Figure 7, the denominator of equation (1) in the example in Figure 7 is the sum of cosθ1 - cosθ5, where each cosθ1 - cosθ5 represents how close the predicted direction of the person's movement is to the direction of each point as seen from the person's perspective. The value of cosine (cos) decreases as the angle approaches 90°.

[0070] For example, in the example in Figure 7, θ1 = 120°, θ2 = 150°, θ3 = 60°, θ4 = 30°, and θ5 = 10°. Based on these values, the calculation is as shown in equation (6). [Number 6] u(cosθ1)=0 (because cosθ1=-0.5) u(cosθ2)=0 (because cosθ2=-0.866) u(cosθ3)=0.5 u(cosθ4)=0.866 u(cosθ5)=0.985 In the example shown in Figure 7, when the person is almost facing the reception counter 16, the angle is θ5, and as calculated in equation (6) above, the cosine value is largest at θ5. However, angles θ3 and θ4 are also still large, so the value in equation (1) above is P5 = 0.42.

[0071] If a person approaches the reception counter 16, angles θ3 and θ4 will also approach 90°, and accordingly, the value in equation (1) will approach "1". Therefore, when the value P5 in equation (1) approaches "1", it can be determined that a person is likely to come to the reception counter 16, and this is used as the confidence level that a person will come to the reception.

[0072] The direction of movement is estimated using the formula derived from the average values ​​of past velocities and accelerations, which can be calculated using equation (3). Specifically, the average velocity is added to the average acceleration to obtain the predicted velocity. However, the average velocity and average acceleration are calculated as weighted averages.

[0073] For example, if N=10, instead of simply adding up the past 9 velocities and calculating the average, the velocity from one time point ago (in this example, calculations are performed at 0.5-second intervals, so 0.5 seconds ago) is given a larger weight, and the velocity from 9 time points ago is given a smaller weight, and these are added together and averaged. This is done to place more confidence in the values ​​from the recent past and less confidence in the values ​​from the distant past.

[0074] Specifically, if N=9, the sum of v1-v9 is (9 / 45)v1+(8 / 45)v2+(7 / 45)v3+…+(1 / 45)v9=0.2v1+0.18v2+0.16v3+…+0.022v9. This is an averaging method that places more weight on recent past values ​​than calculating v1+v2+v3+…+v9. A similar weighted average was calculated for acceleration.

[0075] As described above, in equation (1), the value of P5 indicates the degree of confidence (sometimes simply referred to as "degree of confidence") that a person will reach the reception counter 16.

[0076] In the notification system 10 shown in Figure 1, the operator 22 is notified of the occurrence of an event in which a visitor 18 has come to the reception counter 16 by controlling the volume of ambient sound (ES) output from speaker 38 at remote location 14 according to such confidence level CF. Ambient sound ES refers to sounds that are audible but not noticeable, and examples include the sounds of a crowd, noise in a restaurant, rain, and white noise. It is sometimes called ambient noise (EN).

[0077] In Figure 8, the changes in the confidence level CF and the ambient sound ES are represented by lines 66 and 68, respectively. Here, the ambient sound ES refers to the volume of ambient sound output from speaker 38 at remote location 14. These points are the same in Figures 9-10 and 13-14.

[0078] In Figure 8(A), until the confidence level CF reaches a threshold TH less than "1", the volume of the ambient sound ES is output at a first volume (e.g., -10dB) that is slightly lower than the normalized volume (NV: specified volume), as shown in Figure 8(B). From the timing t1 when the confidence level CF exceeds the threshold TH, over a time tp1, the volume of the ambient sound ES is set to a second volume (e.g., -15dB) that is lower than the first volume (-10dB). However, in this embodiment, the time tp1 required to decrease the volume of the ambient sound ES is set to 0.5 seconds, the same as the time tp2 required to increase the volume of the ambient sound ES, which will be described later. This is because the confidence level prediction in equation (1) is performed every 0.5 seconds.

[0079] It is well known that ambient sounds at a moderate volume (ES) have the effect of increasing the concentration of the listener. Here, we play ambient sounds (ES) at a first volume level that has such an effect, which is lower than the normalization volume (NV), for example, -10dB.

[0080] In the example shown in Figure 8(A), the confidence level CF exceeds the threshold TH but does not reach "1", so after timing t2, it falls below the threshold TH. This is the case when, as shown in routes 64A and 64B in Figure 6, the calculation result by equation (1) predicts that the person (visitor 18) will come part of the way to the reception counter 16, but ultimately does not come to the reception counter 16 and proceeds to the third entrance 48 or the fourth entrance 50.

[0081] In the example in Figure 8, although the confidence level CF exceeds the threshold TH between timings t1 and t2, the confidence level CF never reaches "1", which would confirm the occurrence of the event. Therefore, the volume of the ambient sound ES returns to the first volume after timing t2.

[0082] In other words, the example in Figure 8 shows a case where there is a steady decrease (-5dB) in the volume of ambient sound ES from the first volume to the second volume output from speaker 38, but the decrease is not noticeable to the operator, and no event occurred.

[0083] In contrast, the example shown in Figure 9 illustrates the case where an event has occurred.

[0084] In the example shown in Figure 9, as shown in Figure 9(A), the confidence level CF exceeds the threshold TH at timing t1, and then the confidence level CF reaches "1" at timing t2. Therefore, the volume of the ambient sound ES decreases from the initial first volume (-10dB) to the second volume (-15dB) over time tp1 from timing t1, as shown by line 68 in Figure 9(B), and then decreases over time tp2 to almost zero (0) (third volume), indicated by -∞, at timing t2 when the confidence level CF becomes "1".

[0085] In the example in Figure 9, at timing t2 when the confidence level CF becomes "1", the volume of the ambient sound ES becomes the third volume (for example, zero (-∞)), causing a sense of unease in the hearing of operator 22, who is working while listening to the ambient sound ES (in a state of not actively listening). In other words, operator 22, who is working, can notice that something has happened because the ambient sound ES, which was previously at a comfortable volume (first volume), suddenly becomes inaudible at timing t2. Therefore, operator 22 can look at the display 35 of the remote control computer 34 and confirm that the visitor 18 is approaching the reception counter 16. In short, operator 22 can sense the occurrence of an event even while working on other tasks.

[0086] Conventional technologies that notify of events with a loud buzzer sound can be intrusive and stressful. However, the notification method in this embodiment is less intrusive and causes less stress to the operator.

[0087] When the confidence level CF becomes "1" at timing t2, and the volume of the ambient sound ES is controlled to the third volume level as shown by line 68, operator 22 senses the occurrence of an event and transmits this information from the remote control computer 34 to the avatar control computer 40 via the network 42. Consequently, the reception avatar 20, controlled by the avatar control computer 40, performs reception services for the visitor 18.

[0088] The reason why the first volume of the ambient sound normally played (output) from speaker 38 is set to -10dB instead of the normalized volume NV is that, as in the example described later, when using event-related sound EA, the event-related sound played when an event occurs is made louder than the normal ambient sound, making it easier to notice the event-related sound.

[0089] For example, in Figure 14 described later, at time t2+tp2, the volume of the event-related sound EA is set to -5dB. At this time, by setting the volume of the normal ambient sound ES to -10dB, the event-related sound EA will be played at a louder volume than that. This setting was chosen because sounds louder than the normalized volume NV (0dB) are judged to be too loud for the user. However, this is not the only volume setting that is possible.

[0090] Furthermore, the reason for setting the second volume level to -15dB is that when the confidence level of an event occurrence becomes sufficiently high, the volume is lowered (from the first volume level of -10dB to the second volume level of -15dB) with the expectation that the user will anticipate the event (be prepared for the event to occur). Rather than the volume suddenly changing when an event occurs, the aim is to make it easier for the user to react to the event by slightly lowering the volume when the confidence level of an event occurrence has risen to a certain extent, thus giving them a sense that an event is likely to occur.

[0091] However, the utterances made by the reception avatar 20 to the visitor 18 are performed according to a speech script pre-configured in, for example, the avatar control computer 40. However, the actual reception process itself is not the essence of this embodiment, so a detailed explanation will be omitted.

[0092] As shown by line 68 in Figure 10, at timing t3, when the reception avatar 20 has finished handling the visitor 18, the volume of the ambient sound ES is returned to its original first volume. This timing t3 occurs when the avatar control computer 40 detects that the reception avatar 20 has finished handling the visitor and notifies the remote control computer 34 via the network 42. Therefore, the remote control computer 34 can perform volume control to return the ambient sound ES volume to its original level.

[0093] However, the timing t3 may be determined by the operator 22, who is viewing the display 35 (Figure 4) of the remote control computer 34, and volume control may be executed at that timing.

[0094] Figure 11 is an illustrative diagram showing an example of the memory map of the memory 56 of the remote control computer 34 shown in Figure 5. As shown in Figure 11, the memory 56 includes RAM as described above, and this RAM includes a program storage area 72 and a data storage area 74. The control program of the remote control computer 34 is stored in the program storage area 72. The control program and necessary data may be pre-configured in, for example, the flash memory or hard disk included in the memory 56, and then read out as needed and loaded into the RAM.

[0095] The control program for the remote control computer 34 includes a display control program 76a, an operation detection program 76b, a voice recognition program 76c, a confidence level calculation program 76d, a volume control program 76e, and the like.

[0096] The display control program 76a is a program that displays images on the display 35 (Figure 4).

[0097] The operation detection program 76b is a program that detects operations performed on the input device (such as a keyboard) of the remote control computer 34.

[0098] The speech recognition program 76c is a program that converts speech data acquired by speech recognition into text data, and if necessary, records it in the temporary storage area 78d of the data storage area 74 as, for example, a record of the speech of the reception avatar 20 and the visitor 18.

[0099] However, the voice data of the reception avatar 20 and the visitor 18 can be acquired by the microphone 26 (Figure 1) and transmitted from the avatar control computer 40 to the remote control computer 34 via the network 42.

[0100] Furthermore, it is also possible to use commercially available software such as Google's (trademark) speech recognition software as the speech recognition program 76c.

[0101] The confidence calculation program 76d is a program for calculating the confidence level CF of an event occurrence according to the aforementioned equation (1), etc.

[0102] As explained earlier, the volume control program 76e is a program for controlling the volume of the ambient sound ES to predetermined values ​​at timings t1, t2, and t3.

[0103] However, the sound source data for the ambient sound ES is assumed to be pre-set in the sound source data storage area 78c of the data storage area 74. This sound source data may consist of only one type, or multiple sound source data may be set to be selectable.

[0104] Although not shown in the diagram, the program storage area 72 also stores other programs necessary for controlling the remote control computer 34.

[0105] The data storage area 74 includes an image generation data storage area 78a, an operation data storage area 78b, a sound source data storage area 78c, a temporary storage area 78d, and the like.

[0106] The image generation data storage area 78a stores image generation data, including polygon data and texture data that are pre-configured for generating data for various screens to be displayed on the display 35.

[0107] The operation data storage area 78b stores data input from the input device in chronological order.

[0108] As described above, the sound source data of the ambient sound ES is stored in the sound source data storage area 78c, and the ambient sound ES is output from the speaker 38 (Figure 1) based on this sound source data. The volume control program 76e controls the volume of the ambient sound ES output from the speaker 38.

[0109] The temporary memory area 78d stores not only the speech recognition results mentioned above, but also the results of each calculation, such as the confidence score CF.

[0110] Furthermore, the data storage area 74 stores other data necessary for executing the control program of the remote control computer 34, as well as flags and other counters (timers) necessary for executing the control program.

[0111] Next, referring to Figure 12, the operation of the remote control computer 34, primarily according to the volume control program, will be explained. Note that, as explained earlier, the operation shown in Figure 12 will be performed every 0.5 seconds as an example.

[0112] In the first step S1, the CPU 52 (Figure 5) outputs the ambient sound ES from the speaker 38 at a first volume level (-10dB) lower than the normalized volume level NV, according to the volume control program 76e.

[0113] In step S3, when sensor data is input from the 3D position measurement sensor 30 (Figure 1), in the following step S5, the CPU 52 calculates the confidence score CF according to the confidence score calculation program 76d. The calculated confidence score CF is stored in the temporary storage area 78d of the data storage area 74.

[0114] In the next step, S7, CPU52 determines whether the confidence level CF calculated in this step is "1".

[0115] If the decision in step S7 is "NO," that is, if the confidence level CF calculated this time is not "1," then in step S9, the CPU 52 determines whether the current confidence level CF is equal to or greater than the threshold TH.

[0116] If "NO" is determined in step S9, that is, if the confidence level CF of the current calculation does not reach the threshold TH, then in step S11, it is determined whether the confidence level CF from the previous time (1 time (0.5 seconds) ago) was less than the threshold TH. If "YES", then in the next step S13, the CPU 52 sets the volume of the ambient sound ES from the normalized volume NV to -10dB (first volume), which is less than that. After that, the process proceeds to step S15, just as when "NO" was determined in step S11.

[0117] In step S15, the previous confidence CF is written to the temporary storage area 78d (Figure 11) of the current confidence CF.

[0118] After step S15, in step S3, the system waits for sensor data to be input from the 3D position measurement sensor 30 (Figure 1).

[0119] If the CPU 52 determines "YES" in the previous step S9, that is, if it determines that the confidence level CF calculated this time is equal to or greater than the threshold TH, then in the next step S17, the CPU 52 determines whether the confidence level CF calculated last time is equal to or greater than the threshold TH. If the CPU 52 determines "YES" in this step S17, the CPU 52 sets the volume of the ambient sound ES to the second volume level (-15dB), for example, as shown in Figure 8(B) or Figure 9(B).

[0120] Then, just as when the decision was made to be "NO" in step S17, the process proceeds to step S15.

[0121] In this state, the confidence level of the event will never reach the predetermined value of "1," so the volume of the ambient sound ES will not be reduced to the third volume level.

[0122] If the previous step S7 determined "YES", in the next step S21, the CPU 52 determines whether the previous confidence level CF was also "1". This determination can be made by referring to the confidence level stored each time in the temporary storage area 78d (Figure 11) of the data storage area 74.

[0123] If both step S7 and step S21 determine "YES", the confidence level CF is consecutively "1", so in the following step S23, the CPU 52 sets the volume of the ambient sound ES from the second volume to the third volume (-∞ (zero)).

[0124] However, if “NO” is determined in step S21, the CPU 52 proceeds to step S15, just as after executing step S23, and places the temporary storage area 78d. Write the current confidence flow statement as the previous confidence flow statement.

[0125] Thus, according to the volume control program 76e, when the confidence level CF becomes "1" consecutively, for example at timing t2 in Figure 9(B), the volume of the ambient sound ES is controlled to the third volume level (-∞ (zero)).

[0126] In other words, in this embodiment, when the confidence level CF is "1" consecutively, the operation can be used to notify the operator 22 of the occurrence of an event by controlling the volume of the ambient sound ES to the third volume level (-∞ (zero)).

[0127] In this way, the confidence calculation means (52, 76d, S5) calculates the confidence level (CF) of an event occurring at predetermined time intervals (for example, every 0.5 seconds), and when the confidence level is continuously at a predetermined value, it reduces the volume of the ambient sound (ES) output from the speaker 38. By lowering the volume of the ambient sound when the confidence level reaches a predetermined value, the user can be notified of the occurrence of a definite event.

[0128] Figure 13 is an illustrative diagram showing an example of the change in volume of ambient sound ES and event-associated sound (EA) before and after reception in the embodiment shown in Figure 1, and Figure 14 is an illustrative diagram showing another example. Figures 13 and 14 correspond to Figures 8(B) and 9(B), respectively.

[0129] In the embodiments shown in Figures 13 and 14, the volume of the ambient sound ES and the volume of the event-related sound (sometimes simply called "related sound") EA are controlled according to the confidence level CF. However, the related sound EA is a sound that evokes the occurrence of an event; in embodiments where it is determined whether a person is coming to a specific location, for example, it is footsteps. The sound source data for this related sound EA, such as footsteps, can also be pre-set in the sound source data storage area 78c of the data storage area 74 (Figure 11) of the memory of the remote control computer 34. The CPU 52 (Figure 5) then controls the volume of not only the ambient sound ES but also the related sound EA according to the volume control program 76e (Figure 5) in response to changes in the confidence level CF. In other words, the volume control program 76e is a program that controls the volume of the ambient sound ES and the related sound EA. The CPU 52 that controls the volume of the ambient sound ES according to the volume control program 76e can be called the first volume control means, and the CPU 52 that controls the volume of the related sound EA according to the volume control program 76e can be called the second volume control means.

[0130] In the example in Figure 13, at timing t1, when the confidence level CF exceeds the threshold TH, the ambient sound ES is reduced to the second volume, for example, -15dB, as shown by line 68. Until timing t1, the related sound EA is at the fourth volume (-∞ (zero)), and at timing t1, as shown by line 70, it is output at a predetermined volume, in this example, the fifth volume (-15dB). For convenience, the fourth and fifth volumes of the related sound EA are set to be the same as the third and second volumes of the ambient sound ES, but they do not need to be the same. In the example in Figure 13, the confidence level CF never becomes "1" afterward, and at timing t2, the confidence level CF becomes smaller than the threshold TH. Accordingly, the ambient sound ES is output at its original volume, the first volume (-10dB), as shown by line 68, and the related sound EA is set to its original fourth volume (-∞), as shown by line 70.

[0131] In other words, the example in Figure 13 shows the volume changes of ambient sound ES and related sound EA when the confidence level CF never becomes "1".

[0132] In contrast, the example in Figure 14 shows the volume changes of the ambient sound ES and related sound EA when the confidence level CF becomes "1" at timing t2.

[0133] Similar to the example in Figure 9, at timing t2, when the confidence level CF becomes "1", the volume of the ambient sound ES is controlled to the third volume level (-∞ (zero)). On the other hand, the volume of the related sound EA, shown by line 70, is increased to the fifth volume level (for example, -15 dB) at this timing t2.

[0134] In the example shown in Figure 14, when the confidence level CF becomes "1", the volume of the ambient sound ES is controlled to be as low as possible, and the volume of the associated sound EA is controlled to be as high as possible.

[0135] In the examples in Figures 13 and 14, the associated sound EA is intended to create a cocktail party effect, for example, by increasing the volume of footsteps, in order to make it easier for operator 22 to notice the occurrence of the event. According to Wikipedia, the cocktail party effect is the selective listening of sounds, and is said to have been proposed by psychologist Colin Cherry.

[0136] In this embodiment, "footsteps" were adopted as an example of related sound EA. In a remote operation scenario, human footsteps can represent the arrival of a visitor 18, which is the target of the operator 22's detection. On the other hand, although the ambient sound ES creates a somewhat noisy environment, when the visitor 18 approaches the reception counter 16, the footsteps (related sound EA) emerge from the ambient sound ES, thereby naturally directing the operator 22's attention to the visitor.

[0137] To this end, in this embodiment, when no visitor is approaching, the ambient sound ES is played at a volume of, for example, -10 dB (first volume). However, as the visitor 18 approaches the reception counter 16, the volume of the footsteps (related sound EA) is increased, while the volume of the ambient sound ES is decreased, thereby increasing the signal-to-noise ratio (S / N ratio) of the footsteps (related sound EA) compared to the ambient sound ES.

[0138] Thus, in addition to reducing the volume of the ambient sound ES when the visitor 18 approaches the reception counter 16 as in the previous embodiment, increasing the volume of the related sound EA makes it easier to detect the occurrence of an event, particularly the approach of the visitor 18, to the operator 22 who is working.

[0139] As explained above, the reason why the fifth volume level of the related sound EA is -15dB instead of -10dB under normal circumstances is that when the confidence level of event occurrence becomes sufficiently high, the related sound EA is set to the same volume as the ambient sound ES, thereby burying the related sound EA in the ambient sound ES and preventing the user from clearly recognizing the related sound EA, thus allowing the user to anticipate the event. The aim is to make it easier for the user to react to the event by not making the related sound EA clearly audible and allowing them to anticipate that an event is likely to occur. In other words, the reason why the fifth volume level is not -10dB is because, as explained above, the ambient sound is set to -15dB, and the level is set to match that.

[0140] Figure 15 shows a flowchart illustrating an example of the operation of the remote control computer 34 (volume control operation by the CPU 52) that performs the operations of the embodiments shown in Figures 13 and 14. However, for each step that shows the same or similar operation as in Figure 12, the same reference numerals (step numbers) as in Figure 12 are used to omit redundant explanations.

[0141] Following the first step S1 in Figure 15, in step S2, the CPU 52, according to the volume control program 76e (Figure 11), uses the footstep data pre-set in the sound source data storage area 78c (Figure 11) to play the associated sound EA, which is the footstep, at the fourth volume level (-∞ (zero)). In other words, the associated sound EA is not played in the initial state.

[0142] In step S7, we determine whether the confidence level CF calculated in step S5 is "1".

[0143] If the decision in step S7 is "NO", then in the following step S9, the CPU 52 determines whether the current confidence level CF is equal to or greater than the threshold TH.

[0144] If "NO" is determined in step S9, as explained earlier, in step S11, it is determined whether the confidence level CF from the previous time (1 time (0.5 seconds) ago) is less than the threshold TH. If "YES" is determined, in the next step S13, the CPU 52 sets the volume of the ambient sound ES to -10dB (first volume), and in step S14, the volume of the related sound EA remains at the fourth volume (-∞ (zero)).

[0145] Then, proceed to step S25, just as you did when you determined "NO" in step S11.

[0146] If the CPU 52 determines in step S9 that "YES," that is, that the current confidence level CF is "1," then, as explained earlier, in step S17, it determines whether the previous confidence level CF was above the threshold TH. If the CPU 52 determines "YES" in step S17, in step S19, it sets the volume of the ambient sound ES to the second volume level (-15dB), and in the following step S20, it sets the volume of the related sound EA to the fifth volume level (-15dB).

[0147] Then, proceed to step S25, just as you did when you determined "NO" in step S17.

[0148] Furthermore, when the decision in step S7 is "YES," that is, when the confidence level CF calculated this time is determined to be "1," in the next step S21, the CPU 52 determines whether the previous confidence level CF was also "1."

[0149] If the CPU 52 determines "YES" in both step S7 and step S21, in step S23, it controls the volume of the ambient sound ES to, for example, the third volume level (-∞dB), as described above, and in step S24, it sets the volume of the related sound EA to the fifth volume level (-15dB) (the part shown by the horizontal line in Figures 13 and 14).

[0150] Thus, when the confidence level CF reaches a predetermined value, "1" in this embodiment, the volume of the associated sound EA is controlled from the previous third volume level (for example, -∞dB) to the fifth volume level (for example, -15dB), making it easier for the operator 22 to notice the occurrence of an event (for example, a visitor 18 coming to the reception counter 16).

[0151] If "NO" is determined in step S21, the process proceeds to step S15, and then waits for sensor data input from the 3D position measurement sensor 30 (Figure 1).

[0152] Figure 15 illustrates a block diagram showing a modified example of the event notification system 10 of the embodiment shown in Figure 1.

[0153] The difference between the modified example in Figure 15 and the event notification system 10 in the embodiment of Figure 1 is that the sensor data from one or more three-dimensional position measurement sensors 30 installed in the entrance 12 is input to the confidence calculation computer 80, rather than to the remote control computer 34.

[0154] The confidence calculation computer 80 is configured with the confidence calculation program 76d described in the previous embodiment, and this confidence calculation computer 80 calculates the confidence level CF at which an event occurs, according to the method described using, for example, equation (1).

[0155] The embodiment shown in Figure 16 is effective when the processing power of the remote control computer 34 (Figures 1 and 5) is insufficient, by entrusting the calculation of the confidence level, which takes a long time, to the confidence level calculation computer 80.

[0156] In the above embodiment, the ambient sound ES and related sound EA were to use sound source data pre-set in memory 56. However, the sound source data for such ambient sound ES and related sound EA may also be acquired and used via network 42.

[0157] Note that the processing steps in the flowcharts shown in Figures 12 and 15 are merely examples, and the order of processing steps may be changed if similar results can be obtained.

[0158] Furthermore, the specific numerical values ​​such as angles, durations, and volume mentioned above are merely examples and can be changed as needed.

[0159] Furthermore, in the embodiments described above, the event to be notified was given as an example of a visitor reaching the reception counter, but other situations can be envisioned as events. For example, it is also possible to detect an event when a mobile robot (avatar) reaches a specific location.

[0160] However, the events described in this disclosure are not limited to events in which a person or a mobile object such as a robot (avatar) reaches a specific location. For example, it can be applied to situations where something is being monitored. When there is a task to switch something when a specific measurement value of a control plant reaches a certain value T, the confidence level that the measurement value will reach T can be calculated from the past time-series information of the measurement value, and as the confidence level approaches "1", the timing of the switching operation can be indicated without being intrusive using the method described above. [Explanation of Symbols]

[0161] 10…Event notification system 12… Entrance 16 ... Reception counter 18 ... Visitors 26, 36... Mike 28, 38... Speakers 30...3D position measurement sensor 34… Remote control computer 52 ...CPU 56…memory 76d ... Confidence calculation program 76e ... Volume control program 78c ... Audio data

Claims

1. An event notification system that notifies of the occurrence of an event, An ambient sound output means for outputting ambient sounds from a speaker, and An event notification system comprising a first volume control means that reduces the volume of the ambient sound output from the speaker to a lower level than the volume before the event occurred when an event occurs.

2. The system further includes a confidence calculation means for calculating the degree of confidence that an event will occur at predetermined time intervals. The event notification system according to claim 1, wherein the first volume control means reduces the volume of the ambient sound when the confidence level is continuously at a predetermined value.

3. The aforementioned event is the moving object reaching a specific location. The event notification system according to claim 2, wherein the confidence calculation means calculates the confidence that the movable body reaches the specific location.

4. The event notification system according to claim 2, wherein the first volume control means controls the volume of the ambient sound from a first volume to a second volume that is smaller than the first volume when the confidence level reaches a threshold smaller than the predetermined value, and sets the volume of the ambient sound to a third volume that is smaller than the second volume when the confidence level is continuously at the predetermined value.

5. A related sound output means for outputting event-related sounds from a speaker, and The event notification system according to any one of claims 1 to 4, further comprising a second volume control means for increasing the volume of the associated sound output from the speaker to a level higher than the volume before the event occurred when an event occurs.

6. A program executed by one or more computers of an event notification system, The program uses the one or more computers described above. An ambient sound output means for outputting ambient sounds from a speaker, and A program for an event notification system that causes the volume of the ambient sound output from the speaker to function as a first volume control means that reduces the volume of the ambient sound to a level lower than the volume before the event occurred when an event occurs.

7. The event is that the movable body reaches a specific location, and the program causes the one or more computers to function as confidence calculation means for calculating the degree of confidence that the movable body reaches the specific location. The program for the event notification system according to claim 6, wherein the first volume control means reduces the volume of the ambient sound when the confidence level is continuously at a predetermined value.

8. An event notification method for notifying the occurrence of an event, The speaker outputs ambient sounds, and An event notification method that reduces the volume of the ambient sound output from the speaker to a lower volume than before the event occurred when the event occurs.

Images