Voice guidance system, voice guidance method and voice guidance program

Abstract

To enable voice guidance to effectively function for each user by performing the voice guidance by making a user with visual impairment recognize that the voice guidance being output is the voice guidance for himself / herself.SOLUTION: A voice guidance system comprises: a camera device which captures an image in which one or a plurality of users go by; a speaker device which outputs voice guidance; and an analysis device which outputs the voice guidance corresponding to the analysis result of the photographed image via the speaker device. An image analysis unit of the analysis device analyzes the image captured by the camera device to detect whether or not the user has visual impairment, the current position of the user with visual impairment and a feature of the user with visual impairment. An output voice control unit outputs the voice guidance corresponding to the current position of the user with visual impairment by making the detected user with visual impairment recognize that the voice guidance indicating the detected feature is the voice guidance for himself / herself by outputting the voice guidance via the speaker device.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a voice guidance system, a voice guidance method, and a voice guidance program. [Background technology]

[0002] Nowadays, for example, inside or outside a station or facility, voice guidance such as "The ticket gate is 5 meters ahead" or "The pedestrian crossing is 3 meters ahead" is provided to visually impaired users.

[0003] However, even if there is no visually impaired user nearby, if the voice guidance is repeated, the voice guidance may become a distraction to able-bodied people, store clerks in nearby stores, nearby residents, etc. This is also undesirable from an environmental perspective.

[0004] For this reason, the voice guidance system for visually impaired people disclosed in Patent Document 1 (JP Patent Publication No. 2020-125907) provides voice guidance to a user (visually impaired person) passing through a station premises according to the moving direction of the user. This allows the voice guidance system for visually impaired people to avoid providing unnecessary voice guidance to users passing through a station premises, making it possible to reduce the output of unnecessary voice guidance. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-125907 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional voice guidance systems, including the voice guidance system for visually impaired people described in Patent Document 1, had a problem in that it was difficult for visually impaired users to recognize whether the voice guidance they were hearing was intended for them or for other visually impaired users.

[0007] For example, suppose that multiple visually impaired users are located in the same place, and one user and the other user are walking in different directions. In this situation, suppose that one user is given voice guidance such as "The ticket gate is located 5 meters ahead." If the voice guidance given to one user is mistakenly recognized by the other user, who is walking in the opposite direction to the first user, as being intended for him / herself, the other user will be inconvenienced and will not be able to reach the ticket gate even if he / she walks 5 meters ahead.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a voice guidance system, a voice guidance method, and a voice guidance program that allow visually impaired users to recognize that the voice guidance being output is for them, and that allows the voice guidance to function effectively for each user. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the objective, the present invention comprises a camera device that captures images of users passing by, a speaker device for outputting voice guidance, and an analysis device that, if it is determined that a user appearing in an image captured by the camera device is a visually impaired user, detects characteristics of the user and outputs pre-recognition voice guidance indicating the detected characteristics via the speaker device. Effect of the Invention

[0010] According to the present invention, it is possible to make a visually impaired user recognize that the voice guidance being output is intended for him / her, thereby making it possible to effectively provide voice guidance for each user. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration of a voice guidance system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing another system configuration of the voice guidance system according to the embodiment. [Diagram 3] FIG. 3 is a block diagram of an analysis device provided in the voice guidance system according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a format of map data stored in the analysis device. [Diagram 5] FIG. 5 is a functional block diagram of each function realized by the CPU of the analysis device executing the voice guidance program. [Figure 6] FIG. 6 is a flowchart showing a flow of a voice guidance operation of the voice guidance system according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining an example of the timing of voice guidance and the contents of voice guidance in the voice guidance system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, a voice guidance system according to an embodiment of the present invention will be described with reference to the drawings.

[0013] (System Configuration) 1 is a diagram showing an example of a system configuration of a voice guidance system according to an embodiment. As shown in FIG. 1, the voice guidance system according to the embodiment includes a camera device 1, a speaker device 2, and a terminal device 50 including an analysis device 3.

[0014] The camera device 1 is, for example, a fixed camera device, and captures images of users passing through a passageway or the like within a fixed imaging area. The camera device 1 may be a camera device whose imaging area is changeable. The speaker device 2 outputs audio for voice guidance. The analysis device 3 analyzes the characteristics of a visually impaired user. The analysis device 3 then provides the voice guidance to the visually impaired user after making the user aware that the voice guidance is directed to the user. This will be described in more detail later.

[0015] As an example, the terminal device 50 is formed by physically arranging the camera device 1 to the analysis device 3 in one housing. The terminal devices 50 are installed at various locations where passing users can be monitored, such as in a train station or at the entrance of a large store.

[0016] Such a voice guidance system may operate independently (stand-alone) without being connected to a network or other devices, or may be connected to an administrator terminal device 4 installed in a management room or the like, as shown by the dotted line in Figure 1.

[0017] (Other examples of system configurations) Fig. 2 is a diagram showing another example of the system configuration of the voice guidance system according to the embodiment. In the example of Fig. 2, the voice guidance system is configured by connecting a plurality of terminal devices 60 and an analysis device 3, which is a manager terminal device installed in, for example, a management room, via a wide area network such as the Internet or a private network such as a LAN (Local Area Network).

[0018] The terminal device 60 is provided with a camera device 1 and a speaker device 2. The camera device 1 is, for example, a fixed camera device, and captures images of users passing through a passage or the like within a fixed imaging area. The camera device 1 may be a camera device whose imaging area is changeable. The speaker device 2 outputs audio guidance.

[0019] The analysis device 3 analyzes the image captured by the camera device 1 of each terminal device 60 to analyze the characteristics of the visually impaired user. Then, the analysis device 3 controls the speaker device 2 of the corresponding terminal device 60 so that the visually impaired user is made aware that the target of the voice guidance is him / herself before providing the voice guidance. This will be described in detail later.

[0020] (Hardware configuration of the analysis device) Fig. 3 is a block diagram showing a hardware configuration of the analysis device 3. As shown in Fig. 3, the analysis device 3 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and a communication unit 14. The analysis device 3 also includes an HDD (Hard Disk Drive) 15, an input / output interface (input / output I / F) 16, and a communication interface (communication I / F) 17.

[0021] The communication unit 14 performs wired communication via a network such as the Internet or a LAN, as well as wireless communication via Bluetooth (registered trademark) or Wi-Fi (registered trademark). The HDD 15 stores a voice guidance program, map data, and voice guidance data for providing voice guidance to visually impaired users.

[0022] As shown in FIG. 4, the map data is stored in association with facility information and location information, including names or titles of facilities, tenants, ticket gates, elevator equipment, etc. located in the geographical area where voice guidance is provided, and output condition information indicating the conditions for providing voice guidance for the facilities.

[0023] If necessary, a display unit 18 and an operation unit 19 are connected to the input / output I / F 16. If necessary, the communication I / F 17 is connected to the network 5 via a network cable.

[0024] (Functional configuration of the analysis device) Fig. 5 is a functional block diagram of each function realized by software when the CPU 11 executes the voice guidance program stored in the HDD 15. As shown in Fig. 5, the CPU 11 executes the voice guidance program to function as a video acquisition unit 21, a map data acquisition unit 22, an image analysis unit 23, an output voice control unit 24, and a communication control unit 25.

[0025] The video acquisition unit 21 acquires captured images of users traveling within the geographical range captured by each camera device 1. The map data acquisition unit 22 acquires map data of the geographical range captured by each camera device 1 from the HDD 15. The image analysis unit 23 analyzes the user's characteristics and the presence or absence of visual impairment, etc., based on the captured images captured by each camera device 1. The output voice control unit 24 reads out guidance voice data corresponding to the analysis result of the image analysis unit 23 from the HDD 15 and controls output via the speaker device 2. The communication control unit 25 issues an emergency notification to the administrator terminal device 4, etc., when an action of a visually impaired user, etc., requesting help is analyzed.

[0026] In this example, the video acquisition unit 21 to the communication control unit 25 are realized by software using a voice guidance program. However, all or a part of these may be realized by hardware such as an IC (Integrated Circuit).

[0027] The voice guidance program may be provided by recording it in the form of file information in an installable or executable format on a computer-readable recording medium such as a CD-ROM or a flexible disk (FD). The voice guidance program may be provided by recording it in the form of file information in an installable or executable format on a computer-readable recording medium such as a CD-R, a DVD (Digital Versatile Disc), a Blu-ray (registered trademark) disk, or a semiconductor memory. The voice guidance program may be provided by installing it via a network such as the Internet. The voice guidance program may be provided by being pre-installed in a ROM or the like within the device.

[0028] (Voice guidance operation) Fig. 6 is a flowchart showing the flow of the voice guidance operation in the voice guidance system of the embodiment. In Fig. 6, in step S1, the video acquisition unit 21 acquires a captured image of a user (passerby) captured by each camera device. In step S2, the image analysis unit 23 analyzes the captured image acquired in step S1. Then, in step S3, the image analysis unit 23 determines whether the user shown in the captured image is a normal person or a visually impaired user.

[0029] As an example, a visually impaired user owns a blind safety stick, which is a white cane. On the other hand, an elderly person who is not visually impaired but has difficulty walking uses a brown or black cane, etc. Therefore, the image analysis unit 23 determines whether or not a user in a captured image is visually impaired based on whether or not the cane held by the user is white.

[0030] In addition, visually impaired users have a unique movement of walking while tapping the ground, etc., with a blind safety stick to check for the presence or absence of obstacles, etc. The image analysis unit 23 uses the presence or absence of such a unique movement as a factor for determining whether or not the user is visually impaired.

[0031] Furthermore, some visually impaired users may be accompanied by a guide dog. A normal dog is fitted with a single string-like lead attached to a collar or harness. In contrast, a guide dog is fitted with a uniquely shaped harness called a "U-shaped harness" or a "bar handle type harness." The image analysis unit 23 also uses the shape of such a harness as a factor for determining whether or not the user is visually impaired.

[0032] In addition, the "U-shaped harness" or "bar handle type harness" is often white in color. Therefore, the image analysis unit 23 also uses the color of the harness attached to the dog as a factor for determining whether the user with the dog is visually impaired or not.

[0033] Furthermore, when a user enters a place where entry with dogs is restricted, the dog is likely to be a guide dog and the user is likely to be visually impaired. Therefore, the image analysis unit 23 determines that a user who is bringing a dog to a place where entry with dogs is restricted is a visually impaired user.

[0034] Also, since guide dogs often walk on the left side of the user's direction of travel, the image analysis unit 23 uses whether or not the user is accompanied by a dog walking on the left side of the user's direction of travel as a factor for determining whether or not the user is visually impaired.

[0035] Furthermore, the image analysis unit 23 also uses factors such as whether or not the user is wearing sunglasses, whether or not there is a handicapped person mark, and the like, for determining whether or not the user is visually impaired.

[0036] Next, if the user in the captured image is determined to be a visually impaired user (step S3: Yes), the image analysis unit 23 analyzes the captured image in step S4 to detect the user's features, current location, moving direction, etc. On the other hand, if the user is not determined to be visually impaired (step S3: No), the process returns to step S1.

[0037] As the user's characteristics, the image analysis unit 23 detects the user's age and gender using a predetermined algorithm. In addition, the image analysis unit 23 detects the user's characteristics such as clothing, clothing color, belongings such as a handbag or backpack, and the colors of the belongings by analyzing the captured image. These characteristics are called personal characteristics.

[0038] Furthermore, the image analysis unit 23 detects the current position of the user based on the longitude and latitude corresponding to each coordinate of the captured image captured by the camera device 1. Furthermore, the image analysis unit 23 detects the moving direction of the user from the difference in the current positions of the same user appearing in a series of captured images of, for example, several frames.

[0039] Next, in step S5, the output voice control unit 24 outputs voice guidance (pre-recognition voice guidance) to allow the user to recognize that the voice is directed at him / her, and voice guidance of facilities, etc. corresponding to the user's direction of travel.

[0040] Specifically, the HDD 15 stores divided voice data, which is voice data for each of various words, such as "5," "meters," "ahead," "to," "ticket gate," "exit," "is," "there," "black," "white," "color," "of," "cardigan," "for," "coat," "baseball," "hat," "long," "short," "hair," etc. The output voice control unit 24 reads out and combines the voice data corresponding to the analyzed user's characteristics, and synthesizes the voice to create voice guidance corresponding to the user's characteristics.

[0041] For example, if the analyzed user's characteristic is a woman wearing a black coat, the output voice control unit 24 reads out voice data for each of various words such as "black," "color," "of," "coat," "wear," "woman," "of," and "person" from the HDD 15. The output voice control unit 24 also combines the read out voice data and synthesizes the voice to form a voice guidance (pre-recognition voice guidance) for the analyzed user to recognize that the voice guidance is for the analyzed user, such as "a woman wearing a black coat." The output voice control unit 24 then controls the output of the pre-recognition voice guidance thus formed through the speaker device 2 of the terminal device 50 (or terminal device 60) in which the camera device 1 that captured the captured image of the analyzed user is provided. This allows the analyzed user to recognize in advance that the voice guidance to be output is for him / her.

[0042] Next, the output voice control unit 24 reads out map data corresponding to the analyzed user characteristics (the above-mentioned personal characteristics (age, gender, belongings, etc.)), current location, and direction of movement from the HDD 15. As explained using FIG. 4, the map data includes location information, facility information of facilities existing within that geographical area, and output condition information indicating the output conditions for voice guidance of the facility information, etc. Therefore, the output voice control unit 24 determines the voice guidance to be output based on the analyzed user characteristics, current location, direction of movement, and map data, reads it out from the HDD 15, and controls the output via the corresponding speaker device 2.

[0043] FIG. 7 is a diagram for explaining an example of the operation of the voice guidance system of the embodiment. FIG. 7 shows the positional relationship of the passageway, the tenants, the elevator device (EV) and the restroom inside the store within a predetermined geographical range from the entrance of the store. In addition, one or more camera devices 1 (not shown) are installed inside the store so as to be able to capture the entire passageway shown in FIG. 7, and the position and movement direction of the visually impaired user detected as a result of analysis of the captured image captured by the camera device 1 are shown by arrows in FIG. 7. The display positions and directions of the arrows "A" and "B" in FIG. 7 respectively indicate the position and movement direction of the detected visually impaired user. In addition, blocks for guiding visually impaired people (so-called Braille blocks) are lined up on the floor from in front of the elevator device (EV) to in front of the women's restroom.

[0044] 7, at position "A", an elevator device (EV) is located on the left side of the direction of travel, and a first tenant is located on the right side of the direction of travel. The output voice control unit 24 detects the location information, facility information, and output condition information of such elevator device and first tenant, etc., by referring to map data. Then, when the characteristics of a visually impaired user at position "A" match the output condition information, the voice guidance to be output is read from the HDD 15 and output.

[0045] That is, the visually impaired user at position "A" is analyzed as a "woman about 30 years old," the output condition for the elevator voice guidance corresponding to the position and direction of "A" is "male and female," the first tenant is the "cosmetics counter," and the output condition for the voice guidance is "female." In this case, when the visually impaired user walks to position "A," the output voice control unit 24 provides voice guidance such as "There is an elevator on your left. There is a cosmetics counter on your right."

[0046] As a result, the above-mentioned pre-recognition voice guidance can make the visually impaired female user aware that the voice guidance is directed at her, and can inform her that there is an elevator device to the left of her current location and a cosmetics counter to the right.

[0047] Next, in the example of FIG. 7, it is assumed that a visually impaired user moves from position "A" to position "B". It is also assumed that a "women's restroom" is located on the left side of the position about 5 m from position "B", and a "men's clothing department" is located on the right side of the position about 2 m from position "B". In this case, the output voice control unit 24 provides voice guidance such as "There is a restroom on your left about 5 m away" at position "B". It should be noted that while the visually impaired user is analyzed as a "female", the second tenant is a "men's clothing department". For this reason, the output voice control unit 24 does not provide voice guidance for the second tenant, which is the "men's clothing department", since the voice guidance for the second tenant does not match the output condition for a visually impaired "female" user (does not output voice guidance).

[0048] As a result, the above-mentioned pre-recognition voice guidance allows the visually impaired female user to recognize that the voice guidance is for her, and only voice guidance that is useful to the visually impaired female user at location "B," where a restroom is located 5 meters ahead, can be provided. This makes it possible to prevent the output of unnecessary voice guidance that may become noise.

[0049] Next, in step S6 of the flowchart in Fig. 6, the image analysis unit 23 detects whether or not the visually impaired user is making a "movement calling for help," such as, for example, raising the white cane about 50 cm above the head or waving the white cane from side to side in front of the user's face. If such a "movement calling for help" is not detected (step S6: No), the process returns to step S1. Then, the above-mentioned image analysis (steps S1, S2) → analysis of the visually impaired user's characteristics, current position, and moving direction (steps S3, S4) → audio guidance for advance recognition and audio guidance for the facility (step S5) are repeated.

[0050] On the other hand, if a "movement calling for help" is detected (step S6: Yes), the communication control unit 25 sends an emergency notification to the administrator terminal device 4 shown in FIG. 1 or the analysis device (administrator terminal device) 3 shown in FIG. 2 indicating that the visually impaired user is calling for help (step S7).

[0051] At the same time, in step S8, the output voice control unit 24 issues voice guidance such as "An emergency notification has been sent to the administrator. Help will be on its way soon, so please wait for a while" through the speaker device 2 corresponding to the current location of the user requesting help. That is, the output voice control unit 24 issues voice guidance to the effect that the administrator has been contacted in response to the request for help, and voice guidance requesting the user to wait for a while, by reading out the guidance voice data stored in the HDD 15. This can provide a sense of security to the visually impaired user who has requested help.

[0052] When this emergency notification is received, an assistant such as a manager or a security guard goes directly to the location of the user requesting help to provide assistance. The emergency notification continues until such assistance is provided (step S9: No). Then, when the operator operating the manager terminal device 4 or the analysis device (manager terminal device) 3 confirms that the assistant has provided assistance (step S9: Yes), the operator cancels the emergency notification (step S10). The voice guidance system of the embodiment repeatedly executes each process of steps S1 to S10, for example, during the business hours of the store.

[0053] When a visually impaired user is detected, the communication control unit 25 may notify the manager terminal device 4 or the analysis device (manager terminal device) 3 of the presence of the visually impaired user. This allows preparation for guidance by an assistant to be made, and also allows for immediate response when help is requested.

[0054] (Effects of the embodiment) As is clear from the above description, when providing voice guidance, the voice guidance system of the embodiment first provides pre-recognition voice guidance to make the user aware that the voice guidance to be provided is for a specific user. After providing this pre-recognition voice guidance, voice guidance is provided on the features of the user, the current location, and the direction of travel of the user, and on facilities and the like.

[0055] This allows visually impaired users to recognize that the voice guidance being output is intended for them, and then provides voice guidance for facilities, etc. Therefore, even if multiple visually impaired users are located in the same place and one user and the other user are walking in different directions, each user can be identified before being provided with voice guidance, allowing the voice guidance to function effectively for each user.

[0056] In addition, since voice guidance corresponding to the characteristics, current location, and direction of movement of each user is created and output, it is possible to prevent the voice guidance from being constantly output, which would cause inconvenience to able-bodied people, staff at nearby stores, nearby residents, etc.

[0057] Finally, the above-described embodiment is presented as an example, and is not intended to limit the scope of the present invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. In addition, the embodiment and modifications of the embodiment are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0058] 1 Camera equipment 2. Speaker equipment 3 Analysis device 4. Administrator terminal device 5. Network 11 CPU 12 ROM 13 RAM 14 Communications Department 15 HDD 16 Input / Output Interface (Input / Output I / F) 17 Communication Interface (Communication I / F) 18 Display 19 Control section

Claims

1. A camera device that captures images of users passing by; a speaker device for outputting voice guidance; an analysis device that, when determining that a user appearing in an image captured by the camera device is a visually impaired user, detects characteristics of the user and outputs a pre-recognition voice guidance indicating the detected characteristics via the speaker device; A voice guidance system comprising:

2. The analysis device includes: Further, a current location of the user is detected, the pre-recognition voice guidance is outputted through the speaker device, and then a voice guidance corresponding to the current location of the user is outputted through the speaker device.

2. The voice guidance system according to claim 1,

3. The analysis device includes: Further, a current position and a moving direction of the user are detected, and the pre-recognition voice guidance is outputted through the speaker device, and then a voice guidance corresponding to at least the current position and the moving direction is outputted through the speaker device.

2. The voice guidance system according to claim 1,

4. The analysis device includes: and outputting a voice guidance according to the detected characteristics through the speaker device.

4. The voice guidance system according to claim 2 or 3,

5. The camera device, the speaker device and the analysis device are integrated into a terminal device, which is provided at a location where the captured image is captured.

5. The voice guidance system according to claim 1, wherein the voice guidance system further comprises:

6. the camera device and the speaker device are an integrated terminal device, The analysis device is connected to the terminal device via a network or a communication cable.

5. The voice guidance system according to claim 1, wherein the voice guidance system further comprises:

7. A voice guidance method for a voice guidance system including a camera device that captures an image of a user coming and going, a speaker device that outputs voice guidance, and an analysis device that outputs voice guidance corresponding to an analysis result of the captured image via the speaker device, the method comprising: The analysis device includes: determining whether or not a user appearing in an image captured by the camera device is a visually impaired user; detecting a characteristic of the user when the user is determined to be a visually impaired user; and outputting a pre-recognition voice guidance indicating the detected characteristics via the speaker device. The voice guidance method according to the present invention is characterized in that

8. A voice guidance program for a voice guidance system including a camera device for capturing an image of a user passing by, a speaker device for outputting voice guidance, and an analysis device for outputting voice guidance corresponding to an analysis result of the captured image via the speaker device, Computer, When it is determined that a user appearing in an image captured by the camera device is a visually impaired user, the device detects the user's characteristics and outputs a pre-recognition voice guidance indicating the detected characteristics via the speaker device. A voice guidance program characterized by:

Images