Automatic response system, automatic response method and program

Abstract

The automatic response system (10) according to the present disclosure includes a question acquisition unit (12) that acquires a question about facility usage from a facility user who is a user of the facility, a situation acquisition unit (13) that acquires additional information indicating the situation of the facility user who asked the question, an answer generation unit (14) that generates an answer to the question using the question, the additional information, and facility data that is data about the facility, and an output unit (15) that outputs the answer.

Description

[Technical Field]

[0001] The present disclosure relates to an automatic response system, an automatic response method, and a program for generating answers to questions. [Background technology]

[0002] In recent years, it has become a problem that employees' time is taken up by answering questions from users and guiding users in train stations, commercial facilities, etc. In addition, when a user wants to ask a question, the employee may be busy assisting another user, which can cause the user to be unable to ask a question when they want to. For this reason, automation of user guidance is being considered.

[0003] Patent Document 1 discloses a route guidance system used in shopping malls, event venues, etc. In the route guidance system described in Patent Document 1, when a user tells a robot their destination, a display device shows the route, and the robot explains the route to the destination to the user using voice and body movements. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-265329 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology described in Patent Document 1 can guide users to a specified destination on behalf of employees. However, users have a wide variety of needs and situations. Therefore, the technology described in Patent Document 1 may not be able to provide information that is appropriate for the user.

[0006] The present disclosure has been made in consideration of the above, and aims to provide an automatic response system that can provide more appropriate responses to facility users. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the automatic response system according to the present disclosure includes a question acquisition unit that acquires a question about a destination inside or outside the facility from a facility user who is a user of the facility, a situation acquisition unit that acquires, using an imaging means, additional information indicating the situation of the facility user who asked the question, and facility data that is data related to the question, the additional information, and the facility. A model generated by machine learning that generates answers from questions about destinations inside or outside the facility. The situation was reflected using , acquired by the question acquisition department The system includes an answer generation unit that generates an answer to a question, and an output unit that outputs the answer. The destination is specified by the questioner, and the model is instructed to answer the question taking into consideration the situation of the questioner indicated by the supplementary information and to refer to facility data. The answer generation unit generates an answer indicating a route to the location according to the supplementary information in response to a question about the location of the destination. . [Effects of the Invention]

[0008] The automatic response system according to the present disclosure has the effect of being able to provide more appropriate responses to facility users. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration example of an automatic response system according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing a configuration example of a response generation unit according to the first embodiment; [Figure 3] FIG. 1 is a diagram for explaining an automatic response according to the first embodiment. [Figure 4] FIG. 1 is a diagram for explaining an automatic response according to the first embodiment. [Figure 5] 10 is a flowchart showing an example of an automatic response processing procedure in the automatic response system according to the first embodiment. [Figure 6] 5. A flowchart showing an example of a procedure for generating an answer in the answer generating unit shown in step S5 of FIG. [Figure 7] FIG. 1 is a diagram showing an example of dynamic facility data according to the first embodiment. [Figure 8]FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 9] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 10] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 11] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 12] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 13] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 14] FIG. 10 is a diagram showing another specific example of a response that takes into account the situation in the automatic response system of the first embodiment. [Figure 15] FIG. 1 is a diagram showing an example of the configuration of a computer system that realizes each of the information processing units according to the first embodiment. [Figure 16] FIG. 10 is a diagram illustrating a configuration example of an automatic response system according to a second embodiment. [Figure 17] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the second embodiment. [Figure 18] FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the second embodiment. [Figure 19] FIG. 10 is a diagram illustrating a configuration example of an automatic response system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An automatic response system, an automatic response method, and a program according to an embodiment will be described in detail below with reference to the accompanying drawings.

[0011] Embodiment 1 FIG. 1 is a diagram illustrating an example of the configuration of an automatic answering system according to an embodiment. The automatic answering system 10 of this embodiment accepts questions from facility users about facility use and outputs information about facility guidance as a response to the questions. Examples of facilities include train stations, hospitals, commercial facilities, hotels, buildings, event venues, conference centers, theme parks, railway facilities including multiple stations (including inside railway cars), schools, offices, factories, etc., but are not limited to these as long as they are available to multiple users. The facility may be a facility that is available to an unspecified number of people, such as a train station, a facility that is available to specific people, such as a school or office, or a facility that is available to both specific and unspecified people.

[0012] 1, the automatic response system 10 includes a data management unit 11, a question acquisition unit 12, a situation acquisition unit 13, an answer generation unit 14, and an output unit 15. The data management unit 11 and the answer generation unit 14 constitute an information processing unit 20, which is an information processing device. The automatic response system 10 is installed, for example, within a facility, but at least some of the units constituting the automatic response system 10 may be installed outside the facility.

[0013] The data management unit 11 manages facility data, which is data related to facilities. The facility data is data used by the answer generation unit 14 when generating answers, and may include not only data related to the facility itself but also data related to the surrounding area of the facility and data related to the facility outside the facility. The facility data includes, but is not limited to, at least one of map information (map information) within the facility, facility status information related to the status of the facility, map information around the facility, facility information transmitted by at least one of the facility manager and facility employees, and weather information for the area including the facility. The map information includes information that enables determining routes to potential locations that the user will visit, such as stores and conference rooms within the facility. For example, the map information may include information represented by a graph in which the links represent passageways within the facility and the candidates represent nodes. The map information may also include attributes of the passageways, such as whether the passageways include stairs, slopes, elevators, or escalators. Alternatively, the map information within the facility may include image data showing predetermined routes for each potential location that the user will visit. In this case, the map information may include image data for each passageway attribute, such as whether the passageways include slopes, elevators, or escalators.

[0014] Facility status information includes, but is not limited to, congestion information indicating the degree of congestion at the facility, information such as road closures within the facility, information indicating whether equipment within the facility is broken, and road conditions indicating the condition of surrounding roads. Facility-transmitted information is information that facility managers or employees want to convey to users, such as, for example, "It is raining, so the facility is slippery," or "The venue for X event starting at 10:00 today has been changed from X to Y." Weather information includes, but is not limited to, at least one of weather (sunny, rainy, strong winds, etc.), temperature, and humidity.

[0015] Furthermore, if the facility is located within a station, the facility data may include timetables for trains and other railroad vehicles, and if there is a store within the facility, the facility data may include at least one of information about the products handled by the store, information about the food and drink served at the store, information indicating the store's main target demographic, information about the store's recommended products, store information transmitted by store employees, etc. Store information is information that store employees want to convey to users, such as "Today, Shop A is having a rainy day sale," "Shop B is currently running a limited-time sale," or "Restaurant K has a wide selection of cold drinks available that are perfect for hot days."

[0016] The facility data may include static data that does not change in a short period of time, i.e., does not change for a certain period of time, such as map information within the facility, information indicating the store's main target demographic, etc. The facility data may also include dynamic data that may change in a short period of time, i.e., may change over a certain period of time, such as facility status information, facility information, weather information, etc. The facility data may also include both static data and dynamic data.

[0017] The data management unit 11 includes a data acquisition unit 111, a pre-processing unit 112, and a data storage unit 113. The data acquisition unit 111 acquires facility data and outputs the acquired facility data to the pre-processing unit 112.

[0018] Specifically, the data acquisition unit 111 may acquire static facility data by, for example, accepting data input from a facility manager, operator, store employee, etc., or by receiving the data from another device (not shown). For example, the data acquisition unit 111 may receive map information about the area around the facility from a device that manages the map information. Alternatively, a terminal device that can be operated by a facility manager, operator, store employee, etc. may accept input of facility data from the facility manager, operator, store employee, etc., and the data acquisition unit 111 may receive the facility data from the terminal device.

[0019] Furthermore, the data acquisition unit 111 may acquire dynamic data from facility data by receiving sensor information acquired by a sensor (not shown in FIG. 1 ), such as a surveillance camera or a microphone, and calculating the dynamic data based on the received sensor information. For example, the dynamic data may include data acquired by a sensor installed inside or around the facility. For example, the data acquisition unit 111 may calculate congestion information based on video, which is sensor information acquired by a surveillance camera. Alternatively, the data acquisition unit 111 may output the sensor information to the preprocessing unit 112, and the preprocessing unit 112 may calculate the dynamic data from the sensor information. Alternatively, another device (not shown) may calculate the dynamic data based on the sensor information, and the data acquisition unit 111 may receive the dynamic data from the other device.

[0020] Furthermore, for example, for facility transmission information and store transmission information, which are dynamic data, the data acquisition unit 111 may acquire the facility transmission information and store transmission information as text data by accepting input from a sender, or may accept input of the facility transmission information and store transmission information by voice. Furthermore, information broadcast within the facility may be acquired as facility transmission information and store transmission information from a microphone used for in-facility broadcasting at the facility. Alternatively, microphones may be installed around the location where in-facility broadcasting is to be made, and the information broadcast within the facility may be acquired from the microphone as facility transmission information and store transmission information.

[0021] The preprocessing unit 112 performs preprocessing on the facility data received from the data acquisition unit 111. For example, the preprocessing unit 112 stores the preprocessed facility data in the data storage unit 113 as a database. The preprocessing is, for example, a process of converting the data into a data format accessible by the answer generation unit 14. For example, when the answer generation unit 14 reads vectorized data, the preprocessing unit 112 vectorizes the facility data and stores the vectorized facility data in the data storage unit 113. The vectorization includes, for example, a process of converting words in natural language processing into vectors that are distributed representations according to the meanings of the words.

[0022] The content of the preprocessing performed by the preprocessing unit 112 is not limited to the above example, and may be determined depending on the processing method used in the answer generation unit 14. Depending on the processing method used in the answer generation unit 14, the preprocessing unit 112 may not be provided, and the facility data may be stored directly in the data storage unit 113. The data storage unit 113 stores the facility data.

[0023] The question acquisition unit 12 acquires questions about facility use from facility users (hereinafter also referred to as facility users). Specifically, the question acquisition unit 12 accepts questions about the facility from facility users and outputs the accepted questions to the answer generation unit 14. The question acquisition unit 12 may acquire the questions as voice data, or may acquire the questions as text data by accepting input through operation by the facility users. That is, the question acquisition unit 12 may include a microphone, an input means for accepting input through user operation such as a touch panel, or both a microphone and an input means. The question acquisition unit 12 may accept image input. In this case, for example, the image input is accepted by capturing an image with a camera. For example, a facility user may present a photo of a product, a store, or the like and input a question, such as where the product in the photo is located, or where this store is located, by voice or text. The camera may be a camera used as the situation acquisition unit 13 described later. When the question acquisition unit 12 receives a question using an image, it outputs the question together with the image data to the answer generation unit 14.

[0024] The situation acquisition unit 13 acquires additional information indicating the situation of the facility user who asked the question and outputs the acquired additional information to the answer generation unit 14. For example, the situation acquisition unit 13 includes a sensor such as a camera or an infrared sensor, and acquires the detection result of the sensor as additional information (auxiliary information). For example, the additional information includes video data of the questioner. Furthermore, the situation acquisition unit 13 may acquire the additional information indicating the situation of the facility user as text data by accepting input through operation of the facility user, or may acquire the additional information indicating the situation of the facility user as audio data. When the situation acquisition unit 13 acquires the additional information as text data or audio data, for example, a question asking about the situation is presented to the facility user by the output unit 15, and the situation acquisition unit 13 acquires the answer to the question as the additional information. When the additional information is acquired as text data or audio data, the question acquisition unit 12 may function as the situation acquisition unit 13. Furthermore, the microphone used by the situation acquisition unit 13 to acquire audio data may be shared with the question acquisition unit 12.

[0025] As described above, the additional information may be, for example, situation information indicating the situation of the facility user asking the question. However, the additional information is not limited to this. The additional information may be situation information indicating the situation of the facility user asking the question and attribute information indicating the attributes of the facility user asking the question. For example, video data acquired by a camera can be used to understand both the situation and attributes of the facility user. Furthermore, the situation acquisition unit 13 may present the facility user with a question asking about the attribute in addition to the situation via the output unit 15, and the situation acquisition unit 13 may acquire the answer to the question as the additional information. In this way, the additional information may include the situation information and the attribute information. Alternatively, the situation information and the attribute information may be acquired by different means, such as acquiring video data as situation information and acquiring the attribute information as a response to the question asking about the attribute. Note that the method of acquiring the facility user's attributes is not limited to acquiring them from video data acquired by a camera. For example, as described below, the attributes of an individual may be identified using an individual's identification result.

[0026] The answer generation unit 14 generates an answer regarding facility guidance using the question received from the question acquisition unit 12, the incidental information received from the situation acquisition unit 13, and the facility data stored in the data storage unit 113, and outputs the generated answer to the output unit 15. Specifically, the answer generation unit 14 understands the question and generates an answer using the facility data, taking into account the situation indicated by the incidental information. The answer generation unit 14 can generate an answer using data in multiple formats, including, for example, audio, text, and image, as input, and can output the answer in multiple formats. That is, for example, the answer generation unit 14 can generate a multimodal output in response to a multimodal input such as audio data, text data, and video data, but is not limited thereto. Hereinafter, generating a multimodal output in response to a multimodal input is also referred to as multimodal processing. The answer generation unit 14 will be described in detail below.

[0027] The output unit 15 presents the answer received from the answer generation unit 14 to the facility user who asked the question by outputting the answer. The answer may be output as audio, text data, an image, video, or a combination of two or more of these. Therefore, the output unit 15 includes, for example, at least one of a speaker and a display device such as a display or a touch panel. If the answer generation unit 14 is capable of multimodal processing, the output unit 15 outputs the answer according to the format of the answer received from the answer generation unit 14. For example, if the answer received from the answer generation unit 14 is audio data, the output unit 15 outputs the answer as audio. If the answer received from the answer generation unit 14 is at least one of video data and text data, the output unit 15 outputs the answer by display. If the output unit 15 includes a touch panel, the touch panel may be used as an input means for the question acquisition unit 12 or the situation acquisition unit 13.

[0028] Next, the answer generation unit 14 will be described in detail. FIG. 2 is a diagram illustrating an example of the configuration of the answer generation unit 14 according to the present embodiment. For example, as illustrated in FIG. 2, the answer generation unit 14 includes a situation recognition unit 141 and a generation unit 142. The situation recognition unit 141 recognizes a situation based on the additional information received from the situation acquisition unit 13 and outputs the recognized situation to the generation unit 142. The situation recognition unit 141 recognizes the situation of the facility user who asked the question. However, if the facility user is accompanied by someone, such as a family or a couple, hereinafter, not only the facility user who actually asked the question but also the accompanying person of the facility user will be treated as the facility user who asked the question. Hereinafter, the facility user who asked the question will also be referred to as the questioner. The situation recognized by the situation recognition unit 141 includes, but is not limited to, at least one of the following: what the questioner is wearing (such as the questioner's clothing and accessories), whether the questioner is using an assistive device such as a wheelchair or a cane (whether or not the assistive device is being used), the number of questioners, the questioner's emotions, the size of luggage the questioner is carrying, and the direction of movement of the questioner.

[0029] Furthermore, as described above, when the additional information is information that indicates the situation and attributes of the questioner, the situation recognition unit 141 uses the additional information to recognize the situation and attributes of the questioner, and outputs the recognized situation and attributes to the generation unit 142. The attributes of the questioner may include, for example, at least one of gender, age, generation, presence or absence of a disability, and occupation.

[0030] The generation unit 142 generates an answer regarding facility guidance using the situation (or situation and attributes) received from the situation recognition unit 141, the question received from the question acquisition unit 12, and facility data stored in the data storage unit 113, and outputs the generated answer to the output unit 15.

[0031] In this way, the answer generation unit 14 generates an answer that reflects the situation of the questioner. For example, if a questioner using a wheelchair asks how to get to a destination, the answer generation unit 14 can generate an answer that shows a route that does not have stairs and uses ramps or elevators.

[0032] 3 and 4 are diagrams for explaining the automatic response of this embodiment. In the example shown in FIG. 3, the automatic response system 10 includes a camera as the situation acquisition unit 13, and a touch panel and a speaker as the output unit 15. In the example shown in FIG. 3, a keyboard (software keyboard) for accepting questions is displayed below the touch panel, and the portion where this keyboard is displayed also functions as the question acquisition unit 12. In the example shown in FIG. 3, the automatic response system 10 further includes a microphone as the question acquisition unit 12. As shown in the upper part of FIG. 3, a questioner 30 carrying a large piece of luggage utters, "Where is the station platform?" to ask the automatic response system 10 for the location of the station platform.

[0033] The answer generation unit 14 of the automatic response system 10 recognizes the situation that the questioner 30 is carrying large luggage from video data received from a camera, which is an example of a sensor used as the situation acquisition unit 13, and generates an answer using the recognized situation, the voice data of the question received from the question acquisition unit 12, and the facility data. In the example shown in FIG. 3, the situation recognized is not limited to "carrying large luggage," but may also be "both hands are full" or "difficulty walking." In the example shown in FIG. 3, since the questioner 30 is carrying large luggage, the answer generation unit 14 generates a route to the station platform, i.e., a route using an elevator, as shown in the lower part of FIG. 3, based on map information within the facility in the facility data, rather than a route using stairs or escalators. There are no particular restrictions on the route generation method, and methods such as searching for the shortest route within a range that satisfies the conditions can be used. In the example shown in FIG. 3, the answer generation unit 14 generates a diagram (image) showing the route and text data explaining the diagram as an answer, and outputs the generated answer to the output unit 15, which then outputs the answer. 3, text and a diagram showing a route are displayed as an answer on the touch panel of output unit 15, but the output method is not limited to this. For example, a speaker of output unit 15 may also output the answer as voice, or the speaker may output the answer as voice instead of text.

[0034] In the example shown in Fig. 4, the automatic response system 10 is composed of a camera, which is an example of a sensor used as the situation acquisition unit 13, and a main body 40. The main body 40 includes an information processing unit 20, a question acquisition unit 12, and an output unit 15, which are not shown in Fig. 4. In this way, the sensor used as the situation acquisition unit 13 may be provided separately from the main body 40. Furthermore, the automatic response system 10 may include multiple sensors as the situation acquisition unit 13; for example, sensors may be provided both in the main body 40 and outside the main body 40.

[0035] In the example shown in FIG. 4, a questioner 30 uses a wheelchair and asks the automatic answering system 10, "Where is the exit?" In the example shown in FIG. 4, a microphone, which is the question acquisition unit 12, receives a question. In the example shown in FIG. 4, since the questioner 30 uses a wheelchair, the answer generation unit 14 generates an answer that includes a slope without steps as a route to the exit, and the output unit 15 displays the answer. Furthermore, when a question about the location of the exit is asked, the answer generation unit 14 assumes that the questioner will be leaving the facility, and if it is raining based on the facility data, the answer generation unit 14 may add a sentence that corresponds to the outside weather, such as "It's raining outside, so please be careful on your way home."

[0036] 4, the question is about an exit, but by including map information about the area around the facility in the facility data, for example, if a questioner asks, "Please tell me the route to the department store" when the facility is located inside a train station, the automatic response system 10 may be able to provide guidance on the route from the station to the department store. In this way, the answer generation unit 14 may respond with information about the area around the facility as information related to facility guidance. Also, if the facility is located inside a train station, the facility data may include information indicating the train lines that stop at the station, the stations on the lines, the nearest stores and tourist attractions to the station, etc., so that a question such as, "Which train line should I take to get to VV?" can be answered by indicating which train line to take and at which station to get off.

[0037] In the example shown in FIG. 3, the automatic response system 10 is realized as a single device, and in the example shown in FIG. 4, it is realized by a camera, which is an example of a sensor used as the situation acquisition unit 13, and a main unit 40. However, the hardware configuration of the automatic response system 10 is not limited to these examples. For example, the information processing device, which is the information processing unit 20 shown in FIG. 1, and an input / output device including the question acquisition unit 12, the situation acquisition unit 13, and the output unit 15 may be provided in separate locations. In this case, the input / output device and the information processing device may each include a transceiver unit for communicating, and information may be exchanged via the transceiver unit. Furthermore, FIGS. 3 and 4 are merely examples, and the situations considered by the automatic response system 10 and the answers presented by the automatic response system 10 are not limited to these examples.

[0038] Returning to the description of FIG. 2 , the answer generation unit 14 may be, for example, a generative AI (artificial intelligence). Examples of the generative AI include multimodal AIs such as Gemini (registered trademark) and GPT (Generative Pre-trained Transformer)-4V, image generation AIs such as Stable Diffusion, large-scale language models such as GPT-3, and voice generation AIs such as Murf.AI. When the answer generation unit 14 is a multimodal AI, the answer generation unit 14 includes the functions of a situation recognition unit 141 and a generation unit 142. When the answer generation unit 14 is a multimodal AI, before accepting a question from a facility user, such as when the automatic response system 10 is started, an instruction such as "Please consider the situation of the facility user who asked the question and generate video data, text data, and audio data as necessary to provide an answer" is input to the answer generation unit 14. The answer generation unit 14 is also instructed to use facility data as appropriate. As a result, the answer generating unit 14 takes into consideration the situation of the questioner 30, references the facility data, and generates an answer regarding guidance to the facility using at least one of video data, text data, and audio data.

[0039] Furthermore, specific information to be considered as a situation may be input to the answer generating unit 14, or data indicating a definition of a situation may be included in the facility data. For example, an instruction such as "The situation of the facility user who asked the question includes whether the questioner uses an assistive device such as a wheelchair or a cane, the number of people asking the question, the emotions of the questioner, the clothing of the questioner, what the questioner is wearing, the size of luggage the questioner is carrying, and the direction of movement of the questioner" may be input to the answer generating unit 14.

[0040] When a multimodal AI is used in the answer generation unit 14 and instructions such as those described above are given, the format of the answer is determined by the answer generation unit 14. On the other hand, if it is desired to define specific rules, such as an instruction to provide an answer in the same format as the question, rather than leaving the determination of the answer format to the answer generation unit 14, instructions indicating the rules can be input to the answer generation unit 14, or data indicating the rules can be included in the facility data. For example, typical examples, norms, guidelines, etc. of answers corresponding to questions can be stored in the data storage unit 113 as facility data or separately from the facility data, and the answer generation unit 14 can refer to the rules when creating an answer. Alternatively, pre-learning can be performed to learn these rules. Although facility data is used to generate an answer, a portion of the facility data may be used as part of the answer. For example, if the facility data includes images of products sold by a store, the answer generation unit 14 may include the images in the answer.

[0041] For example, in order for the answer generation unit 14 to generate the answers exemplified in FIGS. 3 and 4 , the answer generation unit 14 may input the situation of the questioner 30 to be considered and the question to the multimodal AI in advance, check whether the desired answer is obtained, and if the desired answer is not obtained, instruct the multimodal AI to provide a correct answer corresponding to the situation and the question. The correct answer corresponding to the situation and the question may also be included in the facility data. For example, as shown in FIG. 3 , when the questioner 30 carrying large luggage asks about the route to the destination, if the answer generation unit 14 generates an answer that includes stairs, the answer generation unit 14 may be given an instruction such as, "If the questioner has large luggage, use the elevator instead of the stairs." The instruction may also be included in the facility data. Alternatively, the instruction may be broken down into information such as, "It is difficult for the questioner to walk if he or she is carrying large luggage or has both hands full," and information such as, "If it is difficult to walk, use the elevator instead of the stairs." In addition, when attributes are taken into consideration in addition to situations, instructions for generating an appropriate answer for each situation and attribute may be given to the answer generation unit 14, or the instructions may be included in the facility data.

[0042] Furthermore, when a large-scale language model is used in the answer generation unit 14, the question acquisition unit 12 may receive the question as text data, and the output unit 15 may output the answer as text data. Alternatively, the question acquisition unit 12 may receive the question as voice, and the question acquisition unit 12 or the answer generation unit 14 may convert the voice data of the question into text data. For example, a large-scale language model may function as the generation unit 142, and a situation recognition model that recognizes a situation from video data may be used as the situation recognition unit 141. The situation recognition model may be, for example, a trained model that has been trained to infer a situation from video data by machine learning. Examples of machine learning used to generate the trained model include supervised learning such as a neural network, but are not limited to unsupervised learning, reinforcement learning, and the like. The situation recognition model may also be a model that recognizes attributes.

[0043] Furthermore, the situation recognition model may be a combination of multiple models, such as a combination of a general emotion recognition model that recognizes emotions from video data and an image recognition model that recognizes the presence or absence of assistive devices, the size of luggage, clothing, etc. from video data. The generation unit 142 generates an answer as text data using, for example, text data indicating the situation (or the situation and attributes) received from the situation recognition unit 141 and the question. Alternatively, a voice generation AI may be further used as the generation unit 142, and the answer generated by the large-scale language model may be converted into voice data by the voice generation AI, or an image generation AI may be further used as the generation unit 142, and the answer generated by the large-scale language model may be converted into image data by the image generation AI.

[0044] In this way, a multimodal output may be realized by combining and using a plurality of models. In this case, for example, output may be performed in all available output formats, or rules for selecting the output format may be determined in advance, in accordance with the rules. For example, a rule may be determined by associating the situation (or situation and attributes) of the questioner 30 with the output format, or a rule may be determined by outputting in the same format as the format of the question asked by the questioner 30. Furthermore, a rule may be determined by associating the type of question content (e.g., a question about the route to a destination, a question asking about recommended shops, etc.) with the output format. The rules for determining the output format are not limited to these examples.

[0045] When a generation AI, a large-scale language model, or the like is used in the answer generation unit 14, the facility data is vectorized by the preprocessing unit 112 and stored in the data storage unit 113, allowing the answer generation unit 14 to use the facility data as is. Therefore, the facility data can be reflected in the answers of the answer generation unit 14 without requiring additional learning.

[0046] Furthermore, a general automatic conversation program such as a chatbot may be used for the generation unit 142. In this case, a situation recognition model may be used as the situation recognition unit 141, similar to when a large-scale language model is used for the generation unit 142, and text data indicating the situation (or the situation and attributes) recognized by the situation recognition unit 141 may be input to the generation unit 142.

[0047] The automatic conversation program may be a rule-based (scenario-based) program in which response rules are defined in advance, or a machine learning program. When a rule-based automatic conversation program is used, for example, answers corresponding to the content of questions for each situation (or situation and attribute) are defined in advance and set in the automatic conversation program. The machine learning in a machine learning automatic conversation program may be supervised learning, unsupervised learning, or reinforcement learning. For example, when supervised learning is used in the automatic conversation program, a trained model is generated using multiple training datasets including questions for each situation (or situation and attribute) and corresponding answers, which are correct answer data. The generation unit 142 generates an answer by inputting the situation (or situation and attribute) and question corresponding to the questioner 30 into the trained model. Note that even when an automatic conversation program is used in the generation unit 142, the answer generated by the automatic conversation program may be converted into voice data by a voice generation AI or into image data by an image generation AI. In this case, output may be performed in all supported output formats, or rules for selecting the output format may be defined in advance and the output format may be determined according to the rules.

[0048] The questioner 30 may not only be an unspecified person visiting the facility, but also a specific person, such as an employee, a facility manager, or a student if the facility is a school. That is, facility users may include a specific person specified in advance. For example, if the facility is intended for a specific person, only specific people may be considered as questioners 30. In this case, for example, a facial photograph of the specific person may be included in the facility data, the situation acquisition unit 13 may acquire video data from a camera as additional information, and the answer generation unit 14 may identify the questioner 30 based on the facial photograph and generate an answer according to the identification result. This video data is an example of personal authentication information for identifying an individual. For example, for specific people who may use the facility, their identification information (hereinafter also referred to as personal identification information) and facial photographs may be associated and stored in the data storage unit 113 as facility data. Furthermore, for each piece of personal identification information, individual information about the person corresponding to the personal identification information may be stored in the data storage unit 113 as facility data. Individual information is information regarding a specific person's individual use of the facility, such as, but not limited to, one or more of the following: schedule, accessible locations, and scope of information that can be obtained (authority to view information).

[0049] For example, if the facility is a school, information indicating the classes each student is taking is included in the individual information of the facility data, and class location information indicating the time and location (classroom, etc.) of each class is included in the facility data. As a result, when a student (questioner 30) asks a question such as, "Where is the next class?", the answer generation unit 14 can determine the next class the student will take based on the individual information of the student and the time and location of the class using the class location information. The answer generation unit 14 generates an answer that reflects the determined information, such as, "The next class is Class Y, which will be held in Classroom X in Building 3." At this time, the answer may include an image showing the route from the current location to Classroom X in Building 3. When generating an answer, as described above, for example, the situation is further taken into consideration.

[0050] Furthermore, if the facility is a school, the facility data may include individual information for not only students but also school staff. For example, the individual information may include information indicating the staff's class schedule and the time and location of the staff's exam supervision. In this case, as in the student example, for example, in response to a question from a staff member such as, "What time and location will I be supervising next?", a response such as, "The next exam supervision will be in classroom Z in building 2, starting at 11:00" is generated. The content of the response described above is merely illustrative, and the content of the response is not limited to the above example. Note that while an example of identifying individuals using facial information has been described here, to identify individuals, for example, a passcode assigned to each individual may be included in the facility data, and the automated response system 10 may identify individuals by accepting input of the passcode when a question is asked. Alternatively, the automated response system 10 may be equipped with a device that performs biometric authentication other than facial recognition, and individuals may be identified using that device. In other words, the personal authentication information may be a passcode, biometric authentication information, or the like.

[0051] Furthermore, for each attribute of a specific person using the facility, such as an employee or student, at least one of information and rules for generating a response to the questioner 30 for that attribute may be defined as attribute-related information, and the attribute-related information may be included in the facility data. The attribute-related information is information regarding the use of the facility for each attribute, and includes schedules, accessible locations, the scope of information that can be obtained (information viewing authority), and contact information. If the facility is a school, the attributes may include, for example, status (student, professor, assistant professor, teacher, office worker, etc.) and affiliation (faculty, department, selection). If the facility is a business establishment, the attributes may include job title, affiliation (department, faculty, division), qualifications held, etc. In this way, for example, the attributes may include at least one of status, affiliation, and job title. For example, by including personal attributes in the individual information in the facility data, the response generation unit 14 may identify the individual using a method similar to the above-described individual identification, grasp the attributes based on the individual information, and generate a response according to the attributes.

[0052] For example, if the attribute-related information specifies classes to be taken for each department, as in the example described above, when a student asks, "Where is the next class?", the answer generation unit 14 identifies the department to which the student belongs as an attribute of the student (questioner 30). The answer generation unit 14 also identifies classes corresponding to the identified attributes based on the attribute-related information and generates an answer indicating the location of the class based on the class location information. For example, if accessible locations are specified for each attribute, when the questioner 30 asks how to get to a destination, the answer generation unit 14 calculates a route that travels within the accessible locations and generates the calculated route as an answer. For example, if the attribute includes information viewing authority, the answer for the questioner 30 is generated using information within the range that is permitted to be viewed. For example, if the attribute-related information includes a message for each attribute, the answer may be a direct answer to the question with the message added, regardless of the content of the question. For example, the message may include a notice of a change in class.

[0053] Furthermore, if the facility is open to the general public, such as a train station, a commercial facility, or a university, the questioner 30 may be both an unspecified person, such as a customer of the facility or a visitor to the university, and a specific person, such as an employee or a student of the facility. The answer generation unit 14 may change the content of the answer depending on whether the questioner 30 is an unspecified person or a specific person. In this case, the attributes of the questioner 30 may include, for example, whether the questioner 30 is a specific person, and rules for generating answers for each attribute, such as accessible locations, the scope of information that can be obtained (information viewing authority), and contact information, may be defined in the attribute-related information, as in the above example. The attribute of whether the questioner 30 is a specific person is determined in the same way as in the case of identifying a specific person described above, for example, by including facial information or the like of the specific person in the facility data in advance. For example, the attribute-related information may include information distinguishing between facility data used to generate answers for specific people and facility data used to generate answers for unspecified people. Furthermore, for specific people such as employees, attributes such as affiliation and position may be further included as attributes, as in the above example, and answers may be generated according to these attributes.

[0054] For example, if the facility is a commercial facility and the questioner 30 asks, "Where is the warehouse where product P is stored?", the answer generation unit 14 generates, as an answer, information indicating the warehouse where product P is stored based on the facility data if the questioner 30 is an employee, and generates an answer indicating that the question cannot be answered if the questioner 30 is not an employee. At this time, for example, the answer generation unit 14 further generates an answer depending on the situation, as described above.

[0055] Furthermore, if the answer generation unit 14 cannot recognize the content of the question from the questioner 30 as language, if the question from the questioner 30 is insufficient for generating an answer, or if information about the situation of the questioner 30 is insufficient for generating an answer, the answer generation unit 14 may cause the output unit 15 to output information to the questioner 30 asking for the missing information. That is, for example, if the answer generation unit 14 cannot recognize the question or if there is insufficient information for generating an answer, the answer generation unit 14 of the automatic answering system 10 may generate an answer to ask the questioner 30 to ask again. For example, if the content of the question from the questioner 30 cannot be recognized as language, the answer generation unit 14 may output at least one of text data and audio data such as "Please repeat the question again" to the output unit 15, and the output unit 15 may output at least one of text and audio. Furthermore, if the facility is located within a station, the answer generation unit 14 may output at least one of text data and voice data for identifying the platform, such as "Is this the platform for the K line or the L line?" to the output unit 15, and the output unit 15 may output at least one of text and voice. Furthermore, for example, if it is recognized that there are multiple questioners 30 but the number of questioners 30 is unclear because they are overlapping, the answer generation unit 14 may output at least one of text data and voice data for identifying the number of people, such as "How many people are there?", and the output unit 15 may output at least one of text and voice.

[0056] Next, the operation of the automatic answering system 10 of this embodiment will be described. Fig. 5 is a flowchart showing an example of an automatic answering process procedure in the automatic answering system 10 of this embodiment. The automatic answering system 10 acquires facility data (step S1). In detail, the data acquisition unit 111 acquires the facility data and outputs it to the preprocessing unit 112. The facility data may be static data as described above, dynamic data, or both.

[0057] The automatic answering system 10 stores the facility data (step S2). Specifically, the preprocessing unit 112 performs preprocessing on the facility data, and stores the preprocessed facility data in the data storage unit 113. The preprocessing is, for example, vectorization as described above, but is not limited to this and may be conversion into a data format that can be accessed by the answer generation unit 14 or a data format that can be accessed at high speed by the answer generation unit 14. Also, as described above, preprocessing does not necessarily have to be performed.

[0058] The automatic answering system 10 acquires the question content (step S3). In detail, the question acquiring unit 12 acquires the question from the questioner 30 as voice or text data, thereby acquiring the question content, and outputs the acquired question to the answer generating unit 14.

[0059] The automatic answering system 10 acquires the situation (step S4). In detail, the situation acquisition unit 13 acquires additional information indicating the situation in which the questioner 30 is placed, and outputs the additional information to the answer generation unit 14. For example, the situation acquisition unit 13 may acquire sensor information acquired by a sensor such as a camera as the additional information, or may acquire the additional information by inputting voice or text data from the questioner 30.

[0060] The automatic answering system 10 generates an answer (step S5). In detail, the answer generating unit 14 grasps the situation (the situation of the questioner 30) from the additional information, generates an answer to the question based on the situation, and outputs the generated answer to the output unit 15. As described above, the automatic answering system 10 may further grasp attributes (the attributes of the questioner 30) from the additional information, and generate an answer to the question based on the situation and attributes.

[0061] The automatic response system 10 outputs the answer (step S6) and ends the automatic response process. Specifically, in step S6, the output unit 15 outputs the answer. The output unit 15 outputs the answer by performing at least one of audio output and display, depending on the format of the answer generated by the answer generation unit 14 (whether the answer is voice data or text data or image data).

[0062] FIG. 6 is a flowchart showing an example of the procedure for the answer generation process in the answer generation unit 14 shown in step S5 of FIG. 5. The answer generation unit 14 recognizes the situation and the question (step S11). Specifically, the situation recognition unit 141 recognizes the situation based on the additional information and outputs it to the generation unit 142. The generation unit 142 recognizes the situation acquired from the situation recognition unit 141 and the question received from the question acquisition unit 12 by converting them into a format required for answer generation processing. Furthermore, for example, when a multimodal AI or a large-scale language model is used in the answer generation unit 14, the generation unit 142 understands the meaning of the situation and the question. Specifically, if the question is "Where is the platform?", the answer generation unit 14 understands that "I am being asked about the route to the platform," and if the recognized situation is "I have large luggage," the answer generation unit 14 understands that "I would prefer a route that avoids stairs as much as possible."

[0063] The answer generation unit 14 refers to the facility data (step S12). In detail, the generation unit 142 reads out facility data necessary for answering the question from the data storage unit 113. For example, in the example where the question is "Where is the platform?" and the situation is "I have large luggage," map information within the station is read out as facility data.

[0064] The answer generation unit 14 generates an answer (step S13) and ends the answer generation process. More specifically, in step S13, the generation unit 142 uses the referenced facility data to generate an answer to the question that takes the situation into consideration, and outputs the generated answer to the output unit 15. For example, in the above example where the question is "Where is the platform?" and the situation is "I have large luggage," map information within the station is used to search for a route to the platform that uses an elevator and does not require stairs, and a diagram showing the route obtained by the search and an explanation for the diagram are generated.

[0065] Through the above processing, the automatic answering system 10 of this embodiment can generate an answer that takes into account the situation of the questioner 30. This allows the automatic answering system 10 to provide a more appropriate answer to the facility user than when simply answering a specific question from the facility user. Furthermore, the automatic answering system 10 may generate an answer based on attributes in addition to the situation, and in this case, it is also possible to provide a facility user with a more appropriate answer according to the attributes. Furthermore, by including dynamic data (dynamic facility data) as facility data, it is possible to provide the facility user with appropriate answers and information based on the current status of the facility.

[0066] FIG. 7 is a diagram illustrating an example of dynamic facility data according to the present embodiment. In the example illustrated in FIG. 7, the facility is a railway facility (including the inside of a railway vehicle) including a station premises or multiple stations. In the example illustrated in FIG. 7, a questioner 30 asks, "Which car is empty?" To generate an answer to this question, it is necessary to know the degree of congestion in each car. In the example illustrated in FIG. 7, the data acquisition unit 111 acquires, as dynamic facility data, video data acquired by a camera 50 installed in each car of the railway vehicle. In addition, a timetable indicating when each train formation of railway vehicles will arrive at the station is also stored as facility data in the data storage unit 113. Note that the dynamic facility data is updated to the latest data when new data is acquired, for example. However, this is not limited to this, and past dynamic facility data may also be stored in the data storage unit 113.

[0067] The preprocessing unit 112 may store video data of the interior of the vehicles in the data storage unit 113, or may determine the degree of congestion of each vehicle based on the video data and store the determined result as facility data in the data storage unit 113. The degree of congestion may be indicated, for example, in two stages of whether or not there is a mixture, or in three stages of crowded, normal, and empty, or may be indicated by a congestion rate, or may be indicated in some other way. When the video data is stored in the data storage unit 113, the answer generation unit 14 recognizes the degree of congestion based on the video data and uses it to generate an answer.

[0068] For example, in the example shown in FIG. 7, the answer generation unit 14 identifies the next arriving train car based on a timetable and can identify which cars are empty by using the degree of congestion calculated from video data captured inside each car of the train car. This allows an appropriate answer that takes into account the actual situation, such as "Car 2 is empty," to be generated in response to the question, "Which car is empty?" In addition, the example shown in FIG. 7 illustrates an example in which the automatic response system 10 is installed on a station platform, and the answer generation unit 14 generates an image showing the location of car 2 and the current location as an answer, and the output unit 15 displays the generated image together with the text answer, "Car 2 is empty." Note that, as described above, the situation (or the situation and attributes) may also be taken into consideration when generating this answer. For example, in the example shown in Figure 7, if the questioner 30 does not have large luggage, an answer may be generated to guide the questioner 30 to the emptiest vehicle available, and if the questioner 30 has large luggage, an answer may be generated to guide the questioner 30 to the nearest vehicle available (even if it is not the emptiest vehicle).

[0069] Furthermore, the dynamic facility data is not limited to the example shown in Figure 7, but may be information indicating the degree of congestion in toilets within the facility, information indicating the degree of congestion in a store, or facility-originated information, store-originated information, etc., as described above.

[0070] Next, a specific example of a response that takes into account the situation in the automatic response system 10 of this embodiment will be described. As described above, the situation of the questioner 30 may include, for example, the number of questioners 30. For example, when the questioner 30 asks, "Please tell me a restaurant you recommend," if there is only one questioner 30, the automatic response system 10 selects a quiet restaurant based on the facility data, generates a response indicating that the selected restaurant is recommended, and outputs the response. On the other hand, if the questioner 30 is with a family, the automatic response system 10 selects a family-oriented restaurant, such as a family restaurant, generates a response indicating that the selected restaurant is recommended, and outputs the response.

[0071] 8 to 14 are diagrams showing specific examples of responses that take the situation into consideration in the automatic response system 10 of this embodiment. In the example shown in FIG. 8, a middle-aged or elderly woman wearing elegant clothing and accessories asks, "Do you know of any clothing stores?" The automatic response system 10 recognizes that the situation of the questioner 30 is that the questioner is wearing accessories and elegant clothing, and based on this recognition result, the question, and information in the facility data indicating the products sold by each store and the target audience of each store, determines that Shop A and Shop B are recommended clothing stores for middle-aged or elderly people. The automatic response system 10 then generates information to guide the questioner to Shop A and Shop B as an answer to the question and displays it on the output unit 15. In this way, an answer that takes the situation of the questioner 30 into consideration is generated.

[0072] In the example shown in FIG. 8, the automatic response system 10 further generates an answer including basis information indicating the basis of the answer based on the situation used to generate the answer. In the example shown in FIG. 8, the automatic response system 10 generates basis information of "mother generation" as the basis for deciding to guide the customer to Shop A and Shop B, based on the situation that the customer is a middle-aged or elderly woman wearing elegant clothing and accessories. As a result, in the example shown in FIG. 8, not only the suggested store names "·A Shop" and "·B Shop" but also the text "How about a fashionable clothing store popular with the mother generation?" are displayed. Furthermore, touching the "·A Shop" and "·B Shop" portions on the display screen shown in FIG. 8 may display detailed information about each store. Note that the specific content of the sentence (text) in the answer is not limited to the example shown in FIG. 8.

[0073] In the example shown in FIG. 9, a young woman wearing a uniform asks, "Do you know of any clothing stores?" The automatic answering system 10 recognizes that the situation of the questioner 30 is that of a young woman wearing a uniform, and based on this recognition result, the question, and information in the facility data indicating the products sold by each store and the target audience of each store, determines that Shop C and Shop D are recommended clothing stores for female high school students (high school girls). The automatic answering system 10 then generates information directing to Shop C and Shop D as an answer to the question and displays it on the output unit 15. In the example shown in FIG. 9, as in the example shown in FIG. 8, evidence information is also displayed. Specifically, the text "high school girl" is generated as evidence information, and an answer including the evidence information is displayed.

[0074] In the example shown in FIG. 10 , a middle-aged man wearing a classic hat and classic clothing asks, "Do you know where a clothing store is?" The automatic response system 10 recognizes that the situation of the questioner 30 is a middle-aged man wearing a classic hat and classic clothing. Based on this recognition result, the question, and information in the facility data indicating the products sold by each store and the target audience of each store, the automatic response system 10 determines that an e-shop is a recommended clothing store for middle-aged men who like classic fashion. The automatic response system 10 then generates information providing directions to the e-shop as an answer to the question and displays it on the output unit 15. In the example shown in FIG. 10 , similar to the examples shown in FIGS. 8 and 9 , evidence information is also displayed. Specifically, the text "dandy" is generated as evidence information, and an answer including the evidence information is displayed. Note that in the examples shown in FIGS. 8 , 9 , and 10 , the gender of the questioner 30 is also taken into consideration. However, the gender of the questioner 30 is also an attribute, and these examples can be said to generate an answer taking into consideration both the situation, such as clothing, and the attribute.

[0075] 11 and 12 show examples in which the facial expression or emotion of the questioner 30 is taken into consideration as the situation of the questioner 30. In both of the examples shown in Fig. 11 and Fig. 12, the questioner 30 is asking where home is.

[0076] In the example shown in FIG. 11, the questioner 30 asks, "Where is the platform?" in a calm and normal manner. The automatic answering system 10 recognizes that the questioner 30 is normal, i.e., calm, from the questioner's facial expression, movements, tone of voice, etc., and generates an answer while engaging in a dialogue with the questioner 30. In the example shown in FIG. 11, the automatic answering system 10 asks the questioner 30, "Where do you want to take the train?", and the questioner 30 replies, "I want to go to XX," and the automatic answering system 10 generates and outputs the answer, "For XX, please proceed to platform 3," based on the answer from the questioner 30.

[0077] On the other hand, in the example shown in FIG. 12, the questioner 30 appears flustered and asks, "Where is the platform?" The automated answering system 10 recognizes that the questioner 30 is in a panic and is in a state of stress based on the questioner's facial expression, movements, tone of voice, etc., and outputs a response including multiple types of information at once without engaging in a dialogue with the questioner 30. In the example shown in FIG. 12, the automated answering system 10 simultaneously displays the information "Platform 3 for XX direction," "Platform 4 for YY direction," and "Platform 6 for ZZ direction." As illustrated in FIGS. 11 and 12, the content of the response, the method of interaction with the questioner 30, etc. may be determined depending on the facial expression or emotions of the questioner 30.

[0078] 13 and 14 show examples in which the facial expression or emotion of questioner 30 and the number of people are taken into consideration as the situation of questioner 30. In the example shown in FIG. 13, there are four questioners 30, who are standing shoulder to shoulder, talking and appearing to be having fun, and they ask, "Can you recommend a bar?" Based on the facial expression, movements, tone of voice, etc. of questioner 30, automatic answering system 10 recognizes that questioner 30 is in a high-energy state (in a good mood) and recognizes that questioner 30 is part of a group, and generates an answer that guides people to lively establishments "Bar A" and "Bar B." In the example shown in FIG. 13, the answer also includes evidence information, as in the examples shown in FIGS. 8, 9, and 10, and the output unit 15 also displays information such as, "For all of you who are excited, how about a lively bar that offers all-you-can-drink?"

[0079] In the example shown in FIG. 14, the questioner 30 is a man and a woman holding hands, and asks, "Can you recommend a bar?" The automatic answering system 10 recognizes that the questioner 30 is a couple based on the questioner's facial expression, movements, tone of voice, etc., and generates a response recommending quiet bars "C Bar" and "D Bar." In the example shown in FIG. 14, the response also includes evidence information, as in the examples shown in FIGS. 8, 9, and 10, and the information "How about a quiet bar for a couple?" is also presented to the output unit 15.

[0080] The specific examples shown above are merely illustrative, and the responses that take the situation into consideration in the automatic response system 10 are not limited to these examples. Furthermore, in FIGS. 8, 9, 10, and 14, responses that take gender into consideration are generated, but this is not limiting. The situation may not be recognized based on gender (male or female), and the response may not include any gender-related content. In other words, the response generator 14 may generate a response that is not dependent on gender. This allows the automatic response system 10 to output responses that take LGBTQ (Lesbian, Gay, Bisexual, Transgender, Queer / Questioning) into consideration.

[0081] The automatic response system 10 may also generate LGBTQ-friendly responses that do not distinguish between genders or that take both genders into consideration. For example, when providing restroom guidance, a response that includes both genders may be generated by providing information about the locations of men's and women's restrooms regardless of the user's gender. Rules for generating LGBTQ-friendly responses may be stored in the data storage unit 113 as facility data or separately from the facility data, and the response generation unit 14 may refer to these rules when generating responses. Ethical standards to be reflected in responses, not limited to LGBTQ-friendly responses, may be defined as rules, and these rules may be stored in the data storage unit 113 as facility data or separately from the facility data, and the response generation unit 14 may refer to these rules when generating responses. When a generation AI is used for the response generation unit 14, these rules may be specified in advance to the generation AI.

[0082] Next, the hardware configuration of the automatic answering system 10 of this embodiment will be described. The information processing units 20 in the automatic answering system 10 of this embodiment function as the information processing units 20 by executing a computer program, which is a computer program that describes the processing of each of the information processing units 20, on a computer system. FIG. 15 is a diagram showing an example configuration of a computer system that realizes each of the information processing units 20 of this embodiment. As shown in FIG. 15, this computer system includes a control unit 101, an input unit 102, a memory unit 103, a display unit 104, a communication unit 105, and an output unit 106, which are connected via a system bus 107.

[0083] In FIG. 15, the control unit 101 is a processor such as a CPU (Central Processing Unit) and executes a program describing the processing performed by the information processing unit 20 of this embodiment. The input unit 102 is composed of, for example, a keyboard, buttons, a mouse, etc., and is used by a user of the computer system to input various information. The memory unit 103 includes various memories such as RAM (Random Access Memory) and ROM (Read Only Memory) and a storage device such as a hard disk, and stores programs to be executed by the control unit 101, necessary data obtained during processing, etc. The memory unit 103 is also used as a temporary storage area for programs. The control unit 101 and the memory unit 103 constitute, for example, a processing circuit. The processing circuit may be a single circuit or multiple circuits. The display unit 104 is composed of a display, an LCD (Liquid Crystal Display), etc., and displays various screens to the user of the computer system. Note that a touch panel in which the input unit 102 and the display unit 104 are integrated may also be used. The communication unit 105 is a receiver and transmitter that perform communication processing. The output unit 106 is a speaker or the like. Note that Fig. 15 is just an example, and the configuration of the computer system that realizes each of the information processing units 20 is not limited to the example shown in Fig. 15. For example, the output unit 106 does not have to be provided.

[0084] Here, an example of the operation of the computer system until the program of this embodiment is ready to be executed will be described. In the computer system having the above-mentioned configuration, the program is installed in storage unit 103 from, for example, a CD-ROM or DVD-ROM inserted in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). Then, when the program is executed, the program read from storage unit 103 is stored in the main storage area of storage unit 103. In this state, control unit 101 executes the processes as each of information processing units 20 of this embodiment in accordance with the program stored in storage unit 103.

[0085] In the above description, a program describing the processing in each of the information processing units 20 is provided using a CD-ROM or DVD-ROM as a recording medium, but this is not limited to this. Depending on the configuration of the computer system, the capacity of the program to be provided, etc., it is also possible to use a program provided via a transmission medium such as the Internet via the communication unit 105.

[0086] The program of this embodiment causes a computer system to execute, for example, the steps of acquiring a question regarding facility usage from a facility user who is a user of the facility, acquiring additional information indicating the status of the facility user who asked the question, generating an answer to the question using the additional information, the question, and facility data that is data regarding the facility, and outputting the answer.

[0087] The preprocessing unit 112 and the answer generation unit 14 shown in FIG. 1 are realized by the control unit 101 shown in FIG. 15 executing a program stored in the storage unit 103 shown in FIG. 15. The storage unit 103 is also used to realize the preprocessing unit 112 and the answer generation unit 14. The data acquisition unit 111 shown in FIG. 1 is realized by at least one of the communication unit 105 and the input unit 102 shown in FIG. 15. Some functions of the data acquisition unit 111 may be realized by the control unit 101 and the storage unit 103. The data storage unit 113 shown in FIG. 1 is part of the storage unit 103 shown in FIG. 15. The information processing unit 20 may be realized by multiple computer systems. For example, the information processing unit 20 may be realized by a cloud computer system.

[0088] Furthermore, as described above, the question acquisition unit 12 of this embodiment is realized by at least one of an input means such as a touch panel or a keyboard, and a microphone. As described above, the situation acquisition unit 13 of this embodiment is realized by at least one of a sensor such as a camera, an input means such as a touch panel or a keyboard, and a microphone. The output unit 15 is realized by at least one of a display such as a touch panel and a speaker. As described above, if the display that realizes the output unit 15 includes a display and is a touch panel, the touch panel may also function as the question acquisition unit 12. Furthermore, the touch panel may also function as the situation acquisition unit 13. Note that when the question acquisition unit 12 and the situation acquisition unit 13 perform processing, the control unit 101 and the memory unit 103 in the computer system described above may be used to realize them. Note that when the question acquisition unit 12, the situation acquisition unit 13, the information processing unit 20, and the output unit 15 are integrated, the entire automatic response system 10 may be regarded as the computer system illustrated in FIG. 15. In this case, for example, the computer system may include a microphone as the input unit 102, and may further include a sensor as the situation acquisition unit 13. In this case, for example, the question acquisition unit 12 is realized by the input unit 102, the situation acquisition unit 13 is realized by at least one of the sensor and the input unit 102, and the output unit 15 is realized by at least one of the display unit 104 and the output unit 106.

[0089] As described above, the automated answering system 10 of this embodiment generates an answer to a question from the questioner 30 using the situation of the questioner 30. This allows a more appropriate answer to be provided to the facility user than simply answering a specific question from the facility user. Furthermore, the automated answering system 10 may generate an answer based on attributes in addition to the situation, and in this case, it is also possible to provide a facility user with a more appropriate answer according to the attributes. Furthermore, by including dynamic facility data as facility data, it is possible to provide the facility user with appropriate answers and information based on the current status of the facility.

[0090] Embodiment 2 FIG. 16 is a diagram showing an example of the configuration of an automatic answering system according to the second embodiment. The automatic answering system 10a of this embodiment includes an automatic answering device 20a and a terminal device 60. The automatic answering device 20a is similar to the information processing unit 20 of the first embodiment, except that a transmitting / receiving unit 16 is added. The terminal device 60 includes a question acquiring unit 12, a situation acquiring unit 13, an output unit 15, and a transmitting / receiving unit 17. Components having the same functions as those of the first embodiment are assigned the same reference numerals as those of the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.

[0091] In the first embodiment, an example has been described in which a facility user present in a facility uses the automatic answering system 10 installed in the facility. In this embodiment, a user who is about to use the facility is also treated as a facility user. The terminal device 60 is a device that can be operated by the facility user, such as a smartphone, tablet, or personal computer. The terminal device 60 may also be a mobile terminal that can be carried by the facility user. The facility user uses the automatic answering system 10a by using the terminal device 60, for example, at home or on the way to the facility. Note that the location where the facility user uses the automatic answering system 10a is not limited to this and may be any location, even within the facility.

[0092] The question acquisition unit 12 of the terminal device 60 acquires a question about the facility from a facility user by at least one of voice and text, similar to the question acquisition unit 12 of the first embodiment. Note that, similar to the first embodiment, the question may include an image. The question acquisition unit 12 outputs the acquired question to the transmission / reception unit 17. The question acquisition unit 12 is, for example, at least one of an input means such as a touch panel or keyboard of a smartphone, tablet, personal computer, etc., and a microphone. When an image is included in the question, for example, the facility user specifies image data stored in the terminal device 60, and the input means accepts the specification. Note that, when a question is input as voice, the question acquisition unit 12 may convert the voice data into text data and output it to the transmission / reception unit 17, or may output the voice data directly to the transmission / reception unit 17.

[0093] Similar to the situation acquisition unit 13 in the first embodiment, the situation acquisition unit 13 acquires additional information indicating the situation of the questioner 30, who is a facility user asking a question (the situation in which the questioner is placed). Similar to the first embodiment, the additional information may further include information indicating the attributes of the facility user. The situation acquisition unit 13 may acquire the additional information in the form of at least one of voice and text, or may acquire image data of the questioner 30 as the additional information. Furthermore, for example, the output unit 15 may output a screen to the questioner 30 asking questions such as the number of people using the facility and whether or not assistive devices are being used, and acquire answers entered on the screen as the additional information. The situation acquisition unit 13 may be realized, for example, by a camera built into the terminal device 60, such as a smartphone, tablet, or personal computer, and may output video data acquired by the camera to the transceiver 17 as the additional information. This camera may be, for example, an internal camera that captures the questioner from inside the terminal device 60.

[0094] The output unit 15 outputs the answer by at least one of audio output and display, similar to the output unit 15 in the first embodiment. The output unit 15 is, for example, at least one of a display of the terminal device 60 and a microphone of the terminal device 60, but is not limited to these. The display of the terminal device 60 may be a touch panel. In this case, the touch panel may also function as at least one of the question acquisition unit 12 and the situation acquisition unit 13.

[0095] The transmitting / receiving unit 17 communicates with the automatic answering device 20a to exchange information with the automatic answering device 20a. For example, the transmitting / receiving unit 17 transmits the question received from the question acquiring unit 12 and the additional information received from the situation acquiring unit 13 to the automatic answering device 20a. The transmitting / receiving unit 17 also outputs the answer received from the automatic answering device 20a to the output unit 15.

[0096] The operations of the question acquisition unit 12, the situation acquisition unit 13, the output unit 15, and the transmission / reception unit 17 may be performed by installing application software that provides a facility guidance service in the terminal device 60, or by the terminal device 60 accessing the automatic answering device 20a. For example, the automatic answering device 20a may have a function as a web server, and the above operations may be realized by the terminal device 60 accessing the web server.

[0097] The transmitter / receiver 16 of the automatic answering device 20a communicates with the terminal device 60 to exchange information with the terminal device 60. For example, the transmitter / receiver 16 outputs a question and additional information received from the terminal device 60 to the answer generation unit 14. The transmitter / receiver 16 also outputs an answer received from the answer generation unit 14 to the terminal device 60. The answer generation unit 14, like the answer generation unit 14 of the first embodiment, generates an answer using the question, additional information, and facility data, and outputs the generated answer to the transmitter / receiver 16. The automatic answering device 20a of this embodiment is realized by a computer system, like the information processing unit 20 of the first embodiment. The automatic answering device 20a may be installed inside or outside the facility.

[0098] In this embodiment, facility users can ask questions using the terminal device 60, allowing them to use the automatic response system 10a regardless of their location. Therefore, for example, before using a facility, they can obtain information about the facility in advance, either at home or on the way to the facility. Therefore, facility users can use the facility efficiently based on the information they obtain in advance. Furthermore, by including map information for generating a route to the facility in the facility data, when a facility user asks how to get to the facility, it is possible to present a route from the facility user's current location to the facility as a response. Furthermore, for example, facility data may also include data on reported lost items. Thus, if a facility user suspects that they may have left something behind at the facility after using it, they can ask about the lost item by indicating where and what they left it, and the automatic response system 10a can respond with whether or not a corresponding lost item has been reported.

[0099] 17 and 18 are diagrams showing specific examples of responses that take the situation into consideration in the automatic response system 10a of this embodiment. In both of the examples shown in Fig. 17 and Fig. 18, it is assumed that it is raining around the facility and that the facility data includes data indicating the weather.

[0100] In the example shown in FIG. 17, the facility is a department store, and questioner 30, who is planning to visit the department store, asks a question using terminal device 60 before arriving at the department store. In the example shown in FIG. 17, it is raining, and questioner 30 is holding an umbrella. Terminal device 60 accepts the question, which is voice data, "Please tell me the way to the department store," and transmits the question and accompanying information, which is video data of questioner 30, to automatic answering device 20a. Using the question, accompanying information, and facility data, automatic answering device 20a determines that since questioner 30 is holding an umbrella even though it is raining, guidance should be provided via a normal route. Automatic answering device 20a generates a normal route to the department store (for example, the shortest route) as an answer and transmits the answer to terminal device 60. Terminal device 60 displays the route to the department store as an answer. 17, the questioner 30 is holding an umbrella, but this is not limiting, and similarly, if the questioner 30 is holding a closed umbrella, the automatic response device 20a may generate a response to guide the questioner 30 along a normal route even if it is raining. In other words, the automatic response device 20a may generate a response to guide the questioner 30 along a normal route as long as the questioner 30 is holding an umbrella, regardless of the state of the umbrella.

[0101] In the example shown in FIG. 18 , the facility is a conference center where a briefing session is scheduled. In the example shown in FIG. 18 , the questioner 30 is dressed formally and does not have an umbrella. The terminal device 60 receives a question in the form of voice data, such as "Please tell me the way to the briefing session venue." The terminal device 60 transmits the question and accompanying information, which is video data of the questioner 30, to the automatic answering device 20a. The automatic answering device 20a uses the question, accompanying information, and facility data to determine that it is raining, the questioner 30 is not carrying an umbrella, and is dressed formally. Therefore, the automatic answering device 20a determines that the questioner 30 should be guided to a covered route that will protect him / her from the rain. The automatic answering device 20a generates an answer that indicates a route that will protect him / her from the rain and transmits the answer to the terminal device 60. The terminal device 60 displays the route to the conference center as the answer. As illustrated in FIGS. 17 and 18 , the automatic answering device 20a can provide an answer appropriate for the questioner 30 by determining the route to the facility based on the situation of the questioner 30.

[0102] As described above, in this embodiment, a facility user asks a question using terminal device 60, and terminal device 60 outputs a response. Therefore, facility users can use automatic response system 10a even when they are outside the facility.

[0103] In addition, a transmission / reception unit 16 may be added to the automatic response system 10 shown in embodiment 1, and the transmission / reception unit 16 may communicate with the terminal device 60, thereby making it possible to respond to both questions from facility users within the facility described in embodiment 1 and questions using the terminal device 60 described in this embodiment.

[0104] Embodiment 3 Fig. 19 is a diagram showing an example of the configuration of an automatic answering system according to the third embodiment. The automatic answering system 10b of this embodiment is similar to the automatic answering system 10 of the first embodiment, except that a rating acquisition unit 18 is added and a data storage unit 113 further stores rating information. Components having the same functions as those of the first embodiment are given the same reference numerals as those of the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.

[0105] In this embodiment, after the exchange with the questioner 30 is completed, the evaluation acquisition unit 18 acquires an evaluation result indicating an evaluation of the answer from the questioner 30 and stores the acquired evaluation result in the data storage unit 113 along with the corresponding question and answer. Here, an example in which evaluation information is stored in the data storage unit 113 is described. However, this is not limiting. An evaluation information storage unit that stores evaluation information separately from the data storage unit 113 may be provided. For example, after outputting the answer, the output unit 15 outputs a question asking the questioner 30 to evaluate the answer from the automatic response system 10b, such as "Did you get the information you wanted?" or "Was the answer appropriate?". Furthermore, if the evaluation from the questioner 30 includes negative words such as "unsatisfactory" or "not good," the evaluation acquisition unit 18 may cause the output unit 15 to output a question further asking the questioner 30 what the problem was and accept input from the questioner 30. The evaluation acquisition unit 18 may acquire the evaluation result by voice or as text data. Alternatively, the output unit 15 may present options indicating the evaluation results, and the evaluation acquisition unit 18 may acquire the selection made by the questioner 30 as the evaluation result.

[0106] Furthermore, the evaluation acquisition unit 18 may evaluate the content of the answer based on the behavior of the questioner 30. For example, after the automatic answering system 10b outputs a route to a destination to the questioner 30 who has asked for directions, it may analyze the behavior of the questioner 30 using video data of the questioner 30, and if the questioner 30 is still lost, it may evaluate that the answer was difficult to understand.

[0107] The evaluation information may be used for re-learning or additional learning by the answer generation unit 14 together with the corresponding questions and answers, for example, or may be referred to when the answer generation unit 14 generates an answer. Alternatively, the operator, manager, etc. of the automatic response system 10b may check the evaluation information, identify areas for improvement using the results of the check, and instruct the identified areas for improvement to the answer generation unit 14. Alternatively, the operator, manager, etc. of the automatic response system 10b may check the evaluation information, use the results of the check, to decide what type of re-learning or additional learning to perform for the answer generation unit 14, and the re-learning or additional learning may be performed using the determined results.

[0108] The evaluation acquisition unit 18 is, for example, at least one of an input means by operation and a microphone, similar to the question acquisition unit 12. The input means, the microphone, and other hardware may be shared with the question acquisition unit 12.

[0109] As described above, in this embodiment, the evaluation from the questioner 30 is acquired and reflected in the answer, thereby improving the accuracy of the answer. Note that the evaluation acquisition unit 18 may be added to the automatic answering system 10a of the second embodiment so that the evaluation result is reflected, or the evaluation acquisition unit 18 may be added to an automatic answering system that combines the first and second embodiments so that the evaluation result is reflected.

[0110] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention. [Explanation of symbols]

[0111] 10, 10a, 10b Automatic response system, 11 Data management unit, 12 Question acquisition unit, 13 Situation acquisition unit, 14 Answer generation unit, 15 Output unit, 16, 17 Transmitting / receiving unit, 18 Evaluation acquisition unit, 20 Information processing unit, 20a Automatic response device, 30 Questioner, 40 Main unit, 50 Camera, 60 Terminal device, 111 Data acquisition unit, 112 Preprocessing unit, 113 Data storage unit, 141 Situation recognition unit, 142 Generation unit.

Claims

1. A question acquisition unit that obtains questions from facility users, who are users of the facility, regarding destinations inside or outside the facility, A situation acquisition unit that uses imaging means to acquire supplementary information indicating the situation of the questioner, who is a user of the facility who asked the question, An answer generation unit generates an answer to the question that reflects the situation, using the aforementioned question, the aforementioned supplementary information, and facility data which is data related to the facility. An output unit that outputs the aforementioned answer, An automated response system characterized by comprising the following features.

2. The automated response system according to claim 1, characterized in that the response generation unit is capable of generating the response using data in multiple formats, including voice, text, and images, as input, and is capable of outputting the response in multiple formats.

3. The automated response system according to claim 1 or 2, characterized in that the circumstances include at least one of the following: what the questioner is wearing, whether the questioner is using any assistive devices, the number of questioners, the questioner's emotions, and the size of the luggage the questioner is carrying.

4. The automated response system according to claim 1 or 2, characterized in that the aforementioned supplementary information includes video data of the questioner.

5. The automated response system according to claim 1 or 2, characterized in that the facility data includes static data that does not change for a certain period of time and dynamic data that may change even during the certain period of time.

6. The automatic response system according to claim 5, characterized in that the dynamic data includes data acquired by sensors installed in at least one of the facilities and the surrounding area.

7. The automated response system according to claim 5, characterized in that the dynamic data includes facility-based information transmitted by at least one of the facility manager and / or the facility's employees.

8. The automated response system according to claim 5, characterized in that the dynamic data includes store-based information transmitted by employees of stores within the facility.

9. The automated response system according to claim 5, characterized in that the dynamic data includes weather information.

10. The aforementioned response generation unit, A situation recognition unit recognizes the situation based on the aforementioned supplementary information, A generation unit that generates the answer to the question using the aforementioned situation, the aforementioned question, and facility data, The automatic response system according to claim 1 or 2, characterized by comprising:

11. The aforementioned supplementary information further indicates the attributes of the questioner, The situation recognition unit recognizes the situation and the attributes based on the supplementary information, The automated response system according to claim 10, characterized in that the generation unit generates the answer to the question using the question, the situation, the attributes, and facility data.

12. The automated response system according to claim 11, characterized in that the aforementioned attributes include at least one of gender, age, age group, disability status, and occupation.

13. The aforementioned facility users include specific persons designated in advance, The facility data includes individual information which is information relating to the use of the facility by each specific person, The aforementioned status acquisition unit acquires personal authentication information for identifying an individual, The automated response system according to claim 11, characterized in that the response generation unit identifies an individual using the personal authentication information and generates the response using the corresponding individual information based on the identification result.

14. The aforementioned attributes include at least one of status, affiliation, and position. The facility data includes attribute-related information, which is information regarding the use of the facility for each attribute, and individual information, which is information regarding the use of the facility for each specific person. The aforementioned individual information includes the aforementioned attributes of the corresponding individual. The aforementioned status acquisition unit acquires personal authentication information for identifying an individual, The automated response system according to claim 12, characterized in that the response generation unit identifies an individual using the personal authentication information, recognizes the attributes of the identified individual based on the identification result and the individual information, and generates the response using the attribute-related information corresponding to the attributes.

15. The automated response system according to claim 1 or 2, characterized in that the response generation unit generates the response, which includes supporting information indicating the basis for the response, based on the circumstances used to generate the response.

16. A preprocessing unit that converts the facility data into data in a format accessible to the response generation unit, A data storage unit that stores the facility data after the conversion performed by the preprocessing unit, Equipped with, The automated response system according to claim 1 or 2, characterized in that the response generation unit generates the response using the facility data stored in the data storage unit.

17. The automated response system according to claim 1 or 2, characterized in that the response generation unit generates the response which is gender-independent.

18. The automated response system according to claim 1 or 2, characterized in that the response generation unit generates a response that prompts the questioner to re-examine the question if it cannot recognize the question or if there is insufficient information to generate a response.

19. After the aforementioned answer is output, an evaluation acquisition unit obtains the evaluation result of the aforementioned answer from the questioner. The automatic response system according to claim 1 or 2, characterized by comprising:

20. A terminal device comprising the question acquisition unit, the status acquisition unit, and the output unit, which can be operated by the questioner, An automatic response device equipped with the aforementioned response generation unit, Equipped with, The terminal device transmits the supplementary information and the question to the automated response device. The automated response device generates the answer using the facility data, the supplementary information and the question received from the terminal device, and transmits the generated answer to the terminal device. The automated response system according to claim 1 or 2, characterized in that the terminal device outputs the response.

21. The automatic response system according to claim 1 or 2, characterized in that the response generation unit determines a route to the destination according to the supplementary information and generates the response including the determined route.

22. The situation includes the size of the luggage the questioner is carrying, The automatic response system according to claim 21, characterized in that the response generation unit determines the route to the destination using the size of the package.

23. The situation includes whether or not the questioner has an umbrella, The automated response system according to claim 21, wherein the response generation unit determines whether or not it is raining in the area using weather information for the area including the facility, and if it determines that it is raining in the area, it determines the route depending on whether or not the questioner has an umbrella.

24. The automated response system according to claim 13, characterized in that the individual information includes the schedule for each individual.

25. The attribute-related information includes the range of obtainable information, The automated response system according to claim 14, characterized in that the response generation unit generates a response using information within the range of information that is permitted to be viewed, based on the range of information that can be obtained.

26. A question acquisition unit that acquires questions regarding the use of the facility from facility users, who are users of the facility, A status acquisition unit that acquires supplementary information indicating the situation of the questioner, who is a user of the aforementioned facility who asked the question, An answer generation unit generates an answer to the question using the aforementioned question, the aforementioned supplementary information, and facility data which is data relating to the facility. An output unit that outputs the aforementioned answer, Equipped with, The aforementioned response generation unit, Taking the aforementioned supplementary information into consideration, a decision is made on whether to generate answers sequentially through dialogue or to generate multiple answers together. An automated response system characterized by the following features.

27. An automated response method in an automated response system, The steps include obtaining questions from facility users regarding destinations inside or outside the facility, The steps include: using imaging means to obtain supplementary information indicating the situation of the questioner, who is a user of the facility who asked the question; Using the aforementioned questions, the aforementioned supplementary information, and facility data which are data relating to the facilities, a step is to generate an answer to the aforementioned questions that reflects the aforementioned situation. The steps include outputting the aforementioned answer, An automated response method characterized by including the following.

28. An automatic response method in an automatic response system, The steps include obtaining questions about facility use from facility users, who are the users of the facility, and The steps include obtaining supplementary information indicating the situation of the questioner, who is a user of the aforementioned facility, and A step of generating an answer to the question using the aforementioned question, the aforementioned supplementary information, and facility data which is data relating to the facility, The steps include outputting the aforementioned answer, Includes, An automated response method characterized in that, in the step of generating an answer to the aforementioned question, the step of deciding whether to generate the answer sequentially through dialogue or to generate multiple answers together, taking into consideration the supplementary information.

29. In a computer system, The steps include obtaining questions from facility users regarding destinations inside or outside the facility, The steps include: using imaging means to obtain supplementary information indicating the situation of the questioner, who is a user of the facility who asked the question; Using the aforementioned questions, the aforementioned supplementary information, and facility data which are data relating to the facilities, a step is to generate an answer to the aforementioned questions that reflects the aforementioned situation. The steps include outputting the aforementioned answer, A program characterized by causing the execution of a specific action.

30. In a computer system, The steps include obtaining questions about facility use from facility users, who are the users of the facility, and The steps include obtaining supplementary information indicating the situation of the questioner, who is a user of the aforementioned facility, and A step of generating an answer to the question using the aforementioned question, the aforementioned supplementary information, and facility data which is data relating to the facility, The steps include outputting the aforementioned answer, Includes, The program is characterized in that, in the step of generating answers to the aforementioned questions, it further includes a step of deciding whether to generate answers sequentially through dialogue or to generate multiple answers together, taking into consideration the aforementioned supplementary information.

Images