Device, method, and program for causing character to speak in digital signage

Abstract

The present invention promotes the providing of informative information to passersby in digital signage. First, customer data is registered in a database 104 (S201). Thereafter, a device 100 receives an input by a viewer of a display 111 (S202). Next, the device 100 retrieves and acquires one or more pieces of knowledge data related to the input (S203 and S204), and may also acquire metadata for each piece of knowledge data at this time. The device 100 then issues, to a generative AI model, a request to generate speech content for a character (S205), acquires a response to the request from a platform 120 (S206), and specifies the metadata associated with the knowledge data identified by means of an identifier included in the response (S207). The device 100 then transmits the speech content and the specified metadata to control equipment 110 (S208).

Description

Device, method, or program for causing a character to speak in digital signage

[0001] The present invention relates to a device, method, or program for causing a character to speak in digital signage.

[0002] Digital signage is a system that uses a display connected to a communication network to convey various information to passersby, and is installed both indoors and outdoors in public transportation, public facilities, and other various locations.

[0003] This system can display visually attractive content using a high-performance display or provide content using data collected in real time via a communication network, thus attracting the interest of viewers.

[0004] However, in order to fully exert the effect of digital signage, it is necessary for passersby to recognize its existence and actively use it. Currently, there are cases where passersby do not notice its existence or, even if they notice it, they may not be able to extract sufficient information.

[0005] The present invention has been made in view of such points, and its problem is to promote the provision of useful information to passersby in digital signage.

[0006] To solve these problems, a first aspect of the present invention provides a method for causing a character to speak, wherein the server constituting a digital signage system comprising a control device connected to a display and a server capable of communicating with the control device, the method comprising: receiving input from a viewer of the display; searching for one or more pieces of knowledge data related to the input; acquiring the one or more pieces of knowledge data that have been searched; making a request to a generating AI model including an instruction to generate speech content for the character corresponding to the input, wherein the instruction includes all or part of the one or more pieces of knowledge data; acquiring a response to the request which includes the speech content; and transmitting the speech content, and at least a portion of the knowledge data used to generate the speech content or at least a portion of the metadata associated with the knowledge data, to the control device.

[0007] Furthermore, a second aspect of the present invention is the method of the first aspect, wherein the response includes one or more identifiers of one or more pieces of knowledge used to generate the utterance content.

[0008] Furthermore, a third aspect of the present invention is the method of the first aspect, wherein the step of acquiring one or more knowledge data is a step of acquiring one or more knowledge data and metadata associated with each of them.

[0009] Furthermore, a fourth aspect of the present invention is a method of the second aspect, further comprising the step of obtaining metadata associated with knowledge data identified by at least one of the one or more identifiers contained in the response, after obtaining the response.

[0010] Furthermore, a fifth aspect of the present invention is the method of the third or fourth aspect, wherein each metadata includes a video or URL relating to the knowledge data to which it is associated.

[0011] Furthermore, a sixth aspect of the present invention is a method according to any of the first to fifth aspects, wherein the instruction includes one or more features of the character.

[0012] Furthermore, a seventh aspect of the present invention is the method of the sixth aspect, wherein the one or more features include at least one of the character's name, age, occupation, favorite things, personality, and appearance.

[0013] Furthermore, an eighth aspect of the present invention is a method according to any of the first to fifth aspects, wherein the instruction includes a description of outputting one or more parameters representing the generated speech content or the emotions of the character uttering it.

[0014] A ninth aspect of the present invention is a method according to any of the first to eighth aspects, wherein the input is a voice input.

[0015] A tenth aspect of the present invention is a method according to any of the first to eighth aspects, wherein the input is the selection of a button displayed on the display.

[0016] An eleventh aspect of the present invention is a method according to any of the first to tenth aspects, further comprising the step of transmitting the content of a verbal utterance directed at a passerby in front of the display to the control device, wherein the step of receiving input from the viewer is the step of receiving input from the passerby in response to the verbal utterance.

[0017] A twelfth aspect of the present invention is a method according to the eleventh aspect, further comprising the steps of: receiving an image captured by a camera connected to the control device; analyzing the image to detect the presence of a passerby; and extracting the characteristics of the detected passerby, wherein the content of the verbal utterance is a verbal utterance corresponding to the characteristics.

[0018] A thirteenth aspect of the present invention is a method according to the eleventh aspect, further comprising the step of transmitting a notification corresponding to the features to the control device.

[0019] A fourteenth aspect of the present invention is a program for causing a server constituting a digital signage system comprising a control device connected to a display and a server capable of communicating with the control device to execute a method for causing a character to speak, the method comprising the steps of: receiving input from a viewer of the display; searching for one or more pieces of knowledge data related to the input; acquiring the one or more pieces of knowledge data that have been searched; and making a request to a generating AI model that includes an instruction to generate a character's speech content corresponding to the input, wherein the instruction includes all or part of the one or more pieces of knowledge data; acquiring a response to the request that includes the speech content; and transmitting the speech content and at least part of the knowledge data used to generate the speech content or at least part of the metadata associated with the knowledge data to the control device.

[0020] A fifteenth aspect of the present invention is a server comprising a control device connected to a display and a server capable of communicating with the control device, wherein the server receives input from a viewer of the display, searches for one or more knowledge data related to the input, obtains the one or more knowledge data, and generates speech content for a character corresponding to the input, the instruction being configured to make a request to a generating AI model including all or part of the one or more knowledge data, obtain a response including the speech content, and transmit the speech content and at least part of the knowledge data used to generate the speech content or at least part of the metadata associated with the knowledge data to the control device.

[0021] A sixteenth aspect of the present invention is a method for generating character speech content, comprising the steps of: acquiring input; searching for one or more knowledge data related to the input; acquiring the one or more retrieved knowledge data; and making a request to a generating AI model, which includes an instruction to generate character speech content corresponding to the input, wherein the instruction includes all or part of the one or more knowledge data; acquiring a response to the request, which includes the speech content; and storing the speech content and at least part of the knowledge data used to generate the speech content or at least part of the metadata associated with the knowledge data.

[0022] According to one aspect of the present invention, in addition to the utterances made by a character in response to input from the viewer, the knowledge data used to generate the utterance content or metadata pre-associated therewith is presented as reliable information, thereby enabling the provision of highly reliable information while strongly attracting the viewer's attention through interaction with the character.

[0023] This figure shows a digital signage system according to the first embodiment of the present invention. This figure shows the flow of how to use the digital signage according to the first embodiment of the present invention. This figure shows an example of the format of knowledge data according to the first embodiment of the present invention. This figure shows an example of a prompt according to the first embodiment of the present invention. This figure shows an example of a character response according to the first embodiment of the present invention. This figure shows the flow of voice prompts according to the second embodiment of the present invention.

[0024] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0025] (First Embodiment) Figure 1 shows a system according to the first embodiment of the present invention. The device 100 communicates with a control device 110 connected to a display 111 and a platform 120 that provides an AI model via an IP network such as the Internet. The device 100 can also communicate with a database 104 in which customer data collected from customers using the service provided by the device 100 (hereinafter also referred to as the "digital signage service") is registered, and acquire the necessary data. The control device 110 may be further connected to sensors 112 such as cameras and microphones. The sensors 112 connected to the control device 110 may be integrated in whole or in part with the display 111. The AI ​​model is described exemplarily as being provided by a platform 120 that can communicate with the device 100, but an application for providing an AI model can also be run on the device 100 so that the AI ​​model is provided by the device 100.

[0026] The device 100 comprises a communication unit 101 such as a communication interface, a processing unit 102 such as a processor or CPU, and a storage unit 103 including a storage device or storage medium such as memory or a hard disk. The device can be configured by executing a program for performing each process or operation in the processing unit 102. The device 100 may include one or more devices, computers, or servers. The program may also include one or more programs and can be recorded on a computer-readable storage medium to form a non-transient program product. The program is stored in a storage device or storage medium such as a database 104 accessible from the storage unit 103 or the device 100 via an IP network, and at least one processor in the processing unit 102 can execute the instructions contained in the program. The data described below as being stored in a storage device or storage medium such as a database 104 may be stored in the storage unit 103, and vice versa.

[0027] This section will explain the information provision process triggered by passersby viewing the display 111 and making inputs. First, the registration of customer data in the database 104 (S201) will be explained, followed by the information provision process according to this embodiment.

[0028] Customers of a digital signage service provide the service operator with customer data that they wish to use in their communication with passersby. This data can be provided by any means, including inputting it through a webpage provided by device 100, or by electronically providing it to the operator's representative through the customer's representative. For example, if database 104 is a vector database, device 100 may register the customer data in database 104 by requesting database 104 to create an index of the received customer data. In this case, the customer data is divided into units called "chunks" and stored in database 104. However, registration of customer data in database 104 is not limited to indexing in a vector database; ultimately, it is sufficient for multiple "knowledge data" representing knowledge about the customer or its products or services to be registered in database 104. For example, the customer data itself may consist of multiple pieces of knowledge data. Each piece of knowledge data can be text data, image data, or a URL thereof.

[0029] Each piece of knowledge data registered in database 104 can be associated with metadata. Each piece of metadata includes at least one of the following: a description of the knowledge data to which it is associated and a video related to that knowledge data or its URL. When customer data is indexed, database 104 can acquire the customer data and its associated metadata, and associate the metadata with each of the multiple pieces of knowledge data based on that customer data. The video can be an image, a video, etc.

[0030] Subsequently, the device 100 receives input from the viewer of the display 111 on which the character is displayed (S202). This input can be input to the display 111, which is a touch panel such as selecting buttons displayed on the display 111; voice input to the microphone 112; gestures to the camera 112, etc. For example, selecting the "Voice Input" button displayed on the display 111 may initiate the reception of voice via the microphone 112. This input is often a question, but it may not be in the form of a question, or its content may not be a question.

[0031] Next, the device 100 searches for one or more knowledge data related to the input (S203). The device 100 may use the input as is for searching the database 104, or it may convert the input into data suitable for searching. For example, the input may be translated into a language such as English or Japanese. The search can be performed by creating a query according to the type of database 104 and inputting the query into the database 104. If the database 104 is a vector database, the search will extract one or more chunks from among the registered knowledge data that have a relatively high relevance to the input.

[0032] The device 100 acquires one or more retrieved knowledge data, and if registered, it may further acquire metadata for each (S204). The knowledge data may be in the format shown in Figure 3, including the metadata associated with it. In the example in Figure 3, the variable {index} is set to an identifier such as the knowledge number, the variable {title} to the title of the knowledge, the variable {description} to the description of the knowledge, the variable {url} to the URL of an image, video, or other visual content corresponding to the knowledge, and the variable {knowledge} to the content of the knowledge. The knowledge data includes an identifier and content, and may further include at least one of the title, description, and URL. The title, description, and URL may be included in metadata in a different format from the knowledge data as needed, for example, by including the identifier of the knowledge in the metadata, it can be associated with the knowledge data.

[0033] Then, the device 100 makes a request to the generating AI model (hereinafter also referred to as the "first generating AI model") which includes an instruction to generate utterance content of the character displayed on the display 111 that corresponds to the input (S205). More specifically, the device 100 can create an instruction to generate utterance content of the character that corresponds to the input and make a request to the first generating AI model which includes this instruction (hereinafter also referred to as the "first request").

[0034] In this specification, "AI model" refers to a machine learning model trained to predict output for a given input, and "generative AI model" refers to a large-scale language model (LLM) trained to generate output not included in the given input. While an LLM applying a transformer architecture is particularly preferred as a generative AI model, it is anticipated that the architecture's name may change with technological advancements. Therefore, in this specification, "transformer architecture" encompasses architectures using one or more features of a transformer architecture or improvements thereof. In the example in Figure 2, the required request is made by a single API call; however, this may be achieved by dividing the request into multiple calls to the API. In this case, the multiple API calls may be made to the same generative AI model or to different generative AI models. In this specification, whether or not "generative AI models" are identical is determined by whether or not the type of generative AI model specified by the user is identical. In the example of the OpenAI API, if the value of the variable "model" is the same, it is expressed as being the same generative AI model. If multiple generative AI models are not identical, they may be provided on the same platform 120. If multiple requests pertain to a generative AI model provided on the same platform 120, those requests may be fulfilled by executing a single code.

[0035] Figure 4 shows an example of instructions according to the first embodiment of the present invention. In the example in Figure 4, the input from the viewer acquired by the device 100 is set to the variable {input}, and the input is described in the instructions. Data obtained by processing the viewer's input, i.e., data based on the input, may also be set to the value of the variable. In addition, one or more knowledge data acquired from the database 104 are also described in the instructions in the format described with reference to Figure 3, for example, as needed. Only a portion, or not all, of the one or more knowledge data acquired from the database 104 may be included in the instructions. Figure 4 shows an example in which two pieces of knowledge are provided, and the variables {knowledge_1} and {knowledge_2} are set to the specific content of the knowledge. Furthermore, if there is knowledge that should be provided regardless of the viewer's input, knowledge data representing that knowledge may be included in the instructions in advance. The instructions may also include one or more characteristics such as the character's name, age, occupation, likes, personality, and appearance.

[0036] The instructions created by device 100 describe considering a character response to user input, and if one or more pieces of knowledge are used in the response, outputting the identifiers of those one or more pieces of knowledge along with the response. More generally, the instructions created by device 100 may include a description of outputting the utterance of the character corresponding to the user input and one or more identifiers of the one or more pieces of knowledge used to generate that utterance. These instructions can be created by storing code for creating instructions for the first generating AI model in the storage unit 103 and executing it on device 100. Then, by storing code for a first request to the first generating AI model in the storage unit 103 and executing it on device 100, the OpenAI API can be called and a request including the created instructions can be made to the AI ​​model provided on platform 120. The code for creating instructions for the first generating AI model may be included as part of the code for the first request to the first generating AI model. The Open API described above is an example, and other APIs may be used.

[0037] The instructions included in the first request may also include a description of generating and outputting one or more parameters that represent the generated speech content or the emotions of the character speaking it, such as joy, anger, sadness, etc. By outputting one or more such parameters, the tone of the character's final speech, such as speed, pitch, and volume, can be adjusted.

[0038] Furthermore, the instructions may also include a statement that if the viewer's input to the display 111 includes a language of their choice, the spoken content will be output in that language.

[0039] Next, the device 100 obtains a response to the first request from the platform 120 (S206), and identifies metadata associated with the knowledge data identified by one or more identifiers included in the response (S207). Then, the device 100 transmits the utterance and at least a portion of the identified metadata to the control device 110 (S208).

[0040] If the first request contains only one piece of knowledge, the knowledge used to generate the utterance is uniquely determined even if the response from the first generating AI model does not include an identifier. Similarly, if all one or more pieces of knowledge included in the first request are used, the knowledge used to generate the utterance is uniquely determined even if the response from the first generating AI model does not include an identifier. Thus, since metadata can be identified even if the identifier is not explicitly included in the response, the response from the first generating AI model does not necessarily need to include an identifier.

[0041] In the above explanation, an example was given in which metadata associated with each knowledge data is obtained when one or more knowledge data are retrieved from the database 104. However, it is also possible to obtain metadata associated with the knowledge data identified by at least one of the one or more identifiers included in the response after obtaining a response from the first generating AI model.

[0042] Then, based on the received speech content, the control device 110 causes the speech of the character displayed on the display 111 to be output to a speaker (not shown) connected to the control device 110, and displays on the display 111 the explanation included in the received metadata and the image included in the metadata or the image based on the URL included in the metadata (S209). Regarding the synthesis and reproduction of the voice based on the speech content and the movement of the character according to the reproduction of the voice, known techniques can be appropriately used. In addition to the reproduction of the voice based on the speech content, the speech content may be displayed on the display 111. The speaker only needs to be directly or indirectly connected to the control device 110. For example, it may be provided in the display 111 or in the camera 112.

[0043] FIG. 5 shows an example of a display screen during speech according to the first embodiment of the present invention. In this example, in addition to the character 510, the speech content 520, the explanation 531, and the image 532 are displayed on the display screen 500.

[0044] As described above, according to an embodiment of the present invention, in addition to the speech by the character corresponding to the input from the viewer, by showing the metadata pre-associated with the knowledge data used for generating the speech content as highly reliable and certain information, it is possible to strongly attract the viewer's interest through the interaction with the character while providing highly reliable information. <000009 "> In the above description, the speech content of the character is described as being such that the voice based on it is reproduced. However, the speech content may be displayed on the display 111 without reproducing the voice. Also, although the character is described as being displayed on the display 111, if the speech content of a specific character is reproduced or displayed, the character itself does not necessarily have to be displayed on the display 111. Further, not necessarily limited to the speech of the character in digital signage, it is also conceivable to apply the invention according to this embodiment when causing a character to speak in response to an input from a user.

[0046] Also, since the knowledge data based on the data collected from customers can be said to be highly reliable in itself, even if the apparatus 100 transmits at least part of the generated utterance content and one or more pieces of knowledge used to generate the utterance content to the control device 110, it is possible to obtain the effect of improving engagement through interaction with the character while providing highly reliable information.

[0047] As a measurement of the effect of interaction with the character, the apparatus 100 may measure the staying time of the viewer of the display 111. As the start time of the measurement, examples include the time when the viewer inputs, the time when the viewer enters the shooting range of the camera 112, and the time when the viewer stops in front of the display 111. As the end time of the measurement, examples include the time when the viewer exits the shooting range of the camera 112 and the time when the viewer starts to move away from in front of the display 111.

[0048] (Second Embodiment) The digital signage according to the first embodiment can be introduced into various environments such as public transportation facilities such as airports and stations, outdoors of stores such as restaurants and retail stores, and tourist attractions. Since the use of digital signage is not promoted unless passersby in front of the display 111 notice its presence, in the system according to this embodiment, the passersby are greeted.

[0049] First, the apparatus 100 receives the video captured by the camera 112 from the control device 110 (S601). Then, the apparatus 100 analyzes the video to detect the presence of a passerby (S602). The analysis can be performed, for example, using an AI model, either on the apparatus 100 or on a server (not shown) capable of communicating with the apparatus 100. The detection can be performed, for example, by detecting a human face.

[0050] If the device 100 detects the presence of a passerby, it extracts the passerby's characteristics as necessary (S603). This extraction can be performed, for example, using an AI model, and may be performed on the device 100 or on a server (not shown) that can communicate with the device 100. This server may be the same as the server for video analysis described above. The characteristics to be extracted may include age, gender, clothing, facial expression, emotions, etc.

[0051] The device 100 then generates the content of the speech directed at the passerby, for example, using a generative AI model, and transmits the generated speech content to the control device 110, which outputs it to the speaker as character speech (S604). The speech content can be tailored to the extracted features. The display 111 may display a notification received by the control device 110 from the device 100, which corresponds to the extracted features.

[0052] When a passerby makes an input in response to the utterance, the device 100 can initiate the processing or operation according to the first embodiment, triggered by the input.

[0053] Thus, according to one embodiment of the present invention, by having a character actively call out to passersby and attract their attention, it becomes possible to turn passersby into viewers and make more people aware of the existence of the digital signage.

[0054] (Third Embodiment) In the first embodiment, viewers of the display 111 make inputs, and character speech content is generated using those inputs. However, various inputs may be made to the device 100 in advance, and the device 100 may generate speech content corresponding to each input it acquires, identify the knowledge data used to generate the speech content, or identify metadata associated with the knowledge data, and store it in the storage unit 103, as necessary, in association with the input. In this way, when a viewer actually makes an input, if speech content corresponding to that input has already been generated, information can be provided to the viewer quickly. If the generated speech content and at least a part of the knowledge data used to generate the speech content, or at least a part of the metadata associated with the knowledge data are stored in the control device 110, information can be provided even more quickly, and the device will be less affected by the communication status with the device 100. Furthermore, the invention according to this embodiment may be applied not only to character speech in digital signage, but also when a character makes a speech in response to user input.

[0055] Furthermore, in the embodiments described above, unless the word "only" is used, such as "based only," "depending only," "in the case of only," or "referencing only," it is assumed in this specification that additional information may also be considered. Also, as an example, the statement "if a, then b" does not necessarily mean "always b in the case of a" or "b immediately after a," unless explicitly stated otherwise. In addition, the statement "each a constituting A" does not necessarily mean that A is composed of multiple components, but includes the possibility that the component is singular.

[0056] Furthermore, it should be noted that the embodiments of the present invention described above are included in the disclosure herein, in any way that they are not inconsistent with each other.

[0057] Furthermore, for the sake of clarity, even if there are aspects of operation in some method, program, terminal, device, server, or system (hereinafter referred to as "method, etc.") that differ from the operation described herein, each aspect of the present invention is intended to cover the same operation as any of the operations described herein, and the existence of operation different from the operation described herein does not mean that the method, etc. is outside the scope of each aspect of the present invention.

[0058] Furthermore, the "start" and "end" shown in Figure 6 are merely examples, and do not necessarily mean that the method according to this embodiment will always start or end in the illustrated procedure.

[0059] 100 Device 101 Communication Unit 102 Processing Unit 103 Storage Unit 104 Database 110 Control Device 111 Display 112 Sensor 120 Platform 500 Display Screen 510 Character 520 Speech Content 531 Description 532 Image

Claims

1. A method for causing a character to speak, comprising a server constituting a digital signage system comprising a control device connected to a display and a server capable of communicating with the control device, the method comprising: receiving input from a viewer of the display; searching for one or more pieces of knowledge data related to the input; acquiring one or more pieces of knowledge data that have been retrieved; making a request to a generating AI model including an instruction to generate speech content for the character corresponding to the input, wherein the instruction includes all or part of the one or more pieces of knowledge data; acquiring a response to the request, which includes the speech content; and transmitting the speech content, and at least a portion of the knowledge data used to generate the speech content or at least a portion of the metadata associated with the knowledge data, to the control device.

2. The method according to claim 1, wherein the response includes one or more identifiers of one or more pieces of knowledge used to generate the utterance content.

3. The method according to claim 1, wherein the step of acquiring the one or more knowledge data is the step of acquiring the one or more knowledge data and metadata associated with each of them.

4. A method according to claim 2, further comprising the step of obtaining metadata associated with knowledge data identified by at least one of the one or more identifiers contained in the response, after obtaining the response.

5. The method according to claim 3 or 4, wherein each metadata includes a video or URL relating to the knowledge data to which it is associated.

6. The method according to any one of claims 1 to 5, wherein the instruction includes one or more features of the character.

7. The method according to claim 6, wherein the one or more features include at least one of the character's name, age, occupation, favorite things, personality, and appearance.

8. A method according to any one of claims 1 to 5, wherein the instruction includes a description of outputting one or more parameters representing the generated utterance or the emotions of the character uttering it.

9. The method according to any one of claims 1 to 8, wherein the input is a voice input.

10. The method according to any one of claims 1 to 8, wherein the input is the selection of a button displayed on the display.

11. A method according to any one of claims 1 to 10, further comprising the step of transmitting to the control device the content of a verbal utterance directed at a passerby in front of the display, wherein the step of receiving input from the viewer is the step of receiving input from the passerby in response to the verbal utterance.

12. The method according to claim 11, further comprising the steps of: receiving video footage captured by a camera connected to the control device; analyzing the video footage to detect the presence of a passerby; and extracting characteristics of the detected passerby, wherein the spoken content of the voice call is a spoken content corresponding to the characteristics.

13. The method according to claim 11, further comprising the step of transmitting a notification corresponding to the features to the control device.

14. A program for causing a server constituting a digital signage system comprising a control device connected to a display and a server capable of communicating with the control device to execute a method for causing a character to speak, the method comprising: receiving input from a viewer of the display; searching for one or more pieces of knowledge data related to the input; acquiring one or more pieces of knowledge data that have been searched; making a request to a generating AI model including an instruction to generate speech content for the character corresponding to the input, wherein the instruction includes all or part of the one or more pieces of knowledge data; acquiring a response to the request, which includes the speech content; and transmitting the speech content, and at least a portion of the knowledge data used to generate the speech content or at least a portion of the metadata associated with the knowledge data, to the control device.

15. A server comprising a digital signage system including a control device connected to a display and a server capable of communicating with the control device, wherein the server receives input from a viewer of the display, searches for one or more pieces of knowledge data related to the input, obtains the one or more pieces of knowledge data, and generates speech content for a character corresponding to the input, the instruction being configured to make a request to a generating AI model including all or part of the one or more pieces of knowledge data, obtain a response including the speech content, and transmit the speech content and at least part of the knowledge data used to generate the speech content or at least part of the metadata associated with the knowledge data to the control device.

16. A method for generating character speech content, comprising the steps of: obtaining input; searching for one or more knowledge data related to the input; obtaining the one or more retrieved knowledge data; making a request to a generating AI model, which includes an instruction to generate character speech content corresponding to the input, wherein the instruction includes all or part of the one or more knowledge data; obtaining a response to the request, which includes the speech content; and storing the speech content and at least part of the knowledge data used to generate the speech content or at least part of the metadata associated with the knowledge data.

Images