System

Abstract

A system is provided.SOLUTION: The system includes a means for collecting information on listed companies from the Internet, a means for converting the collected voice / moving image contents into text, a means for extracting, emphasizing and summarizing important keywords and numerical values from the text data, a means for storing the extracted information in a database so as to be retrievable, a means for providing the text data and the extracted information to a terminal of a user, and a means for responding to a question from the user through an online chat function with automatic or human intervention.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The wide variety and volume of IR information for listed companies makes it difficult for investors to efficiently collect and analyze it. Furthermore, communicating directly with individual companies is time-consuming and labor-intensive, making it difficult for investors to make appropriate investment decisions. It is essential to resolve these issues and enable investors to quickly and accurately grasp a company's business situation and future prospects. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes: means for collecting information on listed companies from the Internet; means for converting the collected audio and video content into text; means for extracting, highlighting, and summarizing important keywords and figures from the converted text; means for storing the extracted information in a database and making it searchable; means for providing the text data and extracted information to users' devices; and means for responding to users' questions automatically or with human intervention via an online chat function. This allows users to efficiently obtain the information they need and make appropriate investment decisions quickly. Furthermore, by processing the information using a generative AI model, the system provides even more accurate information.

[0006] A "listed company" is a company that meets certain criteria and whose shares are publicly traded on a stock exchange.

[0007] "Information" refers to content provided in the form of text, audio, or video, such as various data, reports, and news about a company.

[0008] "Internet collection methods" refers to techniques that use crawlers or APIs to automatically collect publicly available web content.

[0009] "Audio and video content" refers to audio and video data recorded at corporate financial results briefings and general shareholders' meetings.

[0010] "Text conversion means" refers to the process of converting voice data into text information using voice recognition technology.

[0011] "Means for extracting, highlighting, and summarizing important keywords and figures" refers to technology that uses generative AI models to automatically find particularly meaningful and investor-useful information from text data and display it visually in a prominent manner.

[0012] "Means for storing and retrieving information in a database" refers to the technology that stores the processed information in a database and enables efficient retrieval and access.

[0013] "Means of providing information to the user's device" refers to the technology for displaying processed information on the user's PC or smartphone via the Internet.

[0014] "Online chat function" refers to a technology in which a user inputs a text-based question and the system provides an answer through automated responses or human intervention.

[0015] A "generative AI model" is an artificial intelligence technology that learns from large amounts of data and analyzes and generates text. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to a system that efficiently collects, analyzes, and provides investor relations information of listed companies to investors. This system operates in cooperation with a server, a terminal, and a user, and is specifically implemented as follows.

[0038] Server Processing

[0039] 1. Data Collection

[0040] The server periodically collects video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using crawlers and APIs.

[0041] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[0042] 2. Speech-to-text

[0043] The server transmits the stored voice data to a voice recognition API (for example, Google (registered trademark) Cloud Speech-to-Text API) to generate text data.

[0044] Evaluate the quality of the generated text data and manually correct it if necessary.

[0045] 3. Keyword extraction and information emphasis

[0046] The server utilizes generative AI models (e.g., BERT, GPT-3 (registered trademark)) to automatically extract important keywords and numbers from text data and highlight them.

[0047] The extracted information is summarized within the server and stored in a database along with associated metadata.

[0048] 4. Database operation

[0049] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[0050] Terminal handling

[0051] 1. Providing a user interface

[0052] The device (user's PC or smartphone) provides the user with an easy-to-use search interface through web applications or mobile applications.

[0053] The device provides a dashboard that displays the latest information and investor relations information for noteworthy companies.

[0054] 2. Performing a search function

[0055] The device sends the search query entered by the user to the server, formats the data returned by the server, and displays it to the user.

[0056] User operations

[0057] 1. Information Search

[0058] Users enter the company name or industry information of interest in the search box and click the search button.

[0059] For example, a user searches for "XYZ Company's Q2 2023 financial results."

[0060] 2. Viewing Information

[0061] The user can review the search results provided and select information of particular interest for further details.

[0062] Search results highlight important keywords and figures, allowing users to quickly grasp the main points.

[0063] 3. Chat questions

[0064] If users want to know more about a particular piece of information, they can use the chatbot to type in a question, such as, "What's the outlook for next year?"

[0065] The chatbot receives the user's question, extracts relevant information from the appropriate database, generates an answer, and displays it again on the device.

[0066] If necessary, a human agent will step in to provide a more detailed answer or interpretation.

[0067] Specific examples

[0068] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[0069] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Furthermore, if a user has a specific question, they can use the chatbot to ask, and relevant information will be returned instantly. This system allows users to quickly and efficiently obtain the information they need to make investment decisions.

[0070] In this way, the server, terminals, and users work together to collect, analyze, provide, and respond to information in a consistent process. This system allows investors to obtain more accurate information quickly and make appropriate investment decisions.

[0071] The processing flow will be explained below.

[0072] Server Processing

[0073] Step 1:

[0074] The server runs a crawler to collect video and audio content of financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet, including on the company's official website and YouTube® channel.

[0075] Step 2:

[0076] The server stores the collected video and audio data in cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[0077] Step 3:

[0078] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data. The accuracy of the generated text is checked and reprocessed if necessary.

[0079] Step 4:

[0080] The server extracts important keywords and figures from the text data using a generative AI model that automatically detects and highlights important information that frequently appears in the text.

[0081] Step 5:

[0082] The server highlights the extracted keywords and values ​​and stores the summarized information in a database, which is indexed and optimized for fast and efficient searching.

[0083] Terminal handling

[0084] Step 1:

[0085] The device provides users with an intuitive search interface through web and mobile applications, allowing them to easily find information that interests them.

[0086] Step 2:

[0087] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[0088] Step 3:

[0089] The terminal receives the search results returned from the server, formats the results in a list or table format, and displays them to the user.

[0090] User operations

[0091] Step 1:

[0092] Users enter the company name or industry information of interest in the search box and click the search button.

[0093] Step 2:

[0094] The user can check the list of search results provided, select and click on the information that interests them, and detailed information will be displayed.

[0095] Step 3:

[0096] If a user wants more information about a particular piece of information, they can use the chat interface to type in a question, such as "What's the outlook for next year?"

[0097] Step 4:

[0098] The terminal sends the user's question to the server, which extracts relevant information from a database, generates an appropriate answer, and sends it to the terminal.

[0099] Step 5:

[0100] The user sees the answers displayed in the chat interface and can ask more detailed questions if needed.

[0101] The above processing steps enable effective information collection, analysis, and provision between the server, terminals, and users, allowing investors to obtain the information they need to make investment decisions more quickly and accurately.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Currently, it is difficult to efficiently collect, analyze, and provide investor relations information for listed companies to investors. Many challenges exist, particularly in the process of extracting important information from audio and video content and quickly providing it as data necessary for investment decisions. Specifically, there is a need for automation of data collection and conversion, accurate extraction of important information, and provision of information in a format that is easily accessible to users.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes: [means for collecting information on aggregating entities from a communication network;] [means for converting the collected audio and video into text information; and] [means for extracting, emphasizing, and summarizing key identifiable elements and numerical values ​​from the converted text information.] This makes it possible [to streamline the entire process from collection to analysis and provision, enabling investors to quickly obtain the information they need].

[0107] "Aggregate entities" refer to companies that are listed on the stock market and that disclose information about their financial and operational activities.

[0108] "Communications network" refers to a network for transmitting digital information, including the Internet, which is used to collect, send, and receive data.

[0109] "Audio and video" refers to audio and video content released at corporate financial results briefings and general shareholders' meetings, etc., which includes important information about corporate activities.

[0110] "Text information" refers to text data converted from audio or video, formatted in a format that is easy for humans to read.

[0111] "Key identifiable elements" refer to keywords and numerical information that are recognized as being particularly important in text data, and play an important role in investment decisions.

[0112] "Storage" refers to a storage device, such as a database, for storing collected data so that it can be easily accessed at a later time.

[0113] "Information terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to obtain information.

[0114] "Online interactivity" refers to an interface through which a user can input questions or requests to a system, including automated chatbots and human operators.

[0115] "Remote storage" refers to a remotely accessible storage facility, such as cloud storage, that allows large amounts of data to be stored securely and efficiently.

[0116] "Generated AI model" refers to a model that uses artificial intelligence technology to analyze data, thereby automatically extracting key information.

[0117] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[0118] Server Operation

[0119] The server uses multiple hardware and software components to collect, process, and store data. Specifically, it uses crawlers and APIs to gather information from the aggregator across the internet. The collected data is stored as audio and video and stored in cloud storage, such as Amazon S3.

[0120] The server then converts the collected audio and video data into text using the Google Cloud Speech-to-Text API, allowing important information from the video and audio data to be recorded in text format.

[0121] The server then uses the generated AI model (e.g., GPT-3) to analyze the text and send prompts to extract key identifiable elements and numbers, such as "Please extract the key points from the financial report."

[0122] The extracted information is highlighted in bold or color-coded and stored in a database. The stored data is managed using a database system such as MySQL®, and the index is optimized for efficient searches.

[0123] Device behavior

[0124] The terminal provides an interface for users to obtain information. It is designed to allow users to easily search and browse using web and mobile applications. Specifically, dashboards and search interfaces are built using frameworks such as React and Flutter (registered trademark).

[0125] When a user enters the name or information of a company of interest in the search box, the device sends the query to the server, formats the data received from the server, and displays it to the user. For example, enter "XYZ Company's 2023 Q2 financial results information" and click the search button. At this time, important information is highlighted, allowing the user to quickly find the information they need.

[0126] User operations

[0127] Users can easily search and browse information through a search interface. If users have further questions about specific information, they can use the online dialogue function. The chatbot receives the question and generates an answer by extracting relevant information from the database. For example, the chatbot will provide an appropriate answer to the question, "What is the outlook for next year?"

[0128] This system allows servers, terminals, and users to seamlessly link together, efficiently collecting, analyzing, and providing IR information on listed companies, allowing investors to quickly obtain more accurate information and make appropriate investment decisions.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Server Processing Steps

[0131] Step 1: Data collection

[0132] The server periodically runs crawlers and APIs to collect information about aggregators from the Internet, downloading video and audio content from the companies' official websites and investor relations platforms. For example, the server launches the crawler at 2:00 AM every day to check for and retrieve new content. The input is a list of URLs, and the output is the collected video and audio files.

[0133] Input: URL list

[0134] Data processing: Collect information using crawlers and APIs

[0135] Output: Video file, audio file

[0136] Step 2: Save your data

[0137] The collected data is stored in cloud storage (e.g., Amazon S3), along with metadata such as the company name, date, and agenda. For example, a file is saved in the format "Company A_2023Q2 Financial Results.mp4," and the associated metadata is recorded in the database.

[0138] Input: Video files, audio files

[0139] Data processing: Extracting metadata and saving files to cloud storage

[0140] Output: Files stored in cloud storage, metadata in a database

[0141] Step 3: Speech to Text

[0142] The saved audio file is sent to the Google Cloud Speech-to-Text API to convert the audio into text. This allows important information from the audio data to be recorded as text. For example, create a processing job called "transcribeJob" and upload the audio file to the API.

[0143] Input: Audio file

[0144] Data calculation: Convert to text information using speech recognition API

[0145] Output: Text information

[0146] Step 4: Keyword extraction

[0147] The generated AI model (e.g., GPT-3) is used to analyze the text and send prompts to extract key identifiable elements and numbers. For example, a prompt such as "Please extract the key points from the financial report" is used.

[0148] Input: Text information, prompt text

[0149] Data computation: Extracting keywords and values ​​using generative AI models

[0150] Output: Main identifiable elements, numbers

[0151] Step 5: Highlighting Information

[0152] Extracted keywords and numerical values ​​are highlighted in the text data, allowing users to check important information at a glance. For example, important numerical items such as "sales" and "net profit" are highlighted in bold.

[0153] Input: Primary identifiable element, number

[0154] Data processing: highlighting key elements

[0155] Output: Highlighted text data

[0156] Step 6: Operating the Database

[0157] The processed text data and metadata are stored in a database and the index is optimized for efficient searching, for example, by adding a full-text search index to improve response times to user search queries.

[0158] Input: Highlighted text data, metadata

[0159] Data processing: Optimizing database storage and indexing

[0160] Output: A searchable database

[0161] Terminal processing steps

[0162] Step 1: Provide a search interface

[0163] The terminal provides a search interface that can be used by users. It is designed to allow users to easily search and browse using web and mobile applications. For example, interfaces built with React or Flutter are provided.

[0164] Input: User's search query

[0165] Data processing: Providing a search interface

[0166] Output: Search results displayed

[0167] Step 2: Providing a dashboard

[0168] The device provides users with a dashboard that displays the latest IR information and information on noteworthy companies. For example, the top page displays "latest financial information" and "information on noteworthy companies" in card format.

[0169] Input: Database information from the server

[0170] Data processing: Dashboard generation and display

[0171] Output: Dashboard

[0172] User operations

[0173] Step 1: Information search

[0174] Users enter the company name and industry information they are interested in into the search box and click the search button. For example, a user searches for "Company A's financial results for Q2 2023."

[0175] Input: Company name and industry information

[0176] Data transport: sending search queries to the server

[0177] Output: Returning search results

[0178] Step 2: View information

[0179] Users can review the provided search results and select details of the information they are particularly interested in. For example, sales and net profits are highlighted, allowing users to quickly grasp key data.

[0180] Input: Search results

[0181] Data processing: highlighting important information

[0182] Output: Understanding the information

[0183] Step 3: Ask a question via chat

[0184] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[0185] Input: User question

[0186] Data processing: Chatbot response generation

[0187] Output: Answer to the question

[0188] As described above, this system can efficiently collect, analyze, and provide IR information on listed companies through specific processing steps and the accompanying data processing and calculations, allowing investors to quickly and accurately obtain the information they need and make appropriate investment decisions.

[0189] (Application example 1)

[0190] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0191] When efficiently collecting, analyzing, and providing investor relations information for listed companies to investors, it is necessary to improve accessibility across a variety of devices. There is also a need for real-time information provision and interactive response functions via voice commands, providing investors with fast and accurate information when making investment decisions. Furthermore, a system that utilizes generative AI models to intelligently extract, highlight, and summarize important keywords and figures is required.

[0192] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0193] In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, means for extracting, highlighting, and summarizing important keywords and figures from the converted text, means for storing the extracted information in a database and making it searchable, means for providing the text data and the extracted information to a user's terminal, means for responding to user questions automatically or with human intervention via an online chat function, and means for displaying IR information in real time on a smart device and responding to user voice commands, thereby enabling investors to collect, analyze, and view information necessary for investment decisions quickly and accurately using a variety of devices.

[0194] A "listed company" is a company that is listed on the stock market and whose shares can be traded by general investors.

[0195] "Internet collection methods" refers to technologies or processes that use web crawlers or APIs to collect information publicly available on the Internet.

[0196] "Means of converting audio / video content into text" refers to the technology or process that converts audio data into text using speech recognition technology.

[0197] "Means for extracting, highlighting, and summarizing important keywords and figures" refers to a technology or process that automatically detects important information from text data and highlights and summarizes it.

[0198] "Means of database storage and retrieval" refers to the technology or process that efficiently stores collected and analyzed data and makes it easily retrievalable.

[0199] "Means for providing data to a user's device" refers to the technology or process for providing data to a user's device, such as a computer or smartphone, using the Internet or other communication means.

[0200] "Online chat function" refers to a function that enables real-time text-based communication using the Internet.

[0201] "Automated or human-assisted response" refers to a process in which the system automatically generates a response, and in which a human intervenes as necessary.

[0202] "Smart devices" refers to devices such as smartphones, smart glasses, and tablets that can connect to the Internet and run a variety of applications.

[0203] "Real-time display means" refers to a technology or process that allows data to be displayed on a user's device almost immediately from the moment it is generated or captured.

[0204] "Means for responding to voice commands" refers to a technique or process by which a system responds to instructions input by voice from a user by taking appropriate action.

[0205] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[0206] Server Processing

[0207] The server has a means of collecting information about listed companies from the Internet. Specifically, it periodically collects video and audio content from financial results briefings and general shareholders' meetings via the Internet using web crawlers and APIs. This collected data is stored in cloud storage such as Google Cloud Storage.

[0208] The server then converts the collected audio data into text using the Google Cloud Speech-to-Text API. The resulting text is then analyzed by a generative AI model (such as BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The extracted information is then stored in a database in an efficient, searchable format.

[0209] Terminal handling

[0210] The device (user's computer or smartphone) provides the user with an easy-to-use search interface through a web or mobile application. The device also provides a dashboard displaying the latest information and investor relations information of noteworthy companies. The device also sends requests to the server based on the search query entered by the user, formats the returned data, and displays it to the user.

[0211] User operations

[0212] Users can search and view information by entering the company name or information of interest in the search box and clicking the search button. For example, if you search for "a company's financial results for Q2 2023," you can quickly view information with a summary and important keywords highlighted from the corresponding text data.

[0213] Users can also check IR information in real time using their smart devices. When searching for specific information using voice commands, the device processes the command and quickly displays the relevant information. For example, if a user says, "Tell me the results for the first half of 2023," the results will instantly be displayed on their smartphone or smart glasses.

[0214] Additionally, if users want to know more, they can use the online chat feature to receive automated or human-assisted responses. For example, they can send a question like, "What's the outlook for next year?" to the chatbot, which will then respond by retrieving relevant information from its database.

[0215] Specific examples

[0216] For example, suppose a video of a company's financial results briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. A speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures such as sales and profits. This data is then stored in a database. When a user searches for "the company's latest financial results" on their device, the database provides the relevant text data, allowing the user to easily check the important information. Furthermore, by using a voice command such as "Tell me the company's financial results for 2023," the information is displayed on the smart glasses.

[0217] Example prompt sentence:

[0218] "Summarize the following string and extract a list of key keywords: 'A company's first-half 2023 earnings call reported a 20% increase in revenue and a 15% increase in profits compared to the previous year. The economic outlook remains uncertain, necessitating a new market strategy.'"

[0219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0220] Step 1:

[0221] The server periodically collects video and audio content from earnings briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using web crawlers and APIs. The collected data is stored in cloud storage along with metadata such as company names, dates, and agendas. The input of this step is publicly available information on the Internet, and the output is raw data stored in cloud storage.

[0222] Step 2:

[0223] The server sends the voice data stored in the cloud storage to the Google Cloud Speech-to-Text API, which converts the voice data into text. The converted data is then saved in a text file or database. The input of this step is the voice data, and the output is text data.

[0224] Step 3:

[0225] The server analyzes the text data using a generative AI model (e.g., BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The results of this analysis are stored in a database. The input for this step is the text data, and the output is the summarized text data and the extracted and highlighted keywords and values.

[0226] Step 4:

[0227] The server indexes the database so that the analysis results can be searched efficiently, allowing users to search data efficiently. The input of this step is the analysis results, and the output is the indexed database.

[0228] Step 5:

[0229] The terminal receives a search query from the user through a user interface and sends the query to the server, which retrieves relevant information from the database, formats it, and returns it to the terminal. The input of this step is the user's search query, and the output is the formatted data received from the server.

[0230] Step 6:

[0231] The terminal displays the search results to the user, who can see important keywords and summaries on the screen. The input of this step is the formatted data, and the output is the search results that are displayed to the user.

[0232] Step 7:

[0233] Users issue voice commands through their smart devices to search for specific IR information in real time. The devices process the voice commands and request related information from the server. The input of this step is the user's voice command, and the output is a request to the server.

[0234] Step 8:

[0235] The server extracts relevant information from a database based on the voice command and returns the formatted information to the terminal, which displays the results in real time on the user's smart device. The input of this step is the server request, and the output is the information displayed on the smart device.

[0236] Step 9:

[0237] The device receives follow-up questions from the user through the online chat function and sends them to the server, which extracts information from the relevant database and responds appropriately. The input for this step is the user's question, and the output is a response from the chatbot or a human agent.

[0238] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0239] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and recognizes user emotions to provide appropriate information and dialogue. This system operates in cooperation with a server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information. Specifically, it is implemented as follows.

[0240] Server Processing

[0241] 1. Data Collection

[0242] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet.

[0243] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[0244] 2. Speech-to-text

[0245] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text, and reprocesses it if necessary.

[0246] 3. Keyword extraction and information emphasis

[0247] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and numbers from the text data and highlight them.

[0248] 4. Introducing the Emotion Engine

[0249] The server incorporates an emotion engine that analyzes user input and recognizes emotions in real time. The emotion engine uses natural language processing techniques to extract user emotions and stores the information in a database.

[0250] 5. Database Operation

[0251] The server stores the processed data in a database, which is indexed and optimized for efficient searching.

[0252] Terminal handling

[0253] 1. Providing a user interface

[0254] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[0255] 2. Search function and information provision

[0256] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[0257] 3. Chat function

[0258] The terminal provides an online chat function, and when a user inputs a question, the content is sent to the server.

[0259] Based on the emotion recognition results, the server generates a response appropriate to the user's emotion and sends it to the device. If the user shows unstable emotions, the server responds appropriately, such as providing a more detailed explanation or encouragement.

[0260] User operations

[0261] 1. Information Search

[0262] Users enter the company name or industry information of interest in the search box and click the search button.

[0263] 2. Viewing Information

[0264] The user reviews the search results provided and selects information of particular interest.

[0265] 3. Chat questions

[0266] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[0267] The device recognizes the user's emotions and provides appropriate answers according to the nuances of the question.

[0268] Specific examples

[0269] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[0270] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Similarly, when a user asks "What's the outlook for next year?" via chat, the emotion engine recognizes the emotion in the user's input and generates and delivers an answer in the appropriate tone. For example, if the user expresses anxiety, the engine can add more detailed explanations or additional information about the context of the data.

[0271] In this way, by linking servers, devices, and users, and by combining the emotion engine, the process of collecting, analyzing, providing, and responding to information becomes more personalized, allowing investors to not only quickly and efficiently obtain appropriate information, but also receive support that reflects their emotions.

[0272] The processing flow will be explained below.

[0273] Server Processing

[0274] Step 1:

[0275] The server runs a crawler to collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The target URLs and API endpoints are set in advance.

[0276] Step 2:

[0277] The server uploads the collected video and audio data to cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[0278] Step 3:

[0279] The server sends the voice data stored in cloud storage to a speech recognition API (e.g., Google Cloud Speech-to-Text API) and converts the voice data into text data. The accuracy of the generated text is checked and reprocessed if necessary.

[0280] Step 4:

[0281] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and values ​​from the text data, highlighting and summarizing the extracted information.

[0282] Step 5:

[0283] The server stores the extracted keywords and summary information in a database, and sets up an index appropriately to enable efficient searches.

[0284] Step 6:

[0285] The server incorporates an emotion engine to analyze user speech and text input in real time, using natural language processing techniques to extract user emotions and store the information in a database.

[0286] Terminal handling

[0287] Step 1:

[0288] The device provides users with an intuitive and easy-to-use search interface through web and mobile applications, where they enter the name of a company or information of interest into a search box.

[0289] Step 2:

[0290] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[0291] Step 3:

[0292] The device receives the search results from the server, formats them in a list or table format, and displays them to the user. The display method changes dynamically depending on the user's emotions.

[0293] Step 4:

[0294] The terminal provides an online chat interface, and when a user types a question, the content is sent to the server. For example, "What is the outlook for next year?"

[0295] User operations

[0296] Step 1:

[0297] Users enter the company name and industry information they are interested in into the search box and click the search button, for example, "XYZ Company's Q2 2023 financial results."

[0298] Step 2:

[0299] Users can review the search results and select and click on the information they are particularly interested in. Detailed information is displayed, and keywords and figures are highlighted, allowing users to quickly grasp the important points.

[0300] Step 3:

[0301] When a user wants to know more about a specific piece of information, they can use the chatbot to enter a question. The emotion engine analyzes the user's input and detects their emotion. For example, they might enter, "I'm worried about the future."

[0302] Step 4:

[0303] The server selects the most appropriate response based on the user's emotions and sends it to the user's device. For example, a user who expresses anxiety may receive detailed data or a message offering reassurance.

[0304] Step 5:

[0305] The user can check the answer displayed on the device and, if necessary, ask the question again to request more detailed information or explanation.

[0306] In this way, by linking the server, terminals, and users, and by combining it with an emotion engine, the information required by users can be efficiently collected and analyzed, and information provided according to individual emotions can be provided, enabling investors to make quick and accurate investment decisions.

[0307] Example 2

[0308] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] Existing information gathering systems have difficulty efficiently collecting and analyzing IR information from listed companies and providing information that meets user needs. Furthermore, they lack the ability to recognize users' emotions and provide appropriate information, making it difficult for them to understand the information and make decisions. This prevents investors from obtaining the information they need at the right time, limiting their ability to make quick decisions.

[0310] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: [means for collecting corporate information from the Internet]; [means for converting collected audio and video content into text]; [means for extracting and highlighting important keywords and numerical values ​​from the converted text]; [means for storing the extracted information in a database and making it searchable]; [means for recognizing user emotions using an emotion engine]; [means for providing information based on the user's emotions]; and [means for automatically responding to questions from users using an online chat function]. This makes it possible to efficiently collect and analyze IR information of listed companies and provide personalized information that takes user emotions into consideration.

[0311] "Company" means an organization that includes a publicly listed corporation and its affiliates.

[0312] "Information" refers to the content of video, audio, text data, etc. related to financial results briefings, general shareholders' meetings, and other official announcements.

[0313] The "Internet" is a communications network that interconnects computer networks around the world and enables the exchange of information.

[0314] "Collection" refers to the act of obtaining information that is publicly available on the Internet using a crawler or API.

[0315] "Audio and video content" refers to data including audio and video media officially released by companies at financial results briefings, general shareholders' meetings, etc.

[0316] "Text conversion" is the process of converting words extracted from audio or video into text data.

[0317] "Keywords" are important words or phrases extracted from text data.

[0318] "Numbers" refer to important statistics and measurements contained in data and information.

[0319] "Highlighting" means displaying extracted keywords and numbers in a special format to make them stand out.

[0320] A "database" is a system that organizes and stores data to facilitate searching and analysis.

[0321] "Searchable" means that information in a database can be quickly found based on specific keywords or criteria.

[0322] An "emotion engine" is a system that uses natural language processing technology to extract and recognize emotions from user input.

[0323] "User" refers to a person who accesses information through a terminal and performs searches or inquiries.

[0324] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0325] "Information provision" refers to the act of displaying collected, processed, and analyzed data on the user's device.

[0326] The "online chat function" is a function that enables real-time conversation with users via the Internet.

[0327] "Automatic response" is a function that automatically provides pre-programmed answers to questions from users.

[0328] This invention is a system that efficiently collects and analyzes corporate IR information, recognizes user emotions, and provides appropriate information and dialogue. This system works in cooperation with the server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information.

[0329] The server first uses crawlers and APIs to periodically collect video and audio content from corporate earnings briefings and general shareholders' meetings that are publicly available on the Internet. Specifically, it searches for videos of listed companies using, for example, the YouTube Data API, and saves the results. The collected data is stored in cloud storage such as Google Cloud Storage, and metadata such as company names, dates, and agendas is stored in a database.

[0330] The server then sends the collected audio data to the Google Cloud Speech-to-Text API, which converts the audio into text. The converted text is then analyzed using a generative AI model (such as BERT or GPT-3) to extract and highlight important keywords and values. The extracted information is then highlighted through tagging and stored in a database.

[0331] The emotion engine is embedded in the server and analyzes user input in real time to recognize emotions. The emotion engine uses natural language processing techniques to extract emotions from user text input and store them in a database.

[0332] The terminal provides users with an easy-to-use search interface through web and mobile applications. For example, frameworks such as React and Flutter are used to build an intuitive and easy-to-use UI. Users can use the search interface to enter the company name or industry information they are interested in and easily find related information.

[0333] When a user enters a search query, the device sends the query to the server and formats and displays the returned data. The display method and content are dynamically changed based on the user's emotions. By determining the user's emotions, for example, if the user expresses anxiety, a more detailed explanation can be provided.

[0334] The device also provides an online chat function, allowing users to input specific questions and send them to the server. The server then uses an emotion engine to recognize the user's emotions, generates appropriate responses, and sends them to the device. This process allows users to obtain specialized information in real time.

[0335] For example, a server can collect videos from the YouTube API using the search keyword "XYZ company earnings briefing" and store them in Google Cloud Storage. The audio from these videos is then converted to text using the Google Cloud Speech-to-Text API, and a generative AI model is used to extract and highlight keywords such as "sales," "year-over-year comparison," and "10% increase." When a user searches for "XYZ company's latest earnings information," the database provides the relevant text data, highlighting the key information.

[0336] As an example of a prompt sentence, if a user inputs "What are the prospects for next year?", the device will send this question to the server, and the emotion engine will recognize the emotion from the user's input. For example, if the user expresses anxiety, detailed explanations and additional information will be provided. By coordinating the server, device, and user, this invention personalizes and streamlines the process of collecting, analyzing, providing, and responding to information.

[0337] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0338] Step 1: Data collection

[0339] The server uses crawlers and APIs to periodically collect video and audio content from companies' financial results briefings and general shareholders' meetings that are publicly available on the Internet. For example, the crawler is used to explore a specific company's YouTube channel, and when new videos are discovered, the links are collected. The collected data includes metadata (company name, date, agenda), which is stored in a database. The input is the company information URL, and the output is the audio and video data and metadata stored in cloud storage.

[0340] Step 2: Speech to text

[0341] The server sends the collected audio data to the Google Cloud Speech-to-Text API and converts the audio to text. For example, send the audio file of a video titled "XYZ Company Earnings Briefing" to the API endpoint to obtain text data. The input is audio data, and the output is text data converted from the audio.

[0342] Step 3: Keyword extraction and information emphasis

[0343] The server uses a generative AI model (such as BERT or GPT-3) to extract important keywords and numbers from text data. For example, it highlights words and numbers such as "sales" and "profit." The input is text data, and the output is text data containing highlighted keywords and numbers.

[0344] Step 4: Store in the database

[0345] The server stores the highlighted text data in a database, indexes it, and makes it searchable. Specifically, it creates a table in an SQL database with fields for "company name," "date," and "keyword" to store the extracted data. The input is the highlighted text data, and the output is a searchable database entry.

[0346] Step 5: Emotion Recognition with the Emotion Engine

[0347] The server analyzes text input from the user using natural language processing technology and recognizes emotions. For example, if a user asks, "What are the prospects for next year?", the emotion engine analyzes the question and generates an emotion label such as "anxiety." The input is the user's text input, and the output is the recognized emotion label.

[0348] Step 6: Providing a User Interface

[0349] The terminal uses frameworks such as React and Flutter to build the user interface, providing a search bar and filter options. Users can search for information by entering the name of a company or industry they are interested in into the search box. The input is the user's search query, and the output is the search results displayed on the interface.

[0350] Step 7: Serving search results

[0351] The device sends the search query entered by the user to the server and formats and displays the returned text data. For example, if you search for "XYZ Company's latest financial results," that data will be displayed on the screen. The input is the search query, and the output is the search results that are displayed to the user.

[0352] Step 8: Online chat function

[0353] When a user uses the online chat function to input a question, the device sends the content to the server. The server uses an emotion engine to analyze the user's emotions, generates the most appropriate answer, and sends it back to the device. For example, if a user asks, "What are the prospects for next year?" and the emotion is determined to be anxiety, a detailed answer will be generated accordingly. The input is the user's question, and the output is a response message.

[0354] This series of steps enables the server, terminal, and user to work together to efficiently collect, analyze, and provide information, enabling optimal support tailored to the user's emotions.

[0355] (Application example 2)

[0356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Currently, there is no system that efficiently collects and analyzes IR information about listed companies and provides users with real-time, personalized information. Furthermore, there are no services that take user sentiment into consideration when collecting and analyzing this information. As a result, investors and other users are unable to quickly and appropriately obtain the information they need, making it difficult to resolve their concerns and questions.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, and means for extracting, highlighting, and summarizing important keywords and figures from the converted text. This enables efficient analysis and storage of the collected information. Furthermore, by including means for recognizing a user's emotions and dynamically changing the method and content of information provision based on the emotions, more user-friendly information can be provided. Furthermore, an online chat function can respond to user questions automatically or with human intervention, helping to alleviate anxiety. Furthermore, by providing additional information and risk explanations during electronic payments based on the user's emotions, a sense of security can be enhanced during investments and transactions.

[0359] "Listed company" refers to a company that is listed on a stock exchange.

[0360] "IR information" refers to the public relations information that companies provide to investors. Generally, this includes financial results briefings, general shareholders' meetings, and information disclosure on official websites.

[0361] "Collecting from the Internet" means automatically obtaining information that is publicly available on the Internet using web crawlers or APIs.

[0362] "Converting audio / video content to text" means converting the audio information in audio or video into text data using a speech recognition API or other technology.

[0363] "Extracting, highlighting, and summarizing keywords and numbers" means using natural language processing technology to select important information from text data and visually highlight or summarize it.

[0364] "Storing in a database and making it searchable" refers to the process of storing the extracted information in a database so that it can be efficiently searched.

[0365] "Recognizing user emotions" means using natural language processing technology or other emotion analysis technology to read emotions from the user's statements and actions.

[0366] The "online chat function" is a function that allows users to ask questions to the system in real time, and the system or a human responds to them.

[0367] "Automated or human-assisted response" refers to a response method that not only provides the user with an automatically generated response from the system, but also includes human operator intervention as needed.

[0368] "Dynamic change of information based on emotion" refers to appropriately changing the display method of information and the content of information provided based on the result of the user's emotion recognition.

[0369] "Providing additional information and risk explanations during electronic payments" means providing additional information and risk explanations during the financial transaction or electronic payment process, taking into consideration the user's feelings.

[0370] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and also realizes personalized information provision and dialogue by recognizing user emotions. This system operates in cooperation with a server, terminals, and users, and by combining an emotion engine in particular, it is possible to provide more accurate information.

[0371] System configuration

[0372] Server Processing

[0373] 1. Data Collection

[0374] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The collected data is then stored in cloud storage.

[0375] 2. Speech-to-text

[0376] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text data, and reprocesses it if necessary.

[0377] 3. Keyword extraction and information emphasis

[0378] The server uses a generative AI model (such as OpenAI's GPT-3) to extract and highlight important keywords and figures from the text data. This analysis uses natural language processing techniques to efficiently extract information.

[0379] 4. Introducing the Emotion Engine

[0380] The server uses an emotion engine (e.g., OpenAI's emotion analysis API) to analyze user input and recognize emotions in real time. This emotion data is also stored in a database.

[0381] 5. Database Operation

[0382] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[0383] Terminal handling

[0384] 1. Providing a user interface

[0385] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[0386] 2. Search function and information provision

[0387] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[0388] 3. Chat function

[0389] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server generates an answer appropriate to the user's emotion based on the emotion recognition results and sends it to the device.

[0390] User operations

[0391] 1. Information Search

[0392] Users enter the company name or industry information of interest in the search box and click the search button.

[0393] 2. Viewing Information

[0394] The user reviews the search results provided and selects information of particular interest.

[0395] 3. Chat questions

[0396] When a user wants to know more about a specific piece of information, they can use the chatbot to input a question, such as "What is the outlook for next year?" The device will recognize the user's emotions and provide an appropriate answer based on the nuances of the question.

[0397] Specific examples

[0398] For example, suppose a video of a company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. This data is then stored in a database. When a user searches for "latest financial information" on their device, the database provides the relevant text data, allowing the user to easily check important information. Similarly, if a user asks "What are the outlook for next year?" via chat, the emotion engine recognizes the emotion from the user's input and generates and delivers an answer in an appropriate tone. Even if the user expresses anxiety, more detailed explanations or supplementary information about the data's context can be added.

[0399] Prompt Sentence Examples

[0400] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[0401] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[0402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0403] The following are specific processing steps of the system that realizes the application example.

[0404] Step 1:

[0405] Data collection

[0406] The server uses a crawler and API to collect video and audio content from listed companies' financial results briefings and general shareholders' meetings from the Internet. Specifically, the crawler accesses specific websites and retrieves the URLs of the video and audio files. The input is the URL set in the crawler, and the output is the collected video and audio files.

[0407] Step 2:

[0408] Speech to text

[0409] The server sends the collected voice data to the Google Cloud Speech-to-Text API and converts it into text data. The input is the collected voice data, and the output is the data converted from voice to text. The server converts the voice data into binary format, sends it to the API, and saves the returned text data.

[0410] Step 3:

[0411] Keyword extraction and information emphasis

[0412] The server uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords and numbers from text data and highlight them. The input is speech-to-text data, and the output is text data with the extracted keywords and numbers highlighted. Specific operations are performed by the server sending prompts to the generative AI model and receiving the analysis results.

[0413] Example prompt sentence:

[0414] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[0415] Step 4:

[0416] Database storage and index optimization

[0417] The server stores the extracted text data, including keywords and numbers, in a database and optimizes the index for efficient searches. The input is text data with keywords and numbers highlighted, and the output is indexed data stored in the database. The specific operations are to insert data into the database and rebuild the index.

[0418] Step 5:

[0419] Introducing the Emotion Engine

[0420] The server uses the emotion engine to analyze user inputs and search queries in real time and recognize emotions. The input is text input from the user, and the output is the emotion recognition result. Specific operations include sending text data to the emotion engine and storing the emotion analysis result in a database.

[0421] Example prompt sentence:

[0422] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[0423] Step 6:

[0424] Providing a user interface

[0425] The terminal provides an easy-to-use search interface through a web application or a mobile application, where the input is a user's search query and the output is parsed data obtained from the server. The specific operation includes the user entering a query into a search box and sending the query to the server.

[0426] Step 7:

[0427] Search function and information provision

[0428] The terminal transmits the search query entered by the user to the server and displays the data returned by the server, where the input is the user's search query and the output is the data returned by the server. Specific operations include formatting the data received from the server and displaying it to the user.

[0429] Step 8:

[0430] Providing chat functionality and emotional responses

[0431] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server uses an emotion engine to analyze the user's emotions and generate and provide an appropriate answer. The input is the user's question and emotional state, and the output is an appropriate answer returned by the server. Specific operations include generating an answer using a generative AI model based on the emotion recognition results and sending it to the device.

[0432] Step 9:

[0433] Providing additional information when making electronic payments

[0434] The server provides additional information and risk explanations during electronic payments based on the user's emotions. The input is the user's payment request and emotional state, and the output is a transaction screen containing additional information and risk explanations. The specific operation is to refer to the emotion recognition results and provide the necessary information to the user.

[0435] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0436] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0437] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0438] [Second embodiment]

[0439] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0440] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0441] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0442] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0443] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0444] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0445] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0446] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0447] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0448] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0449] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0450] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0451] This invention relates to a system that efficiently collects, analyzes, and provides investor relations information of listed companies to investors. This system operates in cooperation with a server, a terminal, and a user, and is specifically implemented as follows.

[0452] Server Processing

[0453] 1. Data Collection

[0454] The server periodically collects video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using crawlers and APIs.

[0455] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[0456] 2. Speech-to-text

[0457] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data.

[0458] Evaluate the quality of the generated text data and manually correct it if necessary.

[0459] 3. Keyword extraction and information emphasis

[0460] The server uses generative AI models (e.g., BERT, GPT-3) to automatically extract important keywords and numbers from text data and highlight them.

[0461] The extracted information is summarized within the server and stored in a database along with associated metadata.

[0462] 4. Database operation

[0463] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[0464] Terminal handling

[0465] 1. Providing a user interface

[0466] The device (user's PC or smartphone) provides the user with an easy-to-use search interface through web applications or mobile applications.

[0467] The device provides a dashboard that displays the latest information and investor relations information for noteworthy companies.

[0468] 2. Performing a search function

[0469] The device sends the search query entered by the user to the server, formats the data returned by the server, and displays it to the user.

[0470] User operations

[0471] 1. Information Search

[0472] Users enter the company name or industry information of interest in the search box and click the search button.

[0473] For example, a user searches for "XYZ Company's Q2 2023 financial results."

[0474] 2. Viewing Information

[0475] The user can review the search results provided and select information of particular interest for further details.

[0476] Search results highlight important keywords and figures, allowing users to quickly grasp the main points.

[0477] 3. Chat questions

[0478] If users want to know more about a particular piece of information, they can use the chatbot to type in a question, such as, "What's the outlook for next year?"

[0479] The chatbot receives the user's question, extracts relevant information from the appropriate database, generates an answer, and displays it again on the device.

[0480] If necessary, a human agent will step in to provide a more detailed answer or interpretation.

[0481] Specific examples

[0482] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[0483] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Furthermore, if a user has a specific question, they can use the chatbot to ask, and relevant information will be returned instantly. This system allows users to quickly and efficiently obtain the information they need to make investment decisions.

[0484] In this way, the server, terminals, and users work together to collect, analyze, provide, and respond to information in a consistent process. This system allows investors to obtain more accurate information quickly and make appropriate investment decisions.

[0485] The processing flow will be explained below.

[0486] Server Processing

[0487] Step 1:

[0488] The server runs a crawler to collect video and audio content of financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet, including on the company's official website and YouTube channel.

[0489] Step 2:

[0490] The server stores the collected video and audio data in cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[0491] Step 3:

[0492] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data. The accuracy of the generated text is checked and reprocessed if necessary.

[0493] Step 4:

[0494] The server extracts important keywords and figures from the text data using a generative AI model that automatically detects and highlights important information that frequently appears in the text.

[0495] Step 5:

[0496] The server highlights the extracted keywords and values ​​and stores the summarized information in a database, which is indexed and optimized for fast and efficient searching.

[0497] Terminal handling

[0498] Step 1:

[0499] The device provides users with an intuitive search interface through web and mobile applications, allowing them to easily find information that interests them.

[0500] Step 2:

[0501] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[0502] Step 3:

[0503] The terminal receives the search results returned from the server, formats the results in a list or table format, and displays them to the user.

[0504] User operations

[0505] Step 1:

[0506] Users enter the company name or industry information of interest in the search box and click the search button.

[0507] Step 2:

[0508] The user can check the list of search results provided, select and click on the information that interests them, and detailed information will be displayed.

[0509] Step 3:

[0510] If a user wants more information about a particular piece of information, they can use the chat interface to type in a question, such as "What's the outlook for next year?"

[0511] Step 4:

[0512] The terminal sends the user's question to the server, which extracts relevant information from a database, generates an appropriate answer, and sends it to the terminal.

[0513] Step 5:

[0514] The user sees the answers displayed in the chat interface and can ask more detailed questions if needed.

[0515] The above processing steps enable effective information collection, analysis, and provision between the server, terminals, and users, allowing investors to obtain the information they need to make investment decisions more quickly and accurately.

[0516] Example 1

[0517] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0518] Currently, it is difficult to efficiently collect, analyze, and provide investor relations information for listed companies to investors. Many challenges exist, particularly in the process of extracting important information from audio and video content and quickly providing it as data necessary for investment decisions. Specifically, there is a need for automation of data collection and conversion, accurate extraction of important information, and provision of information in a format that is easily accessible to users.

[0519] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0520] In this invention, the server includes: [means for collecting information on aggregating entities from a communication network;] [means for converting the collected audio and video into text information; and] [means for extracting, emphasizing, and summarizing key identifiable elements and numerical values ​​from the converted text information.] This makes it possible [to streamline the entire process from collection to analysis and provision, enabling investors to quickly obtain the information they need].

[0521] "Aggregate entities" refer to companies that are listed on the stock market and that disclose information about their financial and operational activities.

[0522] "Communications network" refers to a network for transmitting digital information, including the Internet, which is used to collect, send, and receive data.

[0523] "Audio and video" refers to audio and video content released at corporate financial results briefings and general shareholders' meetings, etc., which includes important information about corporate activities.

[0524] "Text information" refers to text data converted from audio or video, formatted in a format that is easy for humans to read.

[0525] "Key identifiable elements" refer to keywords and numerical information that are recognized as being particularly important in text data and play an important role in investment decisions.

[0526] "Storage" refers to a storage device, such as a database, for storing collected data so that it can be easily accessed at a later time.

[0527] "Information terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to obtain information.

[0528] "Online interactivity" refers to an interface through which a user can input questions or requests to a system, including automated chatbots and human operators.

[0529] "Remote storage" refers to a remotely accessible storage facility, such as cloud storage, that allows large amounts of data to be stored securely and efficiently.

[0530] "Generated AI model" refers to a model that uses artificial intelligence technology to analyze data, thereby automatically extracting key information.

[0531] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[0532] Server Operation

[0533] The server uses multiple hardware and software components to collect, process, and store data. Specifically, it uses crawlers and APIs to gather information from the aggregator across the internet. The collected data is stored as audio and video and stored in cloud storage, such as Amazon S3.

[0534] The server then converts the collected audio and video data into text using the Google Cloud Speech-to-Text API, allowing important information from the video and audio data to be recorded in text format.

[0535] The server then uses the generated AI model (e.g., GPT-3) to analyze the text and send prompts to extract key identifiable elements and numbers, such as "Please extract the key points from the financial report."

[0536] The extracted information is highlighted using bold or color coding and stored in a database, where it is managed using a database system such as MySQL and indexed for efficient searching.

[0537] Device behavior

[0538] The terminal provides an interface for users to obtain information. It is designed to allow users to easily search and browse using web and mobile applications. Specifically, frameworks such as React and Flutter are used to build dashboards and search interfaces.

[0539] When a user enters the name or information of a company of interest in the search box, the device sends the query to the server, formats the data received from the server, and displays it to the user. For example, enter "XYZ Company's 2023 Q2 financial results information" and click the search button. At this time, important information is highlighted, allowing the user to quickly find the information they need.

[0540] User operations

[0541] Users can easily search and browse information through a search interface. If users have further questions about specific information, they can use the online dialogue function. The chatbot receives the question and generates an answer by extracting relevant information from the database. For example, the chatbot will provide an appropriate answer to the question, "What is the outlook for next year?"

[0542] This system allows servers, terminals, and users to seamlessly link together, efficiently collecting, analyzing, and providing IR information on listed companies, allowing investors to quickly obtain more accurate information and make appropriate investment decisions.

[0543] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0544] Server Processing Steps

[0545] Step 1: Data collection

[0546] The server periodically runs crawlers and APIs to collect information about aggregators from the Internet, downloading video and audio content from the companies' official websites and investor relations platforms. For example, the server launches the crawler at 2:00 AM every day to check for and retrieve new content. The input is a list of URLs, and the output is the collected video and audio files.

[0547] Input: URL list

[0548] Data processing: Collect information using crawlers and APIs

[0549] Output: Video file, audio file

[0550] Step 2: Save your data

[0551] The collected data is stored in cloud storage (e.g., Amazon S3), along with metadata such as the company name, date, and agenda. For example, a file is saved in the format "Company A_2023Q2 Financial Results.mp4," and the associated metadata is recorded in the database.

[0552] Input: Video files, audio files

[0553] Data processing: Extracting metadata and saving files to cloud storage

[0554] Output: Files stored in cloud storage, metadata in a database

[0555] Step 3: Speech to Text

[0556] The saved audio file is sent to the Google Cloud Speech-to-Text API to convert the audio into text. This allows important information from the audio data to be recorded as text. For example, create a processing job called "transcribeJob" and upload the audio file to the API.

[0557] Input: Audio file

[0558] Data calculation: Convert to text information using speech recognition API

[0559] Output: Text information

[0560] Step 4: Keyword extraction

[0561] The generated AI model (e.g., GPT-3) is used to analyze the text and send prompts to extract key identifiable elements and numbers. For example, a prompt such as "Please extract the key points from the financial report" is used.

[0562] Input: Text information, prompt text

[0563] Data computation: Extracting keywords and values ​​using generative AI models

[0564] Output: Main identifiable elements, numbers

[0565] Step 5: Highlighting Information

[0566] Extracted keywords and numerical values ​​are highlighted in the text data, allowing users to check important information at a glance. For example, important numerical items such as "sales" and "net profit" are highlighted in bold.

[0567] Input: Primary identifiable element, number

[0568] Data processing: highlighting key elements

[0569] Output: Highlighted text data

[0570] Step 6: Operating the Database

[0571] The processed text data and metadata are stored in a database and the index is optimized for efficient searching, for example, by adding a full-text search index to improve response times to user search queries.

[0572] Input: Highlighted text data, metadata

[0573] Data processing: Optimizing database storage and indexing

[0574] Output: A searchable database

[0575] Terminal processing steps

[0576] Step 1: Provide a search interface

[0577] The terminal provides a search interface that can be used by users. It is designed to allow users to easily search and browse using web and mobile applications. For example, interfaces built with React or Flutter are provided.

[0578] Input: User's search query

[0579] Data processing: Providing a search interface

[0580] Output: Search results displayed

[0581] Step 2: Providing a dashboard

[0582] The device provides users with a dashboard that displays the latest IR information and information on noteworthy companies. For example, the top page displays "latest financial information" and "information on noteworthy companies" in card format.

[0583] Input: Database information from the server

[0584] Data processing: Dashboard generation and display

[0585] Output: Dashboard

[0586] User operations

[0587] Step 1: Information search

[0588] Users enter the company name and industry information they are interested in into the search box and click the search button. For example, a user searches for "Company A's financial results for Q2 2023."

[0589] Input: Company name and industry information

[0590] Data transport: sending search queries to the server

[0591] Output: Returning search results

[0592] Step 2: View information

[0593] Users can review the provided search results and select details of the information they are particularly interested in. For example, sales and net profits are highlighted, allowing users to quickly grasp key data.

[0594] Input: Search results

[0595] Data processing: highlighting important information

[0596] Output: Understanding the information

[0597] Step 3: Ask a question via chat

[0598] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[0599] Input: User question

[0600] Data processing: Chatbot response generation

[0601] Output: Answer to the question

[0602] As described above, this system can efficiently collect, analyze, and provide IR information on listed companies through specific processing steps and the accompanying data processing and calculations, allowing investors to quickly and accurately obtain the information they need and make appropriate investment decisions.

[0603] (Application example 1)

[0604] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0605] When efficiently collecting, analyzing, and providing investor relations information for listed companies to investors, it is necessary to improve accessibility across a variety of devices. There is also a need for real-time information provision and interactive response functions via voice commands, providing investors with fast and accurate information when making investment decisions. Furthermore, a system that utilizes generative AI models to intelligently extract, highlight, and summarize important keywords and figures is required.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0607] In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, means for extracting, highlighting, and summarizing important keywords and figures from the converted text, means for storing the extracted information in a database and making it searchable, means for providing the text data and the extracted information to a user's terminal, means for responding to user questions automatically or with human intervention via an online chat function, and means for displaying IR information in real time on a smart device and responding to user voice commands, thereby enabling investors to collect, analyze, and view information necessary for investment decisions quickly and accurately using a variety of devices.

[0608] A "listed company" is a company that is listed on the stock market and whose shares can be traded by general investors.

[0609] "Internet collection methods" refers to technologies or processes that use web crawlers or APIs to collect information publicly available on the Internet.

[0610] "Means of converting audio / video content into text" refers to the technology or process that converts audio data into text using speech recognition technology.

[0611] "Means for extracting, highlighting, and summarizing important keywords and figures" refers to a technology or process that automatically detects important information from text data and highlights and summarizes it.

[0612] "Means of database storage and retrieval" refers to the technology or process that efficiently stores collected and analyzed data and makes it easily retrievalable.

[0613] "Means for providing data to a user's device" refers to the technology or process for providing data to a user's device, such as a computer or smartphone, using the Internet or other communication means.

[0614] "Online chat function" refers to a function that enables real-time text-based communication using the Internet.

[0615] "Automated or human-assisted response" refers to a process in which the system automatically generates a response, and in which a human intervenes as necessary.

[0616] "Smart devices" refers to devices such as smartphones, smart glasses, and tablets that can connect to the Internet and run a variety of applications.

[0617] "Real-time display means" refers to a technology or process that allows data to be displayed on a user's device almost immediately from the moment it is generated or captured.

[0618] "Means for responding to voice commands" refers to a technique or process by which a system responds to instructions input by voice from a user by taking appropriate action.

[0619] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[0620] Server Processing

[0621] The server has a means of collecting information about listed companies from the Internet. Specifically, it periodically collects video and audio content from financial results briefings and general shareholders' meetings via the Internet using web crawlers and APIs. This collected data is stored in cloud storage such as Google Cloud Storage.

[0622] The server then converts the collected audio data into text using the Google Cloud Speech-to-Text API. The resulting text is then analyzed by a generative AI model (such as BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The extracted information is then stored in a database in an efficient, searchable format.

[0623] Terminal handling

[0624] The device (user's computer or smartphone) provides the user with an easy-to-use search interface through a web or mobile application. The device also provides a dashboard displaying the latest information and investor relations information of noteworthy companies. The device also sends requests to the server based on the search query entered by the user, formats the returned data, and displays it to the user.

[0625] User operations

[0626] Users can search and view information by entering the company name or information of interest in the search box and clicking the search button. For example, if you search for "a company's financial results for Q2 2023," you can quickly view information with a summary and important keywords highlighted from the corresponding text data.

[0627] Users can also check IR information in real time using their smart devices. When searching for specific information using voice commands, the device processes the command and quickly displays the relevant information. For example, if a user says, "Tell me the results for the first half of 2023," the results will instantly be displayed on their smartphone or smart glasses.

[0628] Additionally, if users want to know more, they can use the online chat feature to receive automated or human-assisted responses. For example, they can send a question like, "What's the outlook for next year?" to the chatbot, which will then respond by retrieving relevant information from its database.

[0629] Specific examples

[0630] For example, suppose a video of a company's financial results briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. A speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures such as sales and profits. This data is then stored in a database. When a user searches for "the company's latest financial results" on their device, the database provides the relevant text data, allowing the user to easily check the important information. Furthermore, by using a voice command such as "Tell me the company's financial results for 2023," the information is displayed on the smart glasses.

[0631] Example prompt sentence:

[0632] "Summarize the following string and extract a list of key keywords: 'A company's first-half 2023 earnings call reported a 20% increase in revenue and a 15% increase in profits compared to the previous year. The economic outlook remains uncertain, necessitating a new market strategy.'"

[0633] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0634] Step 1:

[0635] The server periodically collects video and audio content from earnings briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using web crawlers and APIs. The collected data is stored in cloud storage along with metadata such as company names, dates, and agendas. The input of this step is publicly available information on the Internet, and the output is raw data stored in cloud storage.

[0636] Step 2:

[0637] The server sends the voice data stored in the cloud storage to the Google Cloud Speech-to-Text API, which converts the voice data into text. The converted data is then saved in a text file or database. The input of this step is the voice data, and the output is text data.

[0638] Step 3:

[0639] The server analyzes the text data using a generative AI model (e.g., BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The results of this analysis are stored in a database. The input for this step is the text data, and the output is the summarized text data and the extracted and highlighted keywords and values.

[0640] Step 4:

[0641] The server indexes the database so that the analysis results can be searched efficiently, allowing users to search data efficiently. The input of this step is the analysis results, and the output is the indexed database.

[0642] Step 5:

[0643] The terminal receives a search query from the user through a user interface and sends the query to the server, which retrieves relevant information from the database, formats it, and returns it to the terminal. The input of this step is the user's search query, and the output is the formatted data received from the server.

[0644] Step 6:

[0645] The terminal displays the search results to the user, who can see important keywords and summaries on the screen. The input of this step is the formatted data, and the output is the search results that are displayed to the user.

[0646] Step 7:

[0647] Users issue voice commands through their smart devices to search for specific IR information in real time. The devices process the voice commands and request related information from the server. The input of this step is the user's voice command, and the output is a request to the server.

[0648] Step 8:

[0649] The server extracts relevant information from a database based on the voice command and returns the formatted information to the terminal, which displays the results in real time on the user's smart device. The input of this step is the server request, and the output is the information displayed on the smart device.

[0650] Step 9:

[0651] The device receives follow-up questions from the user through the online chat function and sends them to the server, which extracts information from the relevant database and responds appropriately. The input for this step is the user's question, and the output is a response from the chatbot or a human agent.

[0652] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0653] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and recognizes user emotions to provide appropriate information and dialogue. This system operates in cooperation with a server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information. Specifically, it is implemented as follows.

[0654] Server Processing

[0655] 1. Data Collection

[0656] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet.

[0657] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[0658] 2. Speech-to-text

[0659] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text, and reprocesses it if necessary.

[0660] 3. Keyword extraction and information emphasis

[0661] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and numbers from the text data and highlight them.

[0662] 4. Introducing the Emotion Engine

[0663] The server incorporates an emotion engine that analyzes user input and recognizes emotions in real time. The emotion engine uses natural language processing techniques to extract user emotions and stores the information in a database.

[0664] 5. Database Operation

[0665] The server stores the processed data in a database, which is indexed and optimized for efficient searching.

[0666] Terminal handling

[0667] 1. Providing a user interface

[0668] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[0669] 2. Search function and information provision

[0670] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[0671] 3. Chat function

[0672] The terminal provides an online chat function, and when a user inputs a question, the content is sent to the server.

[0673] Based on the emotion recognition results, the server generates a response appropriate to the user's emotion and sends it to the device. If the user shows unstable emotions, the server responds appropriately, such as providing a more detailed explanation or encouragement.

[0674] User operations

[0675] 1. Information Search

[0676] Users enter the company name or industry information of interest in the search box and click the search button.

[0677] 2. Viewing Information

[0678] The user reviews the search results provided and selects information of particular interest.

[0679] 3. Chat questions

[0680] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[0681] The device recognizes the user's emotions and provides appropriate answers according to the nuances of the question.

[0682] Specific examples

[0683] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[0684] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Similarly, when a user asks "What's the outlook for next year?" via chat, the emotion engine recognizes the emotion in the user's input and generates and delivers an answer in the appropriate tone. For example, if the user expresses anxiety, the engine can add more detailed explanations or additional information about the context of the data.

[0685] In this way, by linking servers, devices, and users, and by combining the emotion engine, the process of collecting, analyzing, providing, and responding to information becomes more personalized, allowing investors to not only quickly and efficiently obtain appropriate information, but also receive support that reflects their emotions.

[0686] The processing flow will be explained below.

[0687] Server Processing

[0688] Step 1:

[0689] The server runs a crawler to collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The target URLs and API endpoints are set in advance.

[0690] Step 2:

[0691] The server uploads the collected video and audio data to cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[0692] Step 3:

[0693] The server sends the voice data stored in cloud storage to a speech recognition API (e.g., Google Cloud Speech-to-Text API) and converts the voice data into text data. The accuracy of the generated text is checked and reprocessed if necessary.

[0694] Step 4:

[0695] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and values ​​from the text data, highlighting and summarizing the extracted information.

[0696] Step 5:

[0697] The server stores the extracted keywords and summary information in a database, and sets up an index appropriately to enable efficient searches.

[0698] Step 6:

[0699] The server incorporates an emotion engine to analyze user speech and text input in real time, using natural language processing techniques to extract user emotions and store the information in a database.

[0700] Terminal handling

[0701] Step 1:

[0702] The device provides users with an intuitive and easy-to-use search interface through web and mobile applications, where they enter the name of a company or information of interest into a search box.

[0703] Step 2:

[0704] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[0705] Step 3:

[0706] The device receives the search results from the server, formats them in a list or table format, and displays them to the user. The display method changes dynamically depending on the user's emotions.

[0707] Step 4:

[0708] The terminal provides an online chat interface, and when a user types a question, the content is sent to the server. For example, "What is the outlook for next year?"

[0709] User operations

[0710] Step 1:

[0711] Users enter the company name and industry information they are interested in into the search box and click the search button, for example, "XYZ Company's Q2 2023 financial results."

[0712] Step 2:

[0713] Users can review the search results and select and click on the information they are particularly interested in. Detailed information is displayed, and keywords and figures are highlighted, allowing users to quickly grasp the important points.

[0714] Step 3:

[0715] When a user wants to know more about a specific piece of information, they can use the chatbot to enter a question. The emotion engine analyzes the user's input and detects their emotion. For example, they might enter, "I'm worried about the future."

[0716] Step 4:

[0717] The server selects the most appropriate response based on the user's emotions and sends it to the user's device. For example, a user who expresses anxiety may receive detailed data or a message offering reassurance.

[0718] Step 5:

[0719] The user can check the answer displayed on the device and, if necessary, ask the question again to request more detailed information or explanation.

[0720] In this way, by linking the server, terminals, and users, and by combining it with an emotion engine, the information required by users can be efficiently collected and analyzed, and information provided according to individual emotions can be provided, enabling investors to make quick and accurate investment decisions.

[0721] Example 2

[0722] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0723] Existing information gathering systems have difficulty efficiently collecting and analyzing IR information from listed companies and providing information that meets user needs. Furthermore, they lack the ability to recognize users' emotions and provide appropriate information, making it difficult for them to understand the information and make decisions. This prevents investors from obtaining the information they need at the right time, limiting their ability to make quick decisions.

[0724] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: [means for collecting corporate information from the Internet]; [means for converting collected audio and video content into text]; [means for extracting and highlighting important keywords and numerical values ​​from the converted text]; [means for storing the extracted information in a database and making it searchable]; [means for recognizing user emotions using an emotion engine]; [means for providing information based on the user's emotions]; and [means for automatically responding to questions from users using an online chat function]. This makes it possible to efficiently collect and analyze IR information of listed companies and provide personalized information that takes user emotions into consideration.

[0725] "Company" means an organization that includes a publicly listed corporation and its affiliates.

[0726] "Information" refers to the content of video, audio, text data, etc. related to financial results briefings, general shareholders' meetings, and other official announcements.

[0727] The "Internet" is a communications network that interconnects computer networks around the world and enables the exchange of information.

[0728] "Collection" refers to the act of obtaining information that is publicly available on the Internet using a crawler or API.

[0729] "Audio and video content" refers to data including audio and video media officially released by companies at financial results briefings, general shareholders' meetings, etc.

[0730] "Text conversion" is the process of converting words extracted from audio or video into text data.

[0731] "Keywords" are important words or phrases extracted from text data.

[0732] "Numbers" refer to important statistics and measurements contained in data and information.

[0733] "Highlighting" means displaying extracted keywords and numbers in a special format to make them stand out.

[0734] A "database" is a system that organizes and stores data to facilitate searching and analysis.

[0735] "Searchable" means that information in a database can be quickly found based on specific keywords or criteria.

[0736] An "emotion engine" is a system that uses natural language processing technology to extract and recognize emotions from user input.

[0737] "User" refers to a person who accesses information through a terminal and performs searches or inquiries.

[0738] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[0739] "Information provision" refers to the act of displaying collected, processed, and analyzed data on the user's device.

[0740] The "online chat function" is a function that enables real-time conversation with users via the Internet.

[0741] "Automatic response" is a function that automatically provides pre-programmed answers to questions from users.

[0742] This invention is a system that efficiently collects and analyzes corporate IR information, recognizes user emotions, and provides appropriate information and dialogue. This system works in cooperation with the server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information.

[0743] The server first uses crawlers and APIs to periodically collect video and audio content from corporate earnings briefings and general shareholders' meetings that are publicly available on the Internet. Specifically, it searches for videos of listed companies using, for example, the YouTube Data API, and saves the results. The collected data is stored in cloud storage such as Google Cloud Storage, and metadata such as company names, dates, and agendas is stored in a database.

[0744] The server then sends the collected audio data to the Google Cloud Speech-to-Text API, which converts the audio into text. The converted text is then analyzed using a generative AI model (such as BERT or GPT-3) to extract and highlight important keywords and values. The extracted information is then highlighted through tagging and stored in a database.

[0745] The emotion engine is embedded in the server and analyzes user input in real time to recognize emotions. The emotion engine uses natural language processing techniques to extract emotions from user text input and store them in a database.

[0746] The terminal provides users with an easy-to-use search interface through web and mobile applications. For example, frameworks such as React and Flutter are used to build an intuitive and easy-to-use UI. Users can use the search interface to enter the company name or industry information they are interested in and easily find related information.

[0747] When a user enters a search query, the device sends the query to the server and formats and displays the returned data. The display method and content are dynamically changed based on the user's emotions. By determining the user's emotions, for example, if the user expresses anxiety, a more detailed explanation can be provided.

[0748] The device also provides an online chat function, allowing users to input specific questions and send them to the server. The server then uses an emotion engine to recognize the user's emotions, generates appropriate responses, and sends them to the device. This process allows users to obtain specialized information in real time.

[0749] For example, a server can collect videos from the YouTube API using the search keyword "XYZ company earnings briefing" and store them in Google Cloud Storage. The audio from these videos is then converted to text using the Google Cloud Speech-to-Text API, and a generative AI model is used to extract and highlight keywords such as "sales," "year-over-year comparison," and "10% increase." When a user searches for "XYZ company's latest earnings information," the database provides the relevant text data, highlighting the key information.

[0750] As an example of a prompt sentence, if a user inputs "What are the prospects for next year?", the device will send this question to the server, and the emotion engine will recognize the emotion from the user's input. For example, if the user expresses anxiety, detailed explanations and additional information will be provided. By coordinating the server, device, and user, this invention personalizes and streamlines the process of collecting, analyzing, providing, and responding to information.

[0751] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0752] Step 1: Data collection

[0753] The server uses crawlers and APIs to periodically collect video and audio content from companies' financial results briefings and general shareholders' meetings that are publicly available on the Internet. For example, the crawler is used to explore a specific company's YouTube channel, and when new videos are discovered, the links are collected. The collected data includes metadata (company name, date, agenda), which is stored in a database. The input is the company information URL, and the output is the audio and video data and metadata stored in cloud storage.

[0754] Step 2: Speech to text

[0755] The server sends the collected audio data to the Google Cloud Speech-to-Text API and converts the audio to text. For example, send the audio file of a video titled "XYZ Company Earnings Briefing" to the API endpoint to obtain text data. The input is audio data, and the output is text data converted from the audio.

[0756] Step 3: Keyword extraction and information emphasis

[0757] The server uses a generative AI model (such as BERT or GPT-3) to extract important keywords and numbers from text data. For example, it highlights words and numbers such as "sales" and "profit." The input is text data, and the output is text data containing highlighted keywords and numbers.

[0758] Step 4: Store in the database

[0759] The server stores the highlighted text data in a database, indexes it, and makes it searchable. Specifically, it creates a table in an SQL database with fields for "company name," "date," and "keyword" to store the extracted data. The input is the highlighted text data, and the output is a searchable database entry.

[0760] Step 5: Emotion Recognition with the Emotion Engine

[0761] The server analyzes text input from the user using natural language processing technology and recognizes emotions. For example, if a user asks, "What are the prospects for next year?", the emotion engine analyzes the question and generates an emotion label such as "anxiety." The input is the user's text input, and the output is the recognized emotion label.

[0762] Step 6: Providing a User Interface

[0763] The terminal uses frameworks such as React and Flutter to build the user interface, providing a search bar and filter options. Users can search for information by entering the name of a company or industry they are interested in into the search box. The input is the user's search query, and the output is the search results displayed on the interface.

[0764] Step 7: Serving search results

[0765] The device sends the search query entered by the user to the server and formats and displays the returned text data. For example, if you search for "XYZ Company's latest financial results," that data will be displayed on the screen. The input is the search query, and the output is the search results that are displayed to the user.

[0766] Step 8: Online chat function

[0767] When a user uses the online chat function to input a question, the device sends the content to the server. The server uses an emotion engine to analyze the user's emotions, generates the most appropriate answer, and sends it back to the device. For example, if a user asks, "What are the prospects for next year?" and the emotion is determined to be anxiety, a detailed answer will be generated accordingly. The input is the user's question, and the output is a response message.

[0768] This series of steps enables the server, terminal, and user to work together to efficiently collect, analyze, and provide information, enabling optimal support tailored to the user's emotions.

[0769] (Application example 2)

[0770] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0771] Currently, there is no system that efficiently collects and analyzes IR information about listed companies and provides users with real-time, personalized information. Furthermore, there are no services that take user sentiment into consideration when collecting and analyzing this information. As a result, investors and other users are unable to quickly and appropriately obtain the information they need, making it difficult to resolve their concerns and questions.

[0772] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, and means for extracting, highlighting, and summarizing important keywords and figures from the converted text. This enables efficient analysis and storage of the collected information. Furthermore, by including means for recognizing a user's emotions and dynamically changing the method and content of information provision based on the emotions, more user-friendly information can be provided. Furthermore, an online chat function can respond to user questions automatically or with human intervention, helping to alleviate anxiety. Furthermore, by providing additional information and risk explanations during electronic payments based on the user's emotions, a sense of security can be enhanced during investments and transactions.

[0773] "Listed company" refers to a company that is listed on a stock exchange.

[0774] "IR information" refers to the public relations information that companies provide to investors. Generally, this includes financial results briefings, general shareholders' meetings, and information disclosure on official websites.

[0775] "Collecting from the Internet" means automatically obtaining information that is publicly available on the Internet using web crawlers or APIs.

[0776] "Converting audio / video content to text" means converting the audio information in audio or video into text data using a speech recognition API or other technology.

[0777] "Extracting, highlighting, and summarizing keywords and numbers" means using natural language processing technology to select important information from text data and visually highlight or summarize it.

[0778] "Storing in a database and making it searchable" refers to the process of storing the extracted information in a database so that it can be efficiently searched.

[0779] "Recognizing user emotions" means using natural language processing technology or other emotion analysis technology to read emotions from the user's statements and actions.

[0780] The "online chat function" is a function that allows users to ask questions to the system in real time, and the system or a human responds to them.

[0781] "Automated or human-assisted response" refers to a response method that not only provides the user with an automatically generated response from the system, but also includes human operator intervention as needed.

[0782] "Dynamic change of information based on emotion" refers to appropriately changing the display method of information and the content of information provided based on the result of the user's emotion recognition.

[0783] "Providing additional information and risk explanations during electronic payments" means providing additional information and risk explanations during the financial transaction or electronic payment process, taking into consideration the user's feelings.

[0784] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and also realizes personalized information provision and dialogue by recognizing user emotions. This system operates in cooperation with a server, terminals, and users, and by combining an emotion engine in particular, it is possible to provide more accurate information.

[0785] System configuration

[0786] Server Processing

[0787] 1. Data Collection

[0788] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The collected data is then stored in cloud storage.

[0789] 2. Speech-to-text

[0790] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text data, and reprocesses it if necessary.

[0791] 3. Keyword extraction and information emphasis

[0792] The server uses a generative AI model (such as OpenAI's GPT-3) to extract and highlight important keywords and figures from the text data. This analysis uses natural language processing techniques to efficiently extract information.

[0793] 4. Introducing the Emotion Engine

[0794] The server uses an emotion engine (e.g., OpenAI's emotion analysis API) to analyze user input and recognize emotions in real time. This emotion data is also stored in a database.

[0795] 5. Database Operation

[0796] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[0797] Terminal handling

[0798] 1. Providing a user interface

[0799] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[0800] 2. Search function and information provision

[0801] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[0802] 3. Chat function

[0803] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server generates an answer appropriate to the user's emotion based on the emotion recognition results and sends it to the device.

[0804] User operations

[0805] 1. Information Search

[0806] Users enter the company name or industry information of interest in the search box and click the search button.

[0807] 2. Viewing Information

[0808] The user reviews the search results provided and selects information of particular interest.

[0809] 3. Chat questions

[0810] When a user wants to know more about a specific piece of information, they can use the chatbot to input a question, such as "What is the outlook for next year?" The device will recognize the user's emotions and provide an appropriate answer based on the nuances of the question.

[0811] Specific examples

[0812] For example, suppose a video of a company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. This data is then stored in a database. When a user searches for "latest financial information" on their device, the database provides the relevant text data, allowing the user to easily check important information. Similarly, if a user asks "What are the outlook for next year?" via chat, the emotion engine recognizes the emotion from the user's input and generates and delivers an answer in an appropriate tone. Even if the user expresses anxiety, more detailed explanations or supplementary information about the data's context can be added.

[0813] Prompt Sentence Examples

[0814] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[0815] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[0816] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0817] The following are specific processing steps of the system that realizes the application example.

[0818] Step 1:

[0819] Data collection

[0820] The server uses a crawler and API to collect video and audio content from listed companies' financial results briefings and general shareholders' meetings from the Internet. Specifically, the crawler accesses specific websites and retrieves the URLs of the video and audio files. The input is the URL set in the crawler, and the output is the collected video and audio files.

[0821] Step 2:

[0822] Speech to text

[0823] The server sends the collected voice data to the Google Cloud Speech-to-Text API and converts it into text data. The input is the collected voice data, and the output is the data converted from voice to text. The server converts the voice data into binary format, sends it to the API, and saves the returned text data.

[0824] Step 3:

[0825] Keyword extraction and information emphasis

[0826] The server uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords and numbers from text data and highlight them. The input is speech-to-text data, and the output is text data with the extracted keywords and numbers highlighted. Specific operations are performed by the server sending prompts to the generative AI model and receiving the analysis results.

[0827] Example prompt sentence:

[0828] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[0829] Step 4:

[0830] Database storage and index optimization

[0831] The server stores the extracted text data, including keywords and numbers, in a database and optimizes the index for efficient searches. The input is text data with keywords and numbers highlighted, and the output is indexed data stored in the database. The specific operations are to insert data into the database and rebuild the index.

[0832] Step 5:

[0833] Introducing the Emotion Engine

[0834] The server uses the emotion engine to analyze user inputs and search queries in real time and recognize emotions. The input is text input from the user, and the output is the emotion recognition result. Specific operations include sending text data to the emotion engine and storing the emotion analysis result in a database.

[0835] Example prompt sentence:

[0836] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[0837] Step 6:

[0838] Providing a user interface

[0839] The terminal provides an easy-to-use search interface through a web application or a mobile application, where the input is a user's search query and the output is parsed data obtained from the server. The specific operation includes the user entering a query into a search box and sending the query to the server.

[0840] Step 7:

[0841] Search function and information provision

[0842] The terminal transmits the search query entered by the user to the server and displays the data returned by the server, where the input is the user's search query and the output is the data returned by the server. Specific operations include formatting the data received from the server and displaying it to the user.

[0843] Step 8:

[0844] Providing chat functionality and emotional responses

[0845] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server uses an emotion engine to analyze the user's emotions and generate and provide an appropriate answer. The input is the user's question and emotional state, and the output is an appropriate answer returned by the server. Specific operations include generating an answer using a generative AI model based on the emotion recognition results and sending it to the device.

[0846] Step 9:

[0847] Providing additional information when making electronic payments

[0848] The server provides additional information and risk explanations during electronic payments based on the user's emotions. The input is the user's payment request and emotional state, and the output is a transaction screen containing additional information and risk explanations. The specific operation is to refer to the emotion recognition results and provide the necessary information to the user.

[0849] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0850] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0851] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0852] [Third embodiment]

[0853] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0854] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0855] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0856] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0857] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0858] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0859] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0860] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0861] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0862] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0863] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0864] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0865] This invention relates to a system that efficiently collects, analyzes, and provides investor relations information of listed companies to investors. This system operates in cooperation with a server, a terminal, and a user, and is specifically implemented as follows.

[0866] Server Processing

[0867] 1. Data Collection

[0868] The server periodically collects video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using crawlers and APIs.

[0869] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[0870] 2. Speech-to-text

[0871] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data.

[0872] Evaluate the quality of the generated text data and manually correct it if necessary.

[0873] 3. Keyword extraction and information emphasis

[0874] The server uses generative AI models (e.g., BERT, GPT-3) to automatically extract important keywords and numbers from text data and highlight them.

[0875] The extracted information is summarized within the server and stored in a database along with associated metadata.

[0876] 4. Database operation

[0877] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[0878] Terminal handling

[0879] 1. Providing a user interface

[0880] The device (user's PC or smartphone) provides the user with an easy-to-use search interface through web applications or mobile applications.

[0881] The device provides a dashboard that displays the latest information and investor relations information for noteworthy companies.

[0882] 2. Performing a search function

[0883] The device sends the search query entered by the user to the server, formats the data returned by the server, and displays it to the user.

[0884] User operations

[0885] 1. Information Search

[0886] Users enter the company name or industry information of interest in the search box and click the search button.

[0887] For example, a user searches for "XYZ Company's Q2 2023 financial results."

[0888] 2. Viewing Information

[0889] The user can review the search results provided and select information of particular interest for further details.

[0890] Search results highlight important keywords and figures, allowing users to quickly grasp the main points.

[0891] 3. Chat questions

[0892] If users want to know more about a particular piece of information, they can use the chatbot to type in a question, such as, "What's the outlook for next year?"

[0893] The chatbot receives the user's question, extracts relevant information from the appropriate database, generates an answer, and displays it again on the device.

[0894] If necessary, a human agent will step in to provide a more detailed answer or interpretation.

[0895] Specific examples

[0896] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[0897] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Furthermore, if a user has a specific question, they can use the chatbot to ask, and relevant information will be returned instantly. This system allows users to quickly and efficiently obtain the information they need to make investment decisions.

[0898] In this way, the server, terminals, and users work together to collect, analyze, provide, and respond to information in a consistent process. This system allows investors to obtain more accurate information quickly and make appropriate investment decisions.

[0899] The processing flow will be explained below.

[0900] Server Processing

[0901] Step 1:

[0902] The server runs a crawler to collect video and audio content of financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet, including on the company's official website and YouTube channel.

[0903] Step 2:

[0904] The server stores the collected video and audio data in cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[0905] Step 3:

[0906] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data. The accuracy of the generated text is checked and reprocessed if necessary.

[0907] Step 4:

[0908] The server extracts important keywords and figures from the text data using a generative AI model that automatically detects and highlights important information that frequently appears in the text.

[0909] Step 5:

[0910] The server highlights the extracted keywords and values ​​and stores the summarized information in a database, which is indexed and optimized for fast and efficient searching.

[0911] Terminal handling

[0912] Step 1:

[0913] The device provides users with an intuitive search interface through web and mobile applications, allowing them to easily find information that interests them.

[0914] Step 2:

[0915] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[0916] Step 3:

[0917] The terminal receives the search results returned from the server, formats the results in a list or table format, and displays them to the user.

[0918] User operations

[0919] Step 1:

[0920] Users enter the company name or industry information of interest in the search box and click the search button.

[0921] Step 2:

[0922] The user can check the list of search results provided, select and click on the information that interests them, and detailed information will be displayed.

[0923] Step 3:

[0924] If a user wants more information about a particular piece of information, they can use the chat interface to type in a question, such as "What's the outlook for next year?"

[0925] Step 4:

[0926] The terminal sends the user's question to the server, which extracts relevant information from a database, generates an appropriate answer, and sends it to the terminal.

[0927] Step 5:

[0928] The user sees the answers displayed in the chat interface and can ask more detailed questions if needed.

[0929] The above processing steps enable effective information collection, analysis, and provision between the server, terminals, and users, allowing investors to obtain the information they need to make investment decisions more quickly and accurately.

[0930] Example 1

[0931] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0932] Currently, it is difficult to efficiently collect, analyze, and provide investor relations information for listed companies to investors. Many challenges exist, particularly in the process of extracting important information from audio and video content and quickly providing it as data necessary for investment decisions. Specifically, there is a need for automation of data collection and conversion, accurate extraction of important information, and provision of information in a format that is easily accessible to users.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0934] In this invention, the server includes: [means for collecting information on aggregating entities from a communication network;] [means for converting the collected audio and video into text information; and] [means for extracting, emphasizing, and summarizing key identifiable elements and numerical values ​​from the converted text information.] This makes it possible [to streamline the entire process from collection to analysis and provision, enabling investors to quickly obtain the information they need].

[0935] "Aggregate entities" refer to companies that are listed on the stock market and that disclose information about their financial and operational activities.

[0936] "Communications network" refers to a network for transmitting digital information, including the Internet, which is used to collect, send, and receive data.

[0937] "Audio and video" refers to audio and video content released at corporate financial results briefings and general shareholders' meetings, etc., which includes important information about corporate activities.

[0938] "Text information" refers to text data converted from audio or video, formatted in a format that is easy for humans to read.

[0939] "Key identifiable elements" refer to keywords and numerical information that are recognized as being particularly important in text data and play an important role in investment decisions.

[0940] "Storage" refers to a storage device, such as a database, for storing collected data so that it can be easily accessed at a later time.

[0941] "Information terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to obtain information.

[0942] "Online interactivity" refers to an interface through which a user can input questions or requests to a system, including automated chatbots and human operators.

[0943] "Remote storage" refers to a remotely accessible storage facility, such as cloud storage, that allows large amounts of data to be stored securely and efficiently.

[0944] "Generated AI model" refers to a model that uses artificial intelligence technology to analyze data, thereby automatically extracting key information.

[0945] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[0946] Server Operation

[0947] The server uses multiple hardware and software components to collect, process, and store data. Specifically, it uses crawlers and APIs to gather information from the aggregator across the internet. The collected data is stored as audio and video and stored in cloud storage, such as Amazon S3.

[0948] The server then converts the collected audio and video data into text using the Google Cloud Speech-to-Text API, allowing important information from the video and audio data to be recorded in text format.

[0949] The server then uses the generated AI model (e.g., GPT-3) to analyze the text and send prompts to extract key identifiable elements and numbers, such as "Please extract the key points from the financial report."

[0950] The extracted information is highlighted using bold or color coding and stored in a database, where it is managed using a database system such as MySQL and indexed for efficient searching.

[0951] Device behavior

[0952] The terminal provides an interface for users to obtain information. It is designed to allow users to easily search and browse using web and mobile applications. Specifically, frameworks such as React and Flutter are used to build dashboards and search interfaces.

[0953] When a user enters the name or information of a company of interest in the search box, the device sends the query to the server, formats the data received from the server, and displays it to the user. For example, enter "XYZ Company's 2023 Q2 financial results information" and click the search button. At this time, important information is highlighted, allowing the user to quickly find the information they need.

[0954] User operations

[0955] Users can easily search and browse information through a search interface. If users have further questions about specific information, they can use the online dialogue function. The chatbot receives the question and generates an answer by extracting relevant information from the database. For example, the chatbot will provide an appropriate answer to the question, "What is the outlook for next year?"

[0956] This system allows servers, terminals, and users to seamlessly link together, efficiently collecting, analyzing, and providing IR information on listed companies, allowing investors to quickly obtain more accurate information and make appropriate investment decisions.

[0957] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0958] Server Processing Steps

[0959] Step 1: Data collection

[0960] The server periodically runs crawlers and APIs to collect information about aggregators from the Internet, downloading video and audio content from the companies' official websites and investor relations platforms. For example, the server launches the crawler at 2:00 AM every day to check for and retrieve new content. The input is a list of URLs, and the output is the collected video and audio files.

[0961] Input: URL list

[0962] Data processing: Collect information using crawlers and APIs

[0963] Output: Video file, audio file

[0964] Step 2: Save your data

[0965] The collected data is stored in cloud storage (e.g., Amazon S3), along with metadata such as the company name, date, and agenda. For example, a file is saved in the format "Company A_2023Q2 Financial Results.mp4," and the associated metadata is recorded in the database.

[0966] Input: Video files, audio files

[0967] Data processing: Extracting metadata and saving files to cloud storage

[0968] Output: Files stored in cloud storage, metadata in a database

[0969] Step 3: Speech to Text

[0970] The saved audio file is sent to the Google Cloud Speech-to-Text API to convert the audio into text. This allows important information from the audio data to be recorded as text. For example, create a processing job called "transcribeJob" and upload the audio file to the API.

[0971] Input: Audio file

[0972] Data calculation: Convert to text information using speech recognition API

[0973] Output: Text information

[0974] Step 4: Keyword extraction

[0975] The generated AI model (e.g., GPT-3) is used to analyze the text and send prompts to extract key identifiable elements and numbers. For example, a prompt such as "Please extract the key points from the financial report" is used.

[0976] Input: Text information, prompt text

[0977] Data computation: Extracting keywords and values ​​using generative AI models

[0978] Output: Main identifiable elements, numbers

[0979] Step 5: Highlighting Information

[0980] Extracted keywords and numerical values ​​are highlighted in the text data, allowing users to check important information at a glance. For example, important numerical items such as "sales" and "net profit" are highlighted in bold.

[0981] Input: Primary identifiable element, number

[0982] Data processing: highlighting key elements

[0983] Output: Highlighted text data

[0984] Step 6: Operating the Database

[0985] The processed text data and metadata are stored in a database and the index is optimized for efficient searching, for example, by adding a full-text search index to improve response times to user search queries.

[0986] Input: Highlighted text data, metadata

[0987] Data processing: Optimizing database storage and indexing

[0988] Output: A searchable database

[0989] Terminal processing steps

[0990] Step 1: Provide a search interface

[0991] The terminal provides a search interface that can be used by users. It is designed to allow users to easily search and browse using web and mobile applications. For example, interfaces built with React or Flutter are provided.

[0992] Input: User's search query

[0993] Data processing: Providing a search interface

[0994] Output: Search results displayed

[0995] Step 2: Providing a dashboard

[0996] The device provides users with a dashboard that displays the latest IR information and information on noteworthy companies. For example, the top page displays "latest financial information" and "information on noteworthy companies" in card format.

[0997] Input: Database information from the server

[0998] Data processing: Dashboard generation and display

[0999] Output: Dashboard

[1000] User operations

[1001] Step 1: Information search

[1002] Users enter the company name and industry information they are interested in into the search box and click the search button. For example, a user searches for "Company A's financial results for Q2 2023."

[1003] Input: Company name and industry information

[1004] Data transport: sending search queries to the server

[1005] Output: Returning search results

[1006] Step 2: View information

[1007] Users can review the provided search results and select details of the information they are particularly interested in. For example, sales and net profits are highlighted, allowing users to quickly grasp key data.

[1008] Input: Search results

[1009] Data processing: highlighting important information

[1010] Output: Understanding the information

[1011] Step 3: Ask a question via chat

[1012] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[1013] Input: User question

[1014] Data processing: Chatbot response generation

[1015] Output: Answer to the question

[1016] As described above, this system can efficiently collect, analyze, and provide IR information on listed companies through specific processing steps and the accompanying data processing and calculations, allowing investors to quickly and accurately obtain the information they need and make appropriate investment decisions.

[1017] (Application example 1)

[1018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] When efficiently collecting, analyzing, and providing investor relations information for listed companies to investors, it is necessary to improve accessibility across a variety of devices. There is also a need for real-time information provision and interactive response functions via voice commands, providing investors with fast and accurate information when making investment decisions. Furthermore, a system that utilizes generative AI models to intelligently extract, highlight, and summarize important keywords and figures is required.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1021] In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, means for extracting, highlighting, and summarizing important keywords and figures from the converted text, means for storing the extracted information in a database and making it searchable, means for providing the text data and the extracted information to a user's terminal, means for responding to user questions automatically or with human intervention via an online chat function, and means for displaying IR information in real time on a smart device and responding to user voice commands, thereby enabling investors to collect, analyze, and view information necessary for investment decisions quickly and accurately using a variety of devices.

[1022] A "listed company" is a company that is listed on the stock market and whose shares can be traded by general investors.

[1023] "Internet collection methods" refers to technologies or processes that use web crawlers or APIs to collect information publicly available on the Internet.

[1024] "Means of converting audio / video content into text" refers to the technology or process that converts audio data into text using speech recognition technology.

[1025] "Means for extracting, highlighting, and summarizing important keywords and figures" refers to a technology or process that automatically detects important information from text data and highlights and summarizes it.

[1026] "Means of database storage and retrieval" refers to the technology or process that efficiently stores collected and analyzed data and makes it easily retrievalable.

[1027] "Means for providing data to a user's device" refers to the technology or process for providing data to a user's device, such as a computer or smartphone, using the Internet or other communication means.

[1028] "Online chat function" refers to a function that enables real-time text-based communication using the Internet.

[1029] "Automated or human-assisted response" refers to a process in which the system automatically generates a response, and in which a human intervenes as necessary.

[1030] "Smart devices" refers to devices such as smartphones, smart glasses, and tablets that can connect to the Internet and run a variety of applications.

[1031] "Real-time display means" refers to a technology or process that allows data to be displayed on a user's device almost immediately from the moment it is generated or captured.

[1032] "Means for responding to voice commands" refers to a technique or process by which a system responds to instructions input by voice from a user by taking appropriate action.

[1033] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[1034] Server Processing

[1035] The server has a means of collecting information about listed companies from the Internet. Specifically, it periodically collects video and audio content from financial results briefings and general shareholders' meetings via the Internet using web crawlers and APIs. This collected data is stored in cloud storage such as Google Cloud Storage.

[1036] The server then converts the collected audio data into text using the Google Cloud Speech-to-Text API. The resulting text is then analyzed by a generative AI model (such as BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The extracted information is then stored in a database in an efficient, searchable format.

[1037] Terminal handling

[1038] The device (user's computer or smartphone) provides the user with an easy-to-use search interface through a web or mobile application. The device also provides a dashboard displaying the latest information and investor relations information of noteworthy companies. The device also sends requests to the server based on the search query entered by the user, formats the returned data, and displays it to the user.

[1039] User operations

[1040] Users can search and view information by entering the company name or information of interest in the search box and clicking the search button. For example, if you search for "a company's financial results for Q2 2023," you can quickly view information with a summary and important keywords highlighted from the corresponding text data.

[1041] Users can also check IR information in real time using their smart devices. When searching for specific information using voice commands, the device processes the command and quickly displays the relevant information. For example, if a user says, "Tell me the results for the first half of 2023," the results will instantly be displayed on their smartphone or smart glasses.

[1042] Additionally, if users want to know more, they can use the online chat feature to receive automated or human-assisted responses. For example, they can send a question like, "What's the outlook for next year?" to the chatbot, which will then respond by retrieving relevant information from its database.

[1043] Specific examples

[1044] For example, suppose a video of a company's financial results briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. A speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures such as sales and profits. This data is then stored in a database. When a user searches for "the company's latest financial results" on their device, the database provides the relevant text data, allowing the user to easily check the important information. Furthermore, by using a voice command such as "Tell me the company's financial results for 2023," the information is displayed on the smart glasses.

[1045] Example prompt sentence:

[1046] "Summarize the following string and extract a list of key keywords: 'A company's first-half 2023 earnings call reported a 20% increase in revenue and a 15% increase in profits compared to the previous year. The economic outlook remains uncertain, necessitating a new market strategy.'"

[1047] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1048] Step 1:

[1049] The server periodically collects video and audio content from earnings briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using web crawlers and APIs. The collected data is stored in cloud storage along with metadata such as company names, dates, and agendas. The input of this step is publicly available information on the Internet, and the output is raw data stored in cloud storage.

[1050] Step 2:

[1051] The server sends the voice data stored in the cloud storage to the Google Cloud Speech-to-Text API, which converts the voice data into text. The converted data is then saved in a text file or database. The input of this step is the voice data, and the output is text data.

[1052] Step 3:

[1053] The server analyzes the text data using a generative AI model (e.g., BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The results of this analysis are stored in a database. The input for this step is the text data, and the output is the summarized text data and the extracted and highlighted keywords and values.

[1054] Step 4:

[1055] The server indexes the database so that the analysis results can be searched efficiently, allowing users to search data efficiently. The input of this step is the analysis results, and the output is the indexed database.

[1056] Step 5:

[1057] The terminal receives a search query from the user through a user interface and sends the query to the server, which retrieves relevant information from the database, formats it, and returns it to the terminal. The input of this step is the user's search query, and the output is the formatted data received from the server.

[1058] Step 6:

[1059] The terminal displays the search results to the user, who can see important keywords and summaries on the screen. The input of this step is the formatted data, and the output is the search results that are displayed to the user.

[1060] Step 7:

[1061] Users issue voice commands through their smart devices to search for specific IR information in real time. The devices process the voice commands and request related information from the server. The input of this step is the user's voice command, and the output is a request to the server.

[1062] Step 8:

[1063] The server extracts relevant information from a database based on the voice command and returns the formatted information to the terminal, which displays the results in real time on the user's smart device. The input of this step is the server request, and the output is the information displayed on the smart device.

[1064] Step 9:

[1065] The device receives follow-up questions from the user through the online chat function and sends them to the server, which extracts information from the relevant database and responds appropriately. The input for this step is the user's question, and the output is a response from the chatbot or a human agent.

[1066] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1067] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and recognizes user emotions to provide appropriate information and dialogue. This system operates in cooperation with a server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information. Specifically, it is implemented as follows.

[1068] Server Processing

[1069] 1. Data Collection

[1070] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet.

[1071] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[1072] 2. Speech-to-text

[1073] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text, and reprocesses it if necessary.

[1074] 3. Keyword extraction and information emphasis

[1075] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and numbers from the text data and highlight them.

[1076] 4. Introducing the Emotion Engine

[1077] The server incorporates an emotion engine that analyzes user input and recognizes emotions in real time. The emotion engine uses natural language processing techniques to extract user emotions and stores the information in a database.

[1078] 5. Database Operation

[1079] The server stores the processed data in a database, which is indexed and optimized for efficient searching.

[1080] Terminal handling

[1081] 1. Providing a user interface

[1082] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[1083] 2. Search function and information provision

[1084] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[1085] 3. Chat function

[1086] The terminal provides an online chat function, and when a user inputs a question, the content is sent to the server.

[1087] Based on the emotion recognition results, the server generates a response appropriate to the user's emotion and sends it to the device. If the user shows unstable emotions, the server responds appropriately, such as providing a more detailed explanation or encouragement.

[1088] User operations

[1089] 1. Information Search

[1090] Users enter the company name or industry information of interest in the search box and click the search button.

[1091] 2. Viewing Information

[1092] The user reviews the search results provided and selects information of particular interest.

[1093] 3. Chat questions

[1094] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[1095] The device recognizes the user's emotions and provides appropriate answers according to the nuances of the question.

[1096] Specific examples

[1097] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[1098] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Similarly, when a user asks "What's the outlook for next year?" via chat, the emotion engine recognizes the emotion in the user's input and generates and delivers an answer in the appropriate tone. For example, if the user expresses anxiety, the engine can add more detailed explanations or additional information about the context of the data.

[1099] In this way, by linking servers, devices, and users, and by combining the emotion engine, the process of collecting, analyzing, providing, and responding to information becomes more personalized, allowing investors to not only quickly and efficiently obtain appropriate information, but also receive support that reflects their emotions.

[1100] The processing flow will be explained below.

[1101] Server Processing

[1102] Step 1:

[1103] The server runs a crawler to collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The target URLs and API endpoints are set in advance.

[1104] Step 2:

[1105] The server uploads the collected video and audio data to cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[1106] Step 3:

[1107] The server sends the voice data stored in cloud storage to a speech recognition API (e.g., Google Cloud Speech-to-Text API) and converts the voice data into text data. The accuracy of the generated text is checked and reprocessed if necessary.

[1108] Step 4:

[1109] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and values ​​from the text data, highlighting and summarizing the extracted information.

[1110] Step 5:

[1111] The server stores the extracted keywords and summary information in a database, and sets up an index appropriately to enable efficient searches.

[1112] Step 6:

[1113] The server incorporates an emotion engine to analyze user speech and text input in real time, using natural language processing techniques to extract user emotions and store the information in a database.

[1114] Terminal handling

[1115] Step 1:

[1116] The device provides users with an intuitive and easy-to-use search interface through web and mobile applications, where they enter the name of a company or information of interest into a search box.

[1117] Step 2:

[1118] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[1119] Step 3:

[1120] The device receives the search results from the server, formats them in a list or table format, and displays them to the user. The display method changes dynamically depending on the user's emotions.

[1121] Step 4:

[1122] The terminal provides an online chat interface, and when a user types a question, the content is sent to the server. For example, "What is the outlook for next year?"

[1123] User operations

[1124] Step 1:

[1125] Users enter the company name and industry information they are interested in into the search box and click the search button, for example, "XYZ Company's Q2 2023 financial results."

[1126] Step 2:

[1127] Users can review the search results and select and click on the information they are particularly interested in. Detailed information is displayed, and keywords and figures are highlighted, allowing users to quickly grasp the important points.

[1128] Step 3:

[1129] When a user wants to know more about a specific piece of information, they can use the chatbot to enter a question. The emotion engine analyzes the user's input and detects their emotion. For example, they might enter, "I'm worried about the future."

[1130] Step 4:

[1131] The server selects the most appropriate response based on the user's emotions and sends it to the user's device. For example, a user who expresses anxiety may receive detailed data or a message offering reassurance.

[1132] Step 5:

[1133] The user can check the answer displayed on the device and, if necessary, ask the question again to request more detailed information or explanation.

[1134] In this way, by linking the server, terminals, and users, and by combining it with an emotion engine, the information required by users can be efficiently collected and analyzed, and information provided according to individual emotions can be provided, enabling investors to make quick and accurate investment decisions.

[1135] Example 2

[1136] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1137] Existing information gathering systems have difficulty efficiently collecting and analyzing IR information from listed companies and providing information that meets user needs. Furthermore, they lack the ability to recognize users' emotions and provide appropriate information, making it difficult for them to understand the information and make decisions. This prevents investors from obtaining the information they need at the right time, limiting their ability to make quick decisions.

[1138] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: [means for collecting corporate information from the Internet]; [means for converting collected audio and video content into text]; [means for extracting and highlighting important keywords and numerical values ​​from the converted text]; [means for storing the extracted information in a database and making it searchable]; [means for recognizing user emotions using an emotion engine]; [means for providing information based on the user's emotions]; and [means for automatically responding to questions from users using an online chat function]. This makes it possible to efficiently collect and analyze IR information of listed companies and provide personalized information that takes user emotions into consideration.

[1139] "Company" means an organization that includes a publicly listed corporation and its affiliates.

[1140] "Information" refers to the content of video, audio, text data, etc. related to financial results briefings, general shareholders' meetings, and other official announcements.

[1141] The "Internet" is a communications network that interconnects computer networks around the world and enables the exchange of information.

[1142] "Collection" refers to the act of obtaining information that is publicly available on the Internet using a crawler or API.

[1143] "Audio and video content" refers to data including audio and video media officially released by companies at financial results briefings, general shareholders' meetings, etc.

[1144] "Text conversion" is the process of converting words extracted from audio or video into text data.

[1145] "Keywords" are important words or phrases extracted from text data.

[1146] "Numbers" refer to important statistics and measurements contained in data and information.

[1147] "Highlighting" means displaying extracted keywords and numbers in a special format to make them stand out.

[1148] A "database" is a system that organizes and stores data to facilitate searching and analysis.

[1149] "Searchable" means that information in a database can be quickly found based on specific keywords or criteria.

[1150] An "emotion engine" is a system that uses natural language processing technology to extract and recognize emotions from user input.

[1151] "User" refers to a person who accesses information through a terminal and performs searches or inquiries.

[1152] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[1153] "Information provision" refers to the act of displaying collected, processed, and analyzed data on the user's device.

[1154] The "online chat function" is a function that enables real-time conversation with users via the Internet.

[1155] "Automatic response" is a function that automatically provides pre-programmed answers to questions from users.

[1156] This invention is a system that efficiently collects and analyzes corporate IR information, recognizes user emotions, and provides appropriate information and dialogue. This system works in cooperation with the server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information.

[1157] The server first uses crawlers and APIs to periodically collect video and audio content from corporate earnings briefings and general shareholders' meetings that are publicly available on the Internet. Specifically, it searches for videos of listed companies using, for example, the YouTube Data API, and saves the results. The collected data is stored in cloud storage such as Google Cloud Storage, and metadata such as company names, dates, and agendas is stored in a database.

[1158] The server then sends the collected audio data to the Google Cloud Speech-to-Text API, which converts the audio into text. The converted text is then analyzed using a generative AI model (such as BERT or GPT-3) to extract and highlight important keywords and values. The extracted information is then highlighted through tagging and stored in a database.

[1159] The emotion engine is embedded in the server and analyzes user input in real time to recognize emotions. The emotion engine uses natural language processing techniques to extract emotions from user text input and store them in a database.

[1160] The terminal provides users with an easy-to-use search interface through web and mobile applications. For example, frameworks such as React and Flutter are used to build an intuitive and easy-to-use UI. Users can use the search interface to enter the company name or industry information they are interested in and easily find related information.

[1161] When a user enters a search query, the device sends the query to the server and formats and displays the returned data. The display method and content are dynamically changed based on the user's emotions. By determining the user's emotions, for example, if the user expresses anxiety, a more detailed explanation can be provided.

[1162] The device also provides an online chat function, allowing users to input specific questions and send them to the server. The server then uses an emotion engine to recognize the user's emotions, generates appropriate responses, and sends them to the device. This process allows users to obtain specialized information in real time.

[1163] For example, a server can collect videos from the YouTube API using the search keyword "XYZ company earnings briefing" and store them in Google Cloud Storage. The audio from these videos is then converted to text using the Google Cloud Speech-to-Text API, and a generative AI model is used to extract and highlight keywords such as "sales," "year-over-year comparison," and "10% increase." When a user searches for "XYZ company's latest earnings information," the database provides the relevant text data, highlighting the key information.

[1164] As an example of a prompt sentence, if a user inputs "What are the prospects for next year?", the device will send this question to the server, and the emotion engine will recognize the emotion from the user's input. For example, if the user expresses anxiety, detailed explanations and additional information will be provided. By coordinating the server, device, and user, this invention personalizes and streamlines the process of collecting, analyzing, providing, and responding to information.

[1165] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1166] Step 1: Data collection

[1167] The server uses crawlers and APIs to periodically collect video and audio content from companies' financial results briefings and general shareholders' meetings that are publicly available on the Internet. For example, the crawler is used to explore a specific company's YouTube channel, and when new videos are discovered, the links are collected. The collected data includes metadata (company name, date, agenda), which is stored in a database. The input is the company information URL, and the output is the audio and video data and metadata stored in cloud storage.

[1168] Step 2: Speech to text

[1169] The server sends the collected audio data to the Google Cloud Speech-to-Text API and converts the audio to text. For example, send the audio file of a video titled "XYZ Company Earnings Briefing" to the API endpoint to obtain text data. The input is audio data, and the output is text data converted from the audio.

[1170] Step 3: Keyword extraction and information emphasis

[1171] The server uses a generative AI model (such as BERT or GPT-3) to extract important keywords and numbers from text data. For example, it highlights words and numbers such as "sales" and "profit." The input is text data, and the output is text data containing highlighted keywords and numbers.

[1172] Step 4: Store in the database

[1173] The server stores the highlighted text data in a database, indexes it, and makes it searchable. Specifically, it creates a table in an SQL database with fields for "company name," "date," and "keyword" to store the extracted data. The input is the highlighted text data, and the output is a searchable database entry.

[1174] Step 5: Emotion Recognition with the Emotion Engine

[1175] The server analyzes text input from the user using natural language processing technology and recognizes emotions. For example, if a user asks, "What are the prospects for next year?", the emotion engine analyzes the question and generates an emotion label such as "anxiety." The input is the user's text input, and the output is the recognized emotion label.

[1176] Step 6: Providing a User Interface

[1177] The terminal uses frameworks such as React and Flutter to build the user interface, providing a search bar and filter options. Users can search for information by entering the name of a company or industry they are interested in into the search box. The input is the user's search query, and the output is the search results displayed on the interface.

[1178] Step 7: Serving search results

[1179] The device sends the search query entered by the user to the server and formats and displays the returned text data. For example, if you search for "XYZ Company's latest financial results," that data will be displayed on the screen. The input is the search query, and the output is the search results that are displayed to the user.

[1180] Step 8: Online chat function

[1181] When a user uses the online chat function to input a question, the device sends the content to the server. The server uses an emotion engine to analyze the user's emotions, generates the most appropriate answer, and sends it back to the device. For example, if a user asks, "What are the prospects for next year?" and the emotion is determined to be anxiety, a detailed answer will be generated accordingly. The input is the user's question, and the output is a response message.

[1182] This series of steps enables the server, terminal, and user to work together to efficiently collect, analyze, and provide information, enabling optimal support tailored to the user's emotions.

[1183] (Application example 2)

[1184] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] Currently, there is no system that efficiently collects and analyzes IR information about listed companies and provides users with real-time, personalized information. Furthermore, there are no services that take user sentiment into consideration when collecting and analyzing this information. As a result, investors and other users are unable to quickly and appropriately obtain the information they need, making it difficult to resolve their concerns and questions.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, and means for extracting, highlighting, and summarizing important keywords and figures from the converted text. This enables efficient analysis and storage of the collected information. Furthermore, by including means for recognizing a user's emotions and dynamically changing the method and content of information provision based on the emotions, more user-friendly information can be provided. Furthermore, an online chat function can respond to user questions automatically or with human intervention, helping to alleviate anxiety. Furthermore, by providing additional information and risk explanations during electronic payments based on the user's emotions, a sense of security can be enhanced during investments and transactions.

[1187] "Listed company" refers to a company that is listed on a stock exchange.

[1188] "IR information" refers to the public relations information that companies provide to investors. Generally, this includes financial results briefings, general shareholders' meetings, and information disclosure on official websites.

[1189] "Collecting from the Internet" means automatically obtaining information that is publicly available on the Internet using web crawlers or APIs.

[1190] "Converting audio / video content to text" means converting the audio information in audio or video into text data using a speech recognition API or other technology.

[1191] "Extracting, highlighting, and summarizing keywords and numbers" means using natural language processing technology to select important information from text data and visually highlight or summarize it.

[1192] "Storing in a database and making it searchable" refers to the process of storing the extracted information in a database so that it can be efficiently searched.

[1193] "Recognizing user emotions" means using natural language processing technology or other emotion analysis technology to read emotions from the user's statements and actions.

[1194] The "online chat function" is a function that allows users to ask questions to the system in real time, and the system or a human responds to them.

[1195] "Automated or human-assisted response" refers to a response method that not only provides the user with an automatically generated response from the system, but also includes human operator intervention as needed.

[1196] "Dynamic change of information based on emotion" refers to appropriately changing the display method of information and the content of information provided based on the result of the user's emotion recognition.

[1197] "Providing additional information and risk explanations during electronic payments" means providing additional information and risk explanations during the financial transaction or electronic payment process, taking into consideration the user's feelings.

[1198] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and also realizes personalized information provision and dialogue by recognizing user emotions. This system operates in cooperation with a server, terminals, and users, and by combining an emotion engine in particular, it is possible to provide more accurate information.

[1199] System configuration

[1200] Server Processing

[1201] 1. Data Collection

[1202] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The collected data is then stored in cloud storage.

[1203] 2. Speech-to-text

[1204] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text data, and reprocesses it if necessary.

[1205] 3. Keyword extraction and information emphasis

[1206] The server uses a generative AI model (such as OpenAI's GPT-3) to extract and highlight important keywords and figures from the text data. This analysis uses natural language processing techniques to efficiently extract information.

[1207] 4. Introducing the Emotion Engine

[1208] The server uses an emotion engine (e.g., OpenAI's emotion analysis API) to analyze user input and recognize emotions in real time. This emotion data is also stored in a database.

[1209] 5. Database Operation

[1210] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[1211] Terminal handling

[1212] 1. Providing a user interface

[1213] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[1214] 2. Search function and information provision

[1215] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[1216] 3. Chat function

[1217] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server generates an answer appropriate to the user's emotion based on the emotion recognition results and sends it to the device.

[1218] User operations

[1219] 1. Information Search

[1220] Users enter the company name or industry information of interest in the search box and click the search button.

[1221] 2. Viewing Information

[1222] The user reviews the search results provided and selects information of particular interest.

[1223] 3. Chat questions

[1224] When a user wants to know more about a specific piece of information, they can use the chatbot to input a question, such as "What is the outlook for next year?" The device will recognize the user's emotions and provide an appropriate answer based on the nuances of the question.

[1225] Specific examples

[1226] For example, suppose a video of a company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. This data is then stored in a database. When a user searches for "latest financial information" on their device, the database provides the relevant text data, allowing the user to easily check important information. Similarly, if a user asks "What are the outlook for next year?" via chat, the emotion engine recognizes the emotion from the user's input and generates and delivers an answer in an appropriate tone. Even if the user expresses anxiety, more detailed explanations or supplementary information about the data's context can be added.

[1227] Prompt Sentence Examples

[1228] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[1229] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[1230] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1231] The following are specific processing steps of the system that realizes the application example.

[1232] Step 1:

[1233] Data collection

[1234] The server uses a crawler and API to collect video and audio content from listed companies' financial results briefings and general shareholders' meetings from the Internet. Specifically, the crawler accesses specific websites and retrieves the URLs of the video and audio files. The input is the URL set in the crawler, and the output is the collected video and audio files.

[1235] Step 2:

[1236] Speech to text

[1237] The server sends the collected voice data to the Google Cloud Speech-to-Text API and converts it into text data. The input is the collected voice data, and the output is the data converted from voice to text. The server converts the voice data into binary format, sends it to the API, and saves the returned text data.

[1238] Step 3:

[1239] Keyword extraction and information emphasis

[1240] The server uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords and numbers from text data and highlight them. The input is speech-to-text data, and the output is text data with the extracted keywords and numbers highlighted. Specific operations are performed by the server sending prompts to the generative AI model and receiving the analysis results.

[1241] Example prompt sentence:

[1242] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[1243] Step 4:

[1244] Database storage and index optimization

[1245] The server stores the extracted text data, including keywords and numbers, in a database and optimizes the index for efficient searches. The input is text data with keywords and numbers highlighted, and the output is indexed data stored in the database. The specific operations are to insert data into the database and rebuild the index.

[1246] Step 5:

[1247] Introducing the Emotion Engine

[1248] The server uses the emotion engine to analyze user inputs and search queries in real time and recognize emotions. The input is text input from the user, and the output is the emotion recognition result. Specific operations include sending text data to the emotion engine and storing the emotion analysis result in a database.

[1249] Example prompt sentence:

[1250] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[1251] Step 6:

[1252] Providing a user interface

[1253] The terminal provides an easy-to-use search interface through a web application or a mobile application, where the input is a user's search query and the output is parsed data obtained from the server. The specific operation includes the user entering a query into a search box and sending the query to the server.

[1254] Step 7:

[1255] Search function and information provision

[1256] The terminal transmits the search query entered by the user to the server and displays the data returned by the server, where the input is the user's search query and the output is the data returned by the server. Specific operations include formatting the data received from the server and displaying it to the user.

[1257] Step 8:

[1258] Providing chat functionality and emotional responses

[1259] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server uses an emotion engine to analyze the user's emotions and generate and provide an appropriate answer. The input is the user's question and emotional state, and the output is an appropriate answer returned by the server. Specific operations include generating an answer using a generative AI model based on the emotion recognition results and sending it to the device.

[1260] Step 9:

[1261] Providing additional information when making electronic payments

[1262] The server provides additional information and risk explanations during electronic payments based on the user's emotions. The input is the user's payment request and emotional state, and the output is a transaction screen containing additional information and risk explanations. The specific operation is to refer to the emotion recognition results and provide the necessary information to the user.

[1263] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1264] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1265] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1266] [Fourth embodiment]

[1267] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1268] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1269] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1270] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1271] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1272] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1273] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1274] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1275] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1276] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1277] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1278] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1279] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1280] This invention relates to a system that efficiently collects, analyzes, and provides investor relations information of listed companies to investors. This system operates in cooperation with a server, a terminal, and a user, and is specifically implemented as follows.

[1281] Server Processing

[1282] 1. Data Collection

[1283] The server periodically collects video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using crawlers and APIs.

[1284] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[1285] 2. Speech-to-text

[1286] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data.

[1287] Evaluate the quality of the generated text data and manually correct it if necessary.

[1288] 3. Keyword extraction and information emphasis

[1289] The server uses generative AI models (e.g., BERT, GPT-3) to automatically extract important keywords and numbers from text data and highlight them.

[1290] The extracted information is summarized within the server and stored in a database along with associated metadata.

[1291] 4. Database operation

[1292] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[1293] Terminal handling

[1294] 1. Providing a user interface

[1295] The device (user's PC or smartphone) provides the user with an easy-to-use search interface through web applications or mobile applications.

[1296] The device provides a dashboard that displays the latest information and investor relations information for noteworthy companies.

[1297] 2. Performing a search function

[1298] The device sends the search query entered by the user to the server, formats the data returned by the server, and displays it to the user.

[1299] User operations

[1300] 1. Information Search

[1301] Users enter the company name or industry information of interest in the search box and click the search button.

[1302] For example, a user searches for "XYZ Company's Q2 2023 financial results."

[1303] 2. Viewing Information

[1304] The user can review the search results provided and select information of particular interest for further details.

[1305] Search results highlight important keywords and figures, allowing users to quickly grasp the main points.

[1306] 3. Chat questions

[1307] If users want to know more about a particular piece of information, they can use the chatbot to type in a question, such as, "What's the outlook for next year?"

[1308] The chatbot receives the user's question, extracts relevant information from the appropriate database, generates an answer, and displays it again on the device.

[1309] If necessary, a human agent will step in to provide a more detailed answer or interpretation.

[1310] Specific examples

[1311] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[1312] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Furthermore, if a user has a specific question, they can use the chatbot to ask, and relevant information will be returned instantly. This system allows users to quickly and efficiently obtain the information they need to make investment decisions.

[1313] In this way, the server, terminals, and users work together to collect, analyze, provide, and respond to information in a consistent process. This system allows investors to obtain more accurate information quickly and make appropriate investment decisions.

[1314] The processing flow will be explained below.

[1315] Server Processing

[1316] Step 1:

[1317] The server runs a crawler to collect video and audio content of financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet, including on the company's official website and YouTube channel.

[1318] Step 2:

[1319] The server stores the collected video and audio data in cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[1320] Step 3:

[1321] The server sends the saved voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data. The accuracy of the generated text is checked and reprocessed if necessary.

[1322] Step 4:

[1323] The server extracts important keywords and figures from the text data using a generative AI model that automatically detects and highlights important information that frequently appears in the text.

[1324] Step 5:

[1325] The server highlights the extracted keywords and values ​​and stores the summarized information in a database, which is indexed and optimized for fast and efficient searching.

[1326] Terminal handling

[1327] Step 1:

[1328] The device provides users with an intuitive search interface through web and mobile applications, allowing them to easily find information that interests them.

[1329] Step 2:

[1330] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[1331] Step 3:

[1332] The terminal receives the search results returned from the server, formats the results in a list or table format, and displays them to the user.

[1333] User operations

[1334] Step 1:

[1335] Users enter the company name or industry information of interest in the search box and click the search button.

[1336] Step 2:

[1337] The user can check the list of search results provided, select and click on the information that interests them, and detailed information will be displayed.

[1338] Step 3:

[1339] If a user wants more information about a particular piece of information, they can use the chat interface to type in a question, such as "What's the outlook for next year?"

[1340] Step 4:

[1341] The terminal sends the user's question to the server, which extracts relevant information from a database, generates an appropriate answer, and sends it to the terminal.

[1342] Step 5:

[1343] The user sees the answers displayed in the chat interface and can ask more detailed questions if needed.

[1344] The above processing steps enable effective information collection, analysis, and provision between the server, terminals, and users, allowing investors to obtain the information they need to make investment decisions more quickly and accurately.

[1345] Example 1

[1346] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1347] Currently, it is difficult to efficiently collect, analyze, and provide investor relations information for listed companies to investors. Many challenges exist, particularly in the process of extracting important information from audio and video content and quickly providing it as data necessary for investment decisions. Specifically, there is a need for automation of data collection and conversion, accurate extraction of important information, and provision of information in a format that is easily accessible to users.

[1348] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1349] In this invention, the server includes: [means for collecting information on aggregating entities from a communication network;] [means for converting the collected audio and video into text information; and] [means for extracting, emphasizing, and summarizing key identifiable elements and numerical values ​​from the converted text information.] This makes it possible [to streamline the entire process from collection to analysis and provision, enabling investors to quickly obtain the information they need].

[1350] "Aggregate entities" refer to companies that are listed on the stock market and that disclose information about their financial and operational activities.

[1351] "Communications network" refers to a network for transmitting digital information, including the Internet, which is used to collect, send, and receive data.

[1352] "Audio and video" refers to audio and video content released at corporate financial results briefings and general shareholders' meetings, etc., which includes important information about corporate activities.

[1353] "Text information" refers to text data converted from audio or video, formatted in a format that is easy for humans to read.

[1354] "Key identifiable elements" refer to keywords and numerical information that are recognized as being particularly important in text data and play an important role in investment decisions.

[1355] "Storage" refers to a storage device, such as a database, for storing collected data so that it can be easily accessed at a later time.

[1356] "Information terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to obtain information.

[1357] "Online interactivity" refers to an interface through which a user can input questions or requests to a system, including automated chatbots and human operators.

[1358] "Remote storage" refers to a remotely accessible storage facility, such as cloud storage, that allows large amounts of data to be stored securely and efficiently.

[1359] "Generated AI model" refers to a model that uses artificial intelligence technology to analyze data, thereby automatically extracting key information.

[1360] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[1361] Server Operation

[1362] The server uses multiple hardware and software components to collect, process, and store data. Specifically, it uses crawlers and APIs to gather information from the aggregator across the internet. The collected data is stored as audio and video and stored in cloud storage, such as Amazon S3.

[1363] The server then converts the collected audio and video data into text using the Google Cloud Speech-to-Text API, allowing important information from the video and audio data to be recorded in text format.

[1364] The server then uses the generated AI model (e.g., GPT-3) to analyze the text and send prompts to extract key identifiable elements and numbers, such as "Please extract the key points from the financial report."

[1365] The extracted information is highlighted using bold or color coding and stored in a database, where it is managed using a database system such as MySQL and indexed for efficient searching.

[1366] Device behavior

[1367] The terminal provides an interface for users to obtain information. It is designed to allow users to easily search and browse using web and mobile applications. Specifically, frameworks such as React and Flutter are used to build dashboards and search interfaces.

[1368] When a user enters the name or information of a company of interest in the search box, the device sends the query to the server, formats the data received from the server, and displays it to the user. For example, enter "XYZ Company's 2023 Q2 financial results information" and click the search button. At this time, important information is highlighted, allowing the user to quickly find the information they need.

[1369] User operations

[1370] Users can easily search and browse information through a search interface. If users have further questions about specific information, they can use the online dialogue function. The chatbot receives the question and generates an answer by extracting relevant information from the database. For example, the chatbot will provide an appropriate answer to the question, "What is the outlook for next year?"

[1371] This system allows servers, terminals, and users to seamlessly link together, efficiently collecting, analyzing, and providing IR information on listed companies, allowing investors to quickly obtain more accurate information and make appropriate investment decisions.

[1372] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1373] Server Processing Steps

[1374] Step 1: Data collection

[1375] The server periodically runs crawlers and APIs to collect information about aggregators from the Internet, downloading video and audio content from the companies' official websites and investor relations platforms. For example, the server launches the crawler at 2:00 AM every day to check for and retrieve new content. The input is a list of URLs, and the output is the collected video and audio files.

[1376] Input: URL list

[1377] Data processing: Collect information using crawlers and APIs

[1378] Output: Video file, audio file

[1379] Step 2: Save your data

[1380] The collected data is stored in cloud storage (e.g., Amazon S3), along with metadata such as the company name, date, and agenda. For example, a file is saved in the format "Company A_2023Q2 Financial Results.mp4," and the associated metadata is recorded in the database.

[1381] Input: Video files, audio files

[1382] Data processing: Extracting metadata and saving files to cloud storage

[1383] Output: Files stored in cloud storage, metadata in a database

[1384] Step 3: Speech to Text

[1385] The saved audio file is sent to the Google Cloud Speech-to-Text API to convert the audio into text. This allows important information from the audio data to be recorded as text. For example, create a processing job called "transcribeJob" and upload the audio file to the API.

[1386] Input: Audio file

[1387] Data calculation: Convert to text information using speech recognition API

[1388] Output: Text information

[1389] Step 4: Keyword extraction

[1390] The generated AI model (e.g., GPT-3) is used to analyze the text and send prompts to extract key identifiable elements and numbers. For example, a prompt such as "Please extract the key points from the financial report" is used.

[1391] Input: Text information, prompt text

[1392] Data computation: Extracting keywords and values ​​using generative AI models

[1393] Output: Main identifiable elements, numbers

[1394] Step 5: Highlighting Information

[1395] Extracted keywords and numerical values ​​are highlighted in the text data, allowing users to check important information at a glance. For example, important numerical items such as "sales" and "net profit" are highlighted in bold.

[1396] Input: Primary identifiable element, number

[1397] Data processing: highlighting key elements

[1398] Output: Highlighted text data

[1399] Step 6: Operating the Database

[1400] The processed text data and metadata are stored in a database and the index is optimized for efficient searching, for example, by adding a full-text search index to improve response times to user search queries.

[1401] Input: Highlighted text data, metadata

[1402] Data processing: Optimizing database storage and indexing

[1403] Output: A searchable database

[1404] Terminal processing steps

[1405] Step 1: Provide a search interface

[1406] The terminal provides a search interface that can be used by users. It is designed to allow users to easily search and browse using web and mobile applications. For example, interfaces built with React or Flutter are provided.

[1407] Input: User's search query

[1408] Data processing: Providing a search interface

[1409] Output: Search results displayed

[1410] Step 2: Providing a dashboard

[1411] The device provides users with a dashboard that displays the latest IR information and information on noteworthy companies. For example, the top page displays "latest financial information" and "information on noteworthy companies" in card format.

[1412] Input: Database information from the server

[1413] Data processing: Dashboard generation and display

[1414] Output: Dashboard

[1415] User operations

[1416] Step 1: Information search

[1417] Users enter the company name and industry information they are interested in into the search box and click the search button. For example, a user searches for "Company A's financial results for Q2 2023."

[1418] Input: Company name and industry information

[1419] Data transport: sending search queries to the server

[1420] Output: Returning search results

[1421] Step 2: View information

[1422] Users can review the provided search results and select details of the information they are particularly interested in. For example, sales and net profits are highlighted, allowing users to quickly grasp key data.

[1423] Input: Search results

[1424] Data processing: highlighting important information

[1425] Output: Understanding the information

[1426] Step 3: Ask a question via chat

[1427] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[1428] Input: User question

[1429] Data processing: Chatbot response generation

[1430] Output: Answer to the question

[1431] As described above, this system can efficiently collect, analyze, and provide IR information on listed companies through specific processing steps and the accompanying data processing and calculations, allowing investors to quickly and accurately obtain the information they need and make appropriate investment decisions.

[1432] (Application example 1)

[1433] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1434] When efficiently collecting, analyzing, and providing investor relations information for listed companies to investors, it is necessary to improve accessibility across a variety of devices. There is also a need for real-time information provision and interactive response functions via voice commands, providing investors with fast and accurate information when making investment decisions. Furthermore, a system that utilizes generative AI models to intelligently extract, highlight, and summarize important keywords and figures is required.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1436] In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, means for extracting, highlighting, and summarizing important keywords and figures from the converted text, means for storing the extracted information in a database and making it searchable, means for providing the text data and the extracted information to a user's terminal, means for responding to user questions automatically or with human intervention via an online chat function, and means for displaying IR information in real time on a smart device and responding to user voice commands, thereby enabling investors to collect, analyze, and view information necessary for investment decisions quickly and accurately using a variety of devices.

[1437] A "listed company" is a company that is listed on the stock market and whose shares can be traded by general investors.

[1438] "Internet collection methods" refers to technologies or processes that use web crawlers or APIs to collect information publicly available on the Internet.

[1439] "Means of converting audio / video content into text" refers to the technology or process that converts audio data into text using speech recognition technology.

[1440] "Means for extracting, highlighting, and summarizing important keywords and figures" refers to a technology or process that automatically detects important information from text data and highlights and summarizes it.

[1441] "Means of database storage and retrieval" refers to the technology or process that efficiently stores collected and analyzed data and makes it easily retrievalable.

[1442] "Means for providing data to a user's device" refers to the technology or process for providing data to a user's device, such as a computer or smartphone, using the Internet or other communication means.

[1443] "Online chat function" refers to a function that enables real-time text-based communication using the Internet.

[1444] "Automated or human-assisted response" refers to a process in which the system automatically generates a response, and in which a human intervenes as necessary.

[1445] "Smart devices" refers to devices such as smartphones, smart glasses, and tablets that can connect to the Internet and run a variety of applications.

[1446] "Real-time display means" refers to a technology or process that allows data to be displayed on a user's device almost immediately from the moment it is generated or captured.

[1447] "Means for responding to voice commands" refers to a technique or process by which a system responds to instructions input by voice from a user by taking appropriate action.

[1448] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies and provides it to investors. This system operates in cooperation with three parties: a server, a terminal, and a user.

[1449] Server Processing

[1450] The server has a means of collecting information about listed companies from the Internet. Specifically, it periodically collects video and audio content from financial results briefings and general shareholders' meetings via the Internet using web crawlers and APIs. This collected data is stored in cloud storage such as Google Cloud Storage.

[1451] The server then converts the collected audio data into text using the Google Cloud Speech-to-Text API. The resulting text is then analyzed by a generative AI model (such as BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The extracted information is then stored in a database in an efficient, searchable format.

[1452] Terminal handling

[1453] The device (user's computer or smartphone) provides the user with an easy-to-use search interface through a web or mobile application. The device also provides a dashboard displaying the latest information and investor relations information of noteworthy companies. The device also sends requests to the server based on the search query entered by the user, formats the returned data, and displays it to the user.

[1454] User operations

[1455] Users can search and view information by entering the company name or information of interest in the search box and clicking the search button. For example, if you search for "a company's financial results for Q2 2023," you can quickly view information with a summary and important keywords highlighted from the corresponding text data.

[1456] Users can also check IR information in real time using their smart devices. When searching for specific information using voice commands, the device processes the command and quickly displays the relevant information. For example, if a user says, "Tell me the results for the first half of 2023," the results will instantly be displayed on their smartphone or smart glasses.

[1457] Additionally, if users want to know more, they can use the online chat feature to receive automated or human-assisted responses. For example, they can send a question like, "What's the outlook for next year?" to the chatbot, which will then respond by retrieving relevant information from its database.

[1458] Specific examples

[1459] For example, suppose a video of a company's financial results briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. A speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures such as sales and profits. This data is then stored in a database. When a user searches for "the company's latest financial results" on their device, the database provides the relevant text data, allowing the user to easily check the important information. Furthermore, by using a voice command such as "Tell me the company's financial results for 2023," the information is displayed on the smart glasses.

[1460] Example prompt sentence:

[1461] "Summarize the following string and extract a list of key keywords: 'A company's first-half 2023 earnings call reported a 20% increase in revenue and a 15% increase in profits compared to the previous year. The economic outlook remains uncertain, necessitating a new market strategy.'"

[1462] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1463] Step 1:

[1464] The server periodically collects video and audio content from earnings briefings and general shareholders' meetings of listed companies that are publicly available on the Internet using web crawlers and APIs. The collected data is stored in cloud storage along with metadata such as company names, dates, and agendas. The input of this step is publicly available information on the Internet, and the output is raw data stored in cloud storage.

[1465] Step 2:

[1466] The server sends the voice data stored in the cloud storage to the Google Cloud Speech-to-Text API, which converts the voice data into text. The converted data is then saved in a text file or database. The input of this step is the voice data, and the output is text data.

[1467] Step 3:

[1468] The server analyzes the text data using a generative AI model (e.g., BERT or GPT-3) to extract, highlight, and summarize important keywords and values. The results of this analysis are stored in a database. The input for this step is the text data, and the output is the summarized text data and the extracted and highlighted keywords and values.

[1469] Step 4:

[1470] The server indexes the database so that the analysis results can be searched efficiently, allowing users to search data efficiently. The input of this step is the analysis results, and the output is the indexed database.

[1471] Step 5:

[1472] The terminal receives a search query from the user through a user interface and sends the query to the server, which retrieves relevant information from the database, formats it, and returns it to the terminal. The input of this step is the user's search query, and the output is the formatted data received from the server.

[1473] Step 6:

[1474] The terminal displays the search results to the user, who can see important keywords and summaries on the screen. The input of this step is the formatted data, and the output is the search results that are displayed to the user.

[1475] Step 7:

[1476] Users issue voice commands through their smart devices to search for specific IR information in real time. The devices process the voice commands and request related information from the server. The input of this step is the user's voice command, and the output is a request to the server.

[1477] Step 8:

[1478] The server extracts relevant information from a database based on the voice command and returns the formatted information to the terminal, which displays the results in real time on the user's smart device. The input of this step is the server request, and the output is the information displayed on the smart device.

[1479] Step 9:

[1480] The device receives follow-up questions from the user through the online chat function and sends them to the server, which extracts information from the relevant database and responds appropriately. The input for this step is the user's question, and the output is a response from the chatbot or a human agent.

[1481] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1482] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and recognizes user emotions to provide appropriate information and dialogue. This system operates in cooperation with a server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information. Specifically, it is implemented as follows.

[1483] Server Processing

[1484] 1. Data Collection

[1485] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet.

[1486] The collected data is stored in cloud storage, and metadata such as company names, dates, and agendas are also collected and stored in a database.

[1487] 2. Speech-to-text

[1488] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text, and reprocesses it if necessary.

[1489] 3. Keyword extraction and information emphasis

[1490] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and numbers from the text data and highlight them.

[1491] 4. Introducing the Emotion Engine

[1492] The server incorporates an emotion engine that analyzes user input and recognizes emotions in real time. The emotion engine uses natural language processing techniques to extract user emotions and stores the information in a database.

[1493] 5. Database Operation

[1494] The server stores the processed data in a database, which is indexed and optimized for efficient searching.

[1495] Terminal handling

[1496] 1. Providing a user interface

[1497] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[1498] 2. Search function and information provision

[1499] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[1500] 3. Chat function

[1501] The terminal provides an online chat function, and when a user inputs a question, the content is sent to the server.

[1502] Based on the emotion recognition results, the server generates a response appropriate to the user's emotion and sends it to the device. If the user shows unstable emotions, the server responds appropriately, such as providing a more detailed explanation or encouragement.

[1503] User operations

[1504] 1. Information Search

[1505] Users enter the company name or industry information of interest in the search box and click the search button.

[1506] 2. Viewing Information

[1507] The user reviews the search results provided and selects information of particular interest.

[1508] 3. Chat questions

[1509] If a user wants to know more about a specific piece of information, they can use the chatbot to type in a question, for example, "What's the outlook for next year?"

[1510] The device recognizes the user's emotions and provides appropriate answers according to the nuances of the question.

[1511] Specific examples

[1512] For example, suppose a video of XYZ Company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. The data is then stored in a database.

[1513] When a user searches for "XYZ Company's latest financial results" on their device, the database provides relevant text data, allowing the user to easily check important information. Similarly, when a user asks "What's the outlook for next year?" via chat, the emotion engine recognizes the emotion in the user's input and generates and delivers an answer in the appropriate tone. For example, if the user expresses anxiety, the engine can add more detailed explanations or additional information about the context of the data.

[1514] In this way, by linking servers, devices, and users, and by combining the emotion engine, the process of collecting, analyzing, providing, and responding to information becomes more personalized, allowing investors to not only quickly and efficiently obtain appropriate information, but also receive support that reflects their emotions.

[1515] The processing flow will be explained below.

[1516] Server Processing

[1517] Step 1:

[1518] The server runs a crawler to collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The target URLs and API endpoints are set in advance.

[1519] Step 2:

[1520] The server uploads the collected video and audio data to cloud storage, along with metadata such as company names, dates, and agendas, which are then stored in a database.

[1521] Step 3:

[1522] The server sends the voice data stored in cloud storage to a speech recognition API (e.g., Google Cloud Speech-to-Text API) and converts the voice data into text data. The accuracy of the generated text is checked and reprocessed if necessary.

[1523] Step 4:

[1524] The server uses a generative AI model (e.g., BERT or GPT-3) to extract important keywords and values ​​from the text data, highlighting and summarizing the extracted information.

[1525] Step 5:

[1526] The server stores the extracted keywords and summary information in a database, and sets up an index appropriately to enable efficient searches.

[1527] Step 6:

[1528] The server incorporates an emotion engine to analyze user speech and text input in real time, using natural language processing techniques to extract user emotions and store the information in a database.

[1529] Terminal handling

[1530] Step 1:

[1531] The device provides users with an intuitive and easy-to-use search interface through web and mobile applications, where they enter the name of a company or information of interest into a search box.

[1532] Step 2:

[1533] The device sends the search query entered by the user to the server. For example, a specific search query such as "XYZ company's financial results for Q2 2023" is entered.

[1534] Step 3:

[1535] The device receives the search results from the server, formats them in a list or table format, and displays them to the user. The display method changes dynamically depending on the user's emotions.

[1536] Step 4:

[1537] The terminal provides an online chat interface, and when a user types a question, the content is sent to the server. For example, "What is the outlook for next year?"

[1538] User operations

[1539] Step 1:

[1540] Users enter the company name and industry information they are interested in into the search box and click the search button, for example, "XYZ Company's Q2 2023 financial results."

[1541] Step 2:

[1542] Users can review the search results and select and click on the information they are particularly interested in. Detailed information is displayed, and keywords and figures are highlighted, allowing users to quickly grasp the important points.

[1543] Step 3:

[1544] When a user wants to know more about a specific piece of information, they can use the chatbot to enter a question. The emotion engine analyzes the user's input and detects their emotion. For example, they might enter, "I'm worried about the future."

[1545] Step 4:

[1546] The server selects the most appropriate response based on the user's emotions and sends it to the user's device. For example, a user who expresses anxiety may receive detailed data or a message offering reassurance.

[1547] Step 5:

[1548] The user can check the answer displayed on the device and, if necessary, ask the question again to request more detailed information or explanation.

[1549] In this way, by linking the server, terminals, and users, and by combining it with an emotion engine, the information required by users can be efficiently collected and analyzed, and information provided according to individual emotions can be provided, enabling investors to make quick and accurate investment decisions.

[1550] Example 2

[1551] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1552] Existing information gathering systems have difficulty efficiently collecting and analyzing IR information from listed companies and providing information that meets user needs. Furthermore, they lack the ability to recognize users' emotions and provide appropriate information, making it difficult for them to understand the information and make decisions. This prevents investors from obtaining the information they need at the right time, limiting their ability to make quick decisions.

[1553] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: [means for collecting corporate information from the Internet]; [means for converting collected audio and video content into text]; [means for extracting and highlighting important keywords and numerical values ​​from the converted text]; [means for storing the extracted information in a database and making it searchable]; [means for recognizing user emotions using an emotion engine]; [means for providing information based on the user's emotions]; and [means for automatically responding to questions from users using an online chat function]. This makes it possible to efficiently collect and analyze IR information of listed companies and provide personalized information that takes user emotions into consideration.

[1554] "Company" means an organization that includes a publicly listed corporation and its affiliates.

[1555] "Information" refers to the content of video, audio, text data, etc. related to financial results briefings, general shareholders' meetings, and other official announcements.

[1556] The "Internet" is a communications network that interconnects computer networks around the world and enables the exchange of information.

[1557] "Collection" refers to the act of obtaining information that is publicly available on the Internet using a crawler or API.

[1558] "Audio and video content" refers to data including audio and video media officially released by companies at financial results briefings, general shareholders' meetings, etc.

[1559] "Text conversion" is the process of converting words extracted from audio or video into text data.

[1560] "Keywords" are important words or phrases extracted from text data.

[1561] "Numbers" refer to important statistics and measurements contained in data and information.

[1562] "Highlighting" means displaying extracted keywords and numbers in a special format to make them stand out.

[1563] A "database" is a system that organizes and stores data to facilitate searching and analysis.

[1564] "Searchable" means that information in a database can be quickly found based on specific keywords or criteria.

[1565] An "emotion engine" is a system that uses natural language processing technology to extract and recognize emotions from user input.

[1566] "User" refers to a person who accesses information through a terminal and performs searches or inquiries.

[1567] A "terminal" is an electronic device used by a user, such as a computer, smartphone, or tablet.

[1568] "Information provision" refers to the act of displaying collected, processed, and analyzed data on the user's device.

[1569] The "online chat function" is a function that enables real-time conversation with users via the Internet.

[1570] "Automatic response" is a function that automatically provides pre-programmed answers to questions from users.

[1571] This invention is a system that efficiently collects and analyzes corporate IR information, recognizes user emotions, and provides appropriate information and dialogue. This system works in cooperation with the server, terminal, and user, and by combining it with an emotion engine, it enables the provision of more personalized information.

[1572] The server first uses crawlers and APIs to periodically collect video and audio content from corporate earnings briefings and general shareholders' meetings that are publicly available on the Internet. Specifically, it searches for videos of listed companies using, for example, the YouTube Data API, and saves the results. The collected data is stored in cloud storage such as Google Cloud Storage, and metadata such as company names, dates, and agendas is stored in a database.

[1573] The server then sends the collected audio data to the Google Cloud Speech-to-Text API, which converts the audio into text. The converted text is then analyzed using a generative AI model (such as BERT or GPT-3) to extract and highlight important keywords and values. The extracted information is then highlighted through tagging and stored in a database.

[1574] The emotion engine is embedded in the server and analyzes user input in real time to recognize emotions. The emotion engine uses natural language processing techniques to extract emotions from user text input and store them in a database.

[1575] The terminal provides users with an easy-to-use search interface through web and mobile applications. For example, frameworks such as React and Flutter are used to build an intuitive and easy-to-use UI. Users can use the search interface to enter the company name or industry information they are interested in and easily find related information.

[1576] When a user enters a search query, the device sends the query to the server and formats and displays the returned data. The display method and content are dynamically changed based on the user's emotions. By determining the user's emotions, for example, if the user expresses anxiety, a more detailed explanation can be provided.

[1577] The device also provides an online chat function, allowing users to input specific questions and send them to the server. The server then uses an emotion engine to recognize the user's emotions, generates appropriate responses, and sends them to the device. This process allows users to obtain specialized information in real time.

[1578] For example, a server can collect videos from the YouTube API using the search keyword "XYZ company earnings briefing" and store them in Google Cloud Storage. The audio from these videos is then converted to text using the Google Cloud Speech-to-Text API, and a generative AI model is used to extract and highlight keywords such as "sales," "year-over-year comparison," and "10% increase." When a user searches for "XYZ company's latest earnings information," the database provides the relevant text data, highlighting the key information.

[1579] As an example of a prompt sentence, if a user inputs "What are the prospects for next year?", the device will send this question to the server, and the emotion engine will recognize the emotion from the user's input. For example, if the user expresses anxiety, detailed explanations and additional information will be provided. By coordinating the server, device, and user, this invention personalizes and streamlines the process of collecting, analyzing, providing, and responding to information.

[1580] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1581] Step 1: Data collection

[1582] The server uses crawlers and APIs to periodically collect video and audio content from companies' financial results briefings and general shareholders' meetings that are publicly available on the Internet. For example, the crawler is used to explore a specific company's YouTube channel, and when new videos are discovered, the links are collected. The collected data includes metadata (company name, date, agenda), which is stored in a database. The input is the company information URL, and the output is the audio and video data and metadata stored in cloud storage.

[1583] Step 2: Speech to text

[1584] The server sends the collected audio data to the Google Cloud Speech-to-Text API and converts the audio to text. For example, send the audio file of a video titled "XYZ Company Earnings Briefing" to the API endpoint to obtain text data. The input is audio data, and the output is text data converted from the audio.

[1585] Step 3: Keyword extraction and information emphasis

[1586] The server uses a generative AI model (such as BERT or GPT-3) to extract important keywords and numbers from text data. For example, it highlights words and numbers such as "sales" and "profit." The input is text data, and the output is text data containing highlighted keywords and numbers.

[1587] Step 4: Store in the database

[1588] The server stores the highlighted text data in a database, indexes it, and makes it searchable. Specifically, it creates a table in an SQL database with fields for "company name," "date," and "keyword" to store the extracted data. The input is the highlighted text data, and the output is a searchable database entry.

[1589] Step 5: Emotion Recognition with the Emotion Engine

[1590] The server analyzes text input from the user using natural language processing technology and recognizes emotions. For example, if a user asks, "What are the prospects for next year?", the emotion engine analyzes the question and generates an emotion label such as "anxiety." The input is the user's text input, and the output is the recognized emotion label.

[1591] Step 6: Providing a User Interface

[1592] The terminal uses frameworks such as React and Flutter to build the user interface, providing a search bar and filter options. Users can search for information by entering the name of a company or industry they are interested in into the search box. The input is the user's search query, and the output is the search results displayed on the interface.

[1593] Step 7: Serving search results

[1594] The device sends the search query entered by the user to the server and formats and displays the returned text data. For example, if you search for "XYZ Company's latest financial results," that data will be displayed on the screen. The input is the search query, and the output is the search results that are displayed to the user.

[1595] Step 8: Online chat function

[1596] When a user uses the online chat function to input a question, the device sends the content to the server. The server uses an emotion engine to analyze the user's emotions, generates the most appropriate answer, and sends it back to the device. For example, if a user asks, "What are the prospects for next year?" and the emotion is determined to be anxiety, a detailed answer will be generated accordingly. The input is the user's question, and the output is a response message.

[1597] This series of steps enables the server, terminal, and user to work together to efficiently collect, analyze, and provide information, enabling optimal support tailored to the user's emotions.

[1598] (Application example 2)

[1599] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1600] Currently, there is no system that efficiently collects and analyzes IR information about listed companies and provides users with real-time, personalized information. Furthermore, there are no services that take user sentiment into consideration when collecting and analyzing this information. As a result, investors and other users are unable to quickly and appropriately obtain the information they need, making it difficult to resolve their concerns and questions.

[1601] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting information on listed companies from the Internet, means for converting the collected audio and video content into text, and means for extracting, highlighting, and summarizing important keywords and figures from the converted text. This enables efficient analysis and storage of the collected information. Furthermore, by including means for recognizing a user's emotions and dynamically changing the method and content of information provision based on the emotions, more user-friendly information can be provided. Furthermore, an online chat function can respond to user questions automatically or with human intervention, helping to alleviate anxiety. Furthermore, by providing additional information and risk explanations during electronic payments based on the user's emotions, a sense of security can be enhanced during investments and transactions.

[1602] "Listed company" refers to a company that is listed on a stock exchange.

[1603] "IR information" refers to the public relations information that companies provide to investors. Generally, this includes financial results briefings, general shareholders' meetings, and information disclosure on official websites.

[1604] "Collecting from the Internet" means automatically obtaining information that is publicly available on the Internet using web crawlers or APIs.

[1605] "Converting audio / video content to text" means converting the audio information in audio or video into text data using a speech recognition API or other technology.

[1606] "Extracting, highlighting, and summarizing keywords and numbers" means using natural language processing technology to select important information from text data and visually highlight or summarize it.

[1607] "Storing in a database and making it searchable" refers to the process of storing the extracted information in a database so that it can be efficiently searched.

[1608] "Recognizing user emotions" means using natural language processing technology or other emotion analysis technology to read emotions from the user's statements and actions.

[1609] The "online chat function" is a function that allows users to ask questions to the system in real time, and the system or a human responds to them.

[1610] "Automated or human-assisted response" refers to a response method that not only provides the user with an automatically generated response from the system, but also includes human operator intervention as needed.

[1611] "Dynamic change of information based on emotion" refers to appropriately changing the display method of information and the content of information provided based on the result of the user's emotion recognition.

[1612] "Providing additional information and risk explanations during electronic payments" means providing additional information and risk explanations during the financial transaction or electronic payment process, taking into consideration the user's feelings.

[1613] This invention relates to a system that efficiently collects and analyzes investor relations information of listed companies, and also realizes personalized information provision and dialogue by recognizing user emotions. This system operates in cooperation with a server, terminals, and users, and by combining an emotion engine in particular, it is possible to provide more accurate information.

[1614] System configuration

[1615] Server Processing

[1616] 1. Data Collection

[1617] The server uses crawlers and APIs to periodically collect video and audio content from financial results briefings and general shareholders' meetings of listed companies that are publicly available on the Internet. The collected data is then stored in cloud storage.

[1618] 2. Speech-to-text

[1619] The server sends the stored voice data to a speech recognition API (e.g., Google Cloud Speech-to-Text API) to generate text data, checks the accuracy of the generated text data, and reprocesses it if necessary.

[1620] 3. Keyword extraction and information emphasis

[1621] The server uses a generative AI model (such as OpenAI's GPT-3) to extract and highlight important keywords and figures from the text data. This analysis uses natural language processing techniques to efficiently extract information.

[1622] 4. Introducing the Emotion Engine

[1623] The server uses an emotion engine (e.g., OpenAI's emotion analysis API) to analyze user input and recognize emotions in real time. This emotion data is also stored in a database.

[1624] 5. Database Operation

[1625] The server stores the processed data in a database and optimizes the index for efficient searching of the database.

[1626] Terminal handling

[1627] 1. Providing a user interface

[1628] Through web and mobile applications, the device provides users with an easy-to-use search interface, allowing them to easily find information that interests them.

[1629] 2. Search function and information provision

[1630] The device sends the search query entered by the user to the server, formats the data returned by the server, and provides it to the user, dynamically changing the way and content of the information displayed based on the user's emotions.

[1631] 3. Chat function

[1632] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server generates an answer appropriate to the user's emotion based on the emotion recognition results and sends it to the device.

[1633] User operations

[1634] 1. Information Search

[1635] Users enter the company name or industry information of interest in the search box and click the search button.

[1636] 2. Viewing Information

[1637] The user reviews the search results provided and selects information of particular interest.

[1638] 3. Chat questions

[1639] When a user wants to know more about a specific piece of information, they can use the chatbot to input a question, such as "What is the outlook for next year?" The device will recognize the user's emotions and provide an appropriate answer based on the nuances of the question.

[1640] Specific examples

[1641] For example, suppose a video of a company's earnings briefing is posted online. The server crawls and retrieves the video and stores it in cloud storage. Next, a speech recognition API is used to convert the audio data into text, which is then analyzed by a generative AI model to extract and highlight important keywords and figures, such as sales and profits. This data is then stored in a database. When a user searches for "latest financial information" on their device, the database provides the relevant text data, allowing the user to easily check important information. Similarly, if a user asks "What are the outlook for next year?" via chat, the emotion engine recognizes the emotion from the user's input and generates and delivers an answer in an appropriate tone. Even if the user expresses anxiety, more detailed explanations or supplementary information about the data's context can be added.

[1642] Prompt Sentence Examples

[1643] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[1644] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[1645] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1646] The following are specific processing steps of the system that realizes the application example.

[1647] Step 1:

[1648] Data collection

[1649] The server uses a crawler and API to collect video and audio content from listed companies' financial results briefings and general shareholders' meetings from the Internet. Specifically, the crawler accesses specific websites and retrieves the URLs of the video and audio files. The input is the URL set in the crawler, and the output is the collected video and audio files.

[1650] Step 2:

[1651] Speech to text

[1652] The server sends the collected voice data to the Google Cloud Speech-to-Text API and converts it into text data. The input is the collected voice data, and the output is the data converted from voice to text. The server converts the voice data into binary format, sends it to the API, and saves the returned text data.

[1653] Step 3:

[1654] Keyword extraction and information emphasis

[1655] The server uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords and numbers from text data and highlight them. The input is speech-to-text data, and the output is text data with the extracted keywords and numbers highlighted. Specific operations are performed by the server sending prompts to the generative AI model and receiving the analysis results.

[1656] Example prompt sentence:

[1657] Analyze text to extract important keywords: A company saw a 20% increase in sales this year.

[1658] Step 4:

[1659] Database storage and index optimization

[1660] The server stores the extracted text data, including keywords and numbers, in a database and optimizes the index for efficient searches. The input is text data with keywords and numbers highlighted, and the output is indexed data stored in the database. The specific operations are to insert data into the database and rebuild the index.

[1661] Step 5:

[1662] Introducing the Emotion Engine

[1663] The server uses the emotion engine to analyze user inputs and search queries in real time and recognize emotions. The input is text input from the user, and the output is the emotion recognition result. Specific operations include sending text data to the emotion engine and storing the emotion analysis result in a database.

[1664] Example prompt sentence:

[1665] Analyzing user sentiment shows: I'm feeling pretty anxious about investing.

[1666] Step 6:

[1667] Providing a user interface

[1668] The terminal provides an easy-to-use search interface through a web application or a mobile application, where the input is a user's search query and the output is parsed data obtained from the server. The specific operation includes the user entering a query into a search box and sending the query to the server.

[1669] Step 7:

[1670] Search function and information provision

[1671] The terminal transmits the search query entered by the user to the server and displays the data returned by the server, where the input is the user's search query and the output is the data returned by the server. Specific operations include formatting the data received from the server and displaying it to the user.

[1672] Step 8:

[1673] Providing chat functionality and emotional responses

[1674] The device provides an online chat function, and when a user inputs a question, the content is sent to the server. The server uses an emotion engine to analyze the user's emotions and generate and provide an appropriate answer. The input is the user's question and emotional state, and the output is an appropriate answer returned by the server. Specific operations include generating an answer using a generative AI model based on the emotion recognition results and sending it to the device.

[1675] Step 9:

[1676] Providing additional information when making electronic payments

[1677] The server provides additional information and risk explanations during electronic payments based on the user's emotions. The input is the user's payment request and emotional state, and the output is a transaction screen containing additional information and risk explanations. The specific operation is to refer to the emotion recognition results and provide the necessary information to the user.

[1678] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1679] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1680] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1681] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1682] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1683] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1684] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1685] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1686] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1687] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1688] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1689] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1690] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1691] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1692] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1693] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1694] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1695] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1696] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1697] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1698] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1699] The following is further disclosed regarding the above embodiment.

[1700] (Claim 1)

[1701] [Means of collecting information on listed companies from the Internet and

[1702] [Means for converting collected audio and video content into text;

[1703] [Methods for extracting, highlighting, and summarizing important keywords and numbers from text data,

[1704] [Means for storing the extracted information in a database and making it searchable; and

[1705] [means for providing the text data and extracted information to a user's terminal;

[1706] [A system that includes a means for responding to user questions via an online chat function, either automatically or with human intervention.

[1707] (Claim 2)

[1708] [The system of claim 1, wherein the collected data is stored in cloud storage.

[1709] (Claim 3)

[1710] [The system of claim 1, which uses a generative AI model to process text data and extract important keywords and numbers.

[1711] "Example 1"

[1712] (Claim 1)

[1713] [Means for collecting information on aggregating entities from a communication network;

[1714] [Means for converting collected audio and video into text information;

[1715] [Means for extracting, highlighting, and summarizing key identifiable elements and numbers from the converted text information;

[1716] [means for storing the extracted information in a storage device and making it searchable;

[1717] [Means for providing character information and extracted information to a user's information terminal;

[1718] [A system that includes a means for responding to user questions through online interactive functions, either automatically or with human intervention.

[1719] (Claim 2)

[1720] [The system of claim 1, wherein the collected data is stored in a remote storage device.

[1721] (Claim 3)

[1722] [The system of claim 1 uses the generated AI model to process text information and extract key identifiable elements and numbers.

[1723] "Application Example 1"

[1724] (Claim 1)

[1725] [Means of collecting information on listed companies from the Internet and

[1726] [Means for converting collected audio and video content into text;

[1727] [Methods for extracting, highlighting, and summarizing important keywords and numbers from text data,

[1728] [Means for storing the extracted information in a database and making it searchable; and

[1729] [means for providing the text data and extracted information to a user's terminal;

[1730] [Means for responding to user questions through an online chat function, either automatically or with human intervention;

[1731] [A system including means for displaying IR information in real time on a smart device and responding to a user's voice commands.

[1732] (Claim 2)

[1733] [The system of claim 1, wherein the collected data is stored in cloud storage.

[1734] (Claim 3)

[1735] [The system of claim 1, which uses a generative AI model to process text data and extract important keywords and numbers.

[1736] "Example 2: Combining Emotion Engines"

[1737] (Claim 1)

[1738] [Means of collecting company information from the Internet and

[1739] [Means for converting collected audio and video content into text;

[1740] [Methods for extracting and highlighting important keywords and numbers from text data,

[1741] ...

Claims

1. A means of collecting information on listed companies from the Internet; A means of converting collected audio and video content into text, A method to extract, highlight, and summarize important keywords and numbers from text data, a means for storing the extracted information in a database and making it searchable; means for providing the text data and extracted information to a user terminal; A system that includes a means for responding to user questions through an online chat function, either automatically or with human intervention.

2. The system of claim 1 , wherein the collected data is stored in cloud storage.

3. The system of claim 1, wherein the system processes text data using a generative AI model to extract important keywords and numbers.

Images