system

Abstract

We provide the system. [Solution] Means for collecting data, A means of filtering the collected data and removing personal information and copyrighted information, A means of generating characteristics from filtered data, A means for selecting and generating sample data based on characteristics, A means of publishing the generated characteristic data and sample data, A means of collecting user ratings and feedback, A system that includes means for improving the dataset and the system based on collected feedback.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the training data of large-scale language models, there is a problem that the transparency of the data is insufficient, making it difficult for users to evaluate the quality and reliability of the models. Also, the training data may contain copyrights and personal information, which may lead to legal risks and privacy violations. Furthermore, there are risks such as fake news created by generative AI and data corruption. There is a need for a method to solve these problems and support the development of reliable AI models.

Means for Solving the Problems

[0005] The present invention provides a system that includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, and means for improving the dataset and system based on the collected feedback. This allows users to verify the transparency of the model's training data and improve the reliability of the model through evaluation. Furthermore, appropriate filtering can avoid copyright and personal information protection issues. In addition, improvements based on user feedback can continuously improve the quality of the dataset.

[0006] "Means of data collection" refers to the functions and algorithms used to collect articles and information from news sites and databases on the internet.

[0007] "Filtering methods" refer to natural language processing algorithms and other filtering functions used to remove or anonymize personal information and copyrighted information from collected data.

[0008] "Means for generating characteristics" refers to a function that analyzes filtered data and extracts features such as data genre, era, author, and keywords.

[0009] "Means for selecting and generating sample data" refers to a function that selects representative data based on the generated characteristic data, converts it into an appropriate format, and provides it.

[0010] "Means of publication" refers to functions for making characteristic data and sample data available to users through web pages, APIs, etc.

[0011] "Means for collecting evaluations and feedback" refers to features that allow users to view publicly available characteristic data and sample data and provide evaluations and feedback on them.

[0012] "Means for improving datasets and systems" refers to functions that analyze collected feedback and improve the dataset and the entire system based on the results. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the language used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption caused by generative AI, while taking into consideration copyright and personal data protection issues.

[0035] Overview of program processing

[0036] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and uses it to improve the dataset and system.

[0037] Program description in natural language

[0038] Data collection and filtering

[0039] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly. The collected data is temporarily stored and then filtered. Filtering uses natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images). This process helps to avoid legal issues.

[0040] Data property generation

[0041] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0042] Generating sample data

[0043] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0044] Disclosure of samples and characteristics

[0045] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0046] Collecting user ratings and feedback

[0047] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments. The server collects these evaluations and feedback and performs analysis. The analysis results are used to improve the dataset and the system based on the collected evaluation data. User feedback is reflected in subsequent data collection and filtering processes, continuously improving the reliability and quality of the model.

[0048] The following describes the processing flow.

[0049] Step 1: Data Collection

[0050] The server collects the latest articles and information from news sites and various databases on the internet. A script for data collection is executed periodically, retrieving data from specified URLs and APIs. The collected data is stored in temporary storage.

[0051] Step 2: Filtering

[0052] The server applies natural language processing algorithms to the temporarily stored data to detect personal information (e.g., name, address, phone number) and copyrighted information. Detected personal information and copyrighted information are deleted or anonymized. The filtered data is stored for use in the next step.

[0053] Step 3: Generating Data Characteristics

[0054] The server generates characteristics from the filtered data. Specifically, it extracts features such as article genre (politics, economics, sports, etc.), age, author, and keywords. The generated characteristic data is compiled as statistical information and stored in a database.

[0055] Step 4: Selecting Sample Data

[0056] The server selects representative sample data based on the generated characteristic data. Random sampling is performed according to specific criteria, and appropriate samples are selected from each genre and characteristic. The selected sample data is stored.

[0057] Step 5: Generating sample data

[0058] The server formats the selected sample data and prepares it as a publicly available dataset. The sample data, along with its characteristic information, is converted into an appropriate format such as JSON or XML.

[0059] Step 6: Disclosure of Samples and Characteristics

[0060] The server publishes characteristic data and sample data through a configured web page or API endpoint. This data is made publicly available on the internet so that users can access it using their devices.

[0061] Step 7: Collecting user ratings and feedback

[0062] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores and comments and are sent to the server via online forms or APIs.

[0063] Step 8: Feedback analysis and system improvement

[0064] The server analyzes the collected feedback to identify areas for improvement in the dataset and filtering algorithms. Based on the analysis results, the next data collection and filtering processes are adjusted to improve overall system performance and reliability.

[0065] (Example 1)

[0066] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0067] Traditional large-scale language models have suffered from a lack of transparency, making it difficult for users to verify the quality and reliability of training data. Furthermore, there are risks of copyright infringement and personal data leaks from collected data, increasing the risk of fake news spread and data corruption through generative AI. Additionally, the lack of mechanisms to effectively collect user feedback and incorporate it into system improvements presents challenges in enhancing the reliability and quality of the models.

[0068] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0069] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for periodically processing the collected data, and means for storing the collected evaluations and feedback in a database. This improves data transparency and reliability, and enables continuous improvement of the system by utilizing user feedback while taking into consideration copyright and personal information protection.

[0070] "Means of data collection" refers to methods for regularly obtaining data from information sources on the internet.

[0071] "Means for removing personal information and copyrighted information" refers to means for detecting personal information such as names and addresses, as well as copyrighted information, from collected data, and for removing or anonymizing them.

[0072] "Means for generating characteristics" refers to methods for automatically extracting characteristic information such as genre, era, and keywords from filtered data.

[0073] "Means for selecting and generating sample data" refers to means for selecting representative sample data based on the generated characteristics and formatting it.

[0074] "Means for publishing characteristic data and sample data" refers to means for publishing the generated characteristic data and sample data through web pages or APIs.

[0075] "Means for collecting user ratings and feedback" refers to means for collecting user ratings and feedback on publicly available data.

[0076] "Means for improving datasets and systems" refers to means for improving datasets and systems based on collected feedback.

[0077] "Means for periodically processing collected data" refers to means of regularly performing data collection and processing to ensure that the latest information is always available.

[0078] "Means for storing collected ratings and feedback in a database" refers to means for storing ratings and feedback collected from users in a database and making them available for subsequent processing.

[0079] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption generated by AI, while also considering copyright and personal data protection. This system is implemented with the following configuration and procedure.

[0080] 1. Data Collection

[0081] The server periodically collects data from news sites and various databases. For this purpose, it uses scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, when collecting the latest political news from a specific news site, the server uses "BeautifulSoup" to analyze the news site's HTML and extract article titles and body text.

[0082] 2. Data filtering

[0083] The collected data is temporarily stored, and the server uses natural language processing libraries such as "Apache® OpenNLP" to filter out personal and copyrighted information. For example, it detects and deletes or anonymizes personal information such as names and addresses. It also detects and deletes or anonymizes copyrighted text and images.

[0084] 3. Generating Data Characteristics

[0085] Once the filtering is complete, the server generates characteristics from the data. This is done using tools like "Scikit-learn" and "NLTK". For example, it automatically classifies the genre of articles (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. The characteristics are displayed as statistical information, distributed as follows: "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%".

[0086] 4. Generating sample data

[0087] Based on its characteristics, the server selects representative sample data and formats it using tools like "Pandas" or "Numpy." For example, when creating training data for a generative AI model, it includes a balanced mix of articles from diverse genres based on specific criteria.

[0088] 5. Data Disclosure

[0089] The server uses web frameworks such as "Flask" or "Django" to expose the generated characteristic data and sample data as web pages or APIs. For example, the server uses "Flask" to build a web page to display the characteristic data and sample data, and sets up an API endpoint so that the data can be accessed from programs as well.

[0090] 6. Collecting user ratings and feedback

[0091] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection.

[0092] As a concrete example, to create training data for a generative AI model and to verify the transparency and reliability of the data, one might consider inputting prompt statements like the following into the generative AI model.

[0093] Example of a prompt:

[0094] "Collect the latest political news and remove personal information such as names and addresses. Then, extract characteristics such as the genre, year, and author of the articles and display them as statistical information."

[0095] This makes it possible to quickly build transparent and reliable datasets. This process ensures that the training data for generative AI models is of high quality and reliable, effectively reducing the risk of fake news and data collapse.

[0096] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0097] Step 1: Data Collection

[0098] The server collects the latest articles and information from news sites and various databases. The collected data includes text data, image data, etc. This step utilizes scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, the server uses "BeautifulSoup" to parse the HTML of news sites and extract article titles and body text. A list of URLs is provided as input, and the collected raw data (HTML) is obtained as output.

[0099] Step 2: Data Filtering

[0100] The server temporarily stores the collected data and then performs a filtering process. This step uses natural language processing libraries such as "Apache OpenNLP". The server detects personal information and copyrighted information and removes or anonymizes them. Specifically, the server uses "Apache OpenNLP" to detect and remove personal information such as names and addresses. It also detects copyrighted text and images and removes or anonymizes them as well. Raw data collected is provided as input, and filtered, clean data is obtained as output.

[0101] Step 3: Generating Data Characteristics

[0102] The server generates characteristics from the filtered data. This step uses libraries such as "Scikit-learn" and "NLTK". For example, the server automatically classifies article genres (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. Specifically, the server uses "Scikit-learn" to classify article genres and "NLTK" to extract keywords, dates, and author information. Filtered data is provided as input, and the generated characteristic data is obtained as output.

[0103] Step 4: Generating sample data

[0104] The server selects sample data based on the generated characteristics. This step uses data manipulation libraries such as "Pandas" and "Numpy." For example, the server uses "Pandas" to apply characteristic-based filters and select representative sample data. Then, it uses "Numpy" to format the data and build a dataset for publication. Characteristic data is provided as input, and the generated sample data is obtained as output.

[0105] Step 5: Data Release

[0106] The server uses web frameworks such as Flask or Django to expose the generated characteristic data and sample data as web pages or APIs. For example, the server might use Flask to build a web page to display the characteristic data and sample data, and configure an API endpoint to allow programmatic access to the data. Sample data is provided as input, and the exposed web pages or API endpoints are obtained as output.

[0107] Step 6: Collecting user ratings and feedback

[0108] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection. User evaluations and feedback are provided as input, and the collected evaluation data and analysis results are obtained as output.

[0109] (Application Example 1)

[0110] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] In today's information society, fake news and data reliability are major problems. In particular, the quality and transparency of the source data are crucial when generating information using large-scale language models. Traditional methods lacked systems to enhance data transparency and enable users to evaluate reliability, making it difficult to handle data while considering personal information protection and copyright protection. Furthermore, there was a lack of means to effectively utilize user feedback to improve datasets and systems. In addition, there were no established methods for evaluating the quality of news data, anonymization, or publishing the generated data. This led to problems such as a decline in the accuracy of information and user trust.

[0112] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0113] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for evaluating the quality of news data based on the feedback, means for using a large-scale language model to generate article summaries, and means for storing and publishing the anonymized news data on the server. This makes it possible to enhance data transparency, provide a system that allows users to evaluate reliability, and improve the quality of news data while considering the protection of personal information and copyrights.

[0114] "Means of data collection" refers to the function of obtaining information from various information sources on the internet.

[0115] "Filtering" refers to the process of removing or anonymizing personal information or copyrighted material from collected data.

[0116] "Means for generating characteristics" refers to a function that extracts features such as genres and keywords from filtered data and forms characteristic information about the data.

[0117] "Means for selecting and generating sample data" refers to a function that selects and generates representative sample data based on its characteristics.

[0118] "Means for publishing generated characteristic data and sample data" refers to a function that makes characteristic data and sample data publicly available through a web page or API so that users can view them.

[0119] "Means for collecting user ratings and feedback" refers to functions for collecting ratings and opinions provided by users.

[0120] "Means for improving datasets and systems" refers to functions that improve datasets and systems based on collected feedback.

[0121] "Means for evaluating the quality of news data" refers to a function for evaluating the reliability and quality of news data based on feedback.

[0122] "Methods using large-scale language models" refer to algorithms that use natural language processing techniques to summarize and classify news articles.

[0123] "Means for storing and publishing news data generated through anonymization processing on a server" refers to a function that deletes or anonymizes personal information or identifiable information, then stores the news data on a server and makes it available for user access.

[0124] A specific embodiment of this invention will now be described. This system is implemented as a smartphone application called "News Reader" that allows users to view the latest news and evaluate its reliability and quality.

[0125] System Configuration

[0126] 1. Data acquisition methods

[0127] The server automatically collects information from various sources on the internet. This collection process utilizes web scraping techniques and the Requests library. Data sources include news sites and public databases.

[0128] 2. Filtering means

[0129] The server filters the collected data, removing or anonymizing personal information and copyrighted material. This filtering uses natural language processing algorithms (e.g., BeautifulSoup and the re library). The filtered data is temporarily stored in a database.

[0130] 3. Characteristic generation means

[0131] The server generates characteristics from the filtered data. These characteristics include article genres (e.g., politics, economics, sports), keywords, author names, and dates. A machine learning model (e.g., HuggingFace's Transformers library) is used for this characteristic generation.

[0132] 4. Sample data selection and generation methods

[0133] Representative sample data is selected and generated based on its characteristics. Random sampling techniques or conditional selection algorithms are used to select samples based on specific criteria. The selected data is formatted and prepared for publication.

[0134] 5. Means for publishing characteristic data and sample data

[0135] The server publishes characteristic and sample data via a webpage or API. Users can access this information using the smartphone app "News Reader." The data is published in JSON format and visualized for easy user understanding.

[0136] 6. Means of collecting user evaluations and feedback

[0137] Users can view published data through the "News Reader" and provide ratings and feedback. Ratings can be given in the form of stars or comments, and the collected feedback is stored in a database.

[0138] 7. Dataset and System Improvement Means

[0139] The server analyzes the collected feedback and uses it to improve the dataset and the system. This analysis employs statistical methods and machine learning algorithms.

[0140] 8. Methods for evaluating the quality of news data

[0141] The quality of news data is evaluated based on user feedback. Evaluation criteria include article length, keyword score, and user confidence. Data analysis tools are used to assess data quality.

[0142] 9. Methods using large-scale language models

[0143] The server uses a large-scale language model (e.g., HuggingFace's Transformers) to summarize and classify news articles. This model generates article summaries, allowing users to quickly grasp the content.

[0144] 10. Anonymization and Data Disclosure Methods

[0145] The data is anonymized, stored on a server, and then made public. The anonymization process uses filtering algorithms to remove personal and identifying information.

[0146] Examples of specific cases and prompt statements

[0147] For example, if an article contains the sentence, "John Smith has moved to a new address, 123 Main St," this text will be filtered to "John Smith has moved to a new address." Furthermore, summarizing this sentence would result in "John is moving to a new residence."

[0148] Examples of prompts for a generative AI model:

[0149] Please summarize the following news article:

[0150] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0151] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0152] Step 1:

[0153] Data collection

[0154] The server automatically collects data from news sites and public databases on the internet. Input is web article URLs or RSS feeds, and output is the text data of the web pages. Specifically, it uses the Requests library to retrieve the HTML content of web pages and BeautifulSoup to extract the text portion of the articles.

[0155] Step 2:

[0156] filtering

[0157] The server filters out personal and copyright-protected information from the collected text data. The input is the text data of the web page obtained in step 1, and the output is the filtered text data. Specifically, it uses Natural Language Processing (NLP) algorithms and regular expressions (re library) to detect and remove or anonymize personal information such as names and addresses.

[0158] Step 3:

[0159] Characteristic generation

[0160] The server generates characteristics from the filtered data. The input is the text data filtered in step 2, and the output is characteristic information (e.g., genre, keywords, publication year, etc.). Specifically, a machine learning model (HuggingFace's Transformers library) is used to extract keywords and classify genres.

[0161] Step 4:

[0162] Selection and generation of sample data

[0163] The server selects and generates representative sample data based on the characteristics. The input is the characteristic information and filtered text data generated in step 3, and the output is the selected and generated sample data. Specifically, random sampling techniques and condition-based selection algorithms are used to select data that matches the characteristics.

[0164] Step 5:

[0165] Disclosure of sample data and characteristic information

[0166] The server publishes sample data and characteristic information. The input is the sample data and characteristic information generated in step 4, and the output is a data file (in JSON format) for publication. Specifically, you configure the server or API to allow users to access this data.

[0167] Step 6:

[0168] Collecting user ratings and feedback

[0169] Users view the published sample data and characteristic information and provide ratings and feedback. The input is the data published in step 5, and the output is user ratings and feedback information. Specifically, star ratings and text comments are collected using the rating form and comment function within the app.

[0170] Step 7:

[0171] Dataset and system improvements

[0172] The server analyzes the collected feedback and uses it to improve the dataset and system. The input is the user ratings and feedback information collected in step 6, and the output is the improved dataset and system settings. Specifically, statistical analysis tools and machine learning algorithms are used to analyze the feedback and reflect it in the next data collection and filtering process.

[0173] Step 8:

[0174] Quality assessment of news data

[0175] The server evaluates the quality of the news data based on feedback. The input is the user ratings and feedback information collected in step 6, and the output is the evaluation result. Specifically, it calculates factors such as article length, keyword score, and user confidence to generate an overall quality score.

[0176] Step 9:

[0177] Summarization and classification using large-scale language models

[0178] The server uses a generative AI model (HuggingFace's Transformers) to generate summaries of news articles. The input is the collected and filtered news article text, and the output is the summarized text. Specifically, a specified prompt sentence is input to the generative AI model to obtain a concise summary.

[0179] Example of a prompt:

[0180] Please summarize the following news article:

[0181] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0182] Step 10:

[0183] Anonymization and data disclosure

[0184] The server stores and publishes anonymized news data. The input is filtered news article text, and the output is anonymized text and a data file for publication. Specifically, a filtering algorithm is used to remove or mask personal information and identifying information, and a data file (in JSON format) for publication is generated and stored on the server.

[0185] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0186] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0187] Overview of program processing

[0188] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and analyzes user emotions using a sentiment engine. Based on these analysis results, improvements are made to the dataset and system.

[0189] Program description in natural language

[0190] Data collection and filtering

[0191] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly, using specific scripts to retrieve data from designated URLs and APIs. The collected data is stored in temporary storage. Next, a filtering process is performed, using natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images).

[0192] Data property generation

[0193] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0194] Generating sample data

[0195] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0196] Disclosure of samples and characteristics

[0197] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0198] Collecting user ratings and feedback

[0199] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0200] Analysis using an emotion engine

[0201] The server applies an emotion engine to the collected ratings and feedback to analyze the user's emotions. This emotion analysis identifies the emotions behind the feedback and classifies them, for example, as "Positive: 80%, Negative: 20%." The emotion engine uses text analysis and machine learning algorithms to accurately analyze emotions.

[0202] Feedback analysis and system improvement

[0203] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, subsequent data collection and filtering processes are adjusted to improve overall system performance and reliability. For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm is introduced to resolve the issue.

[0204] The following describes the processing flow.

[0205] Step 1: Data Collection

[0206] The server periodically runs scripts to collect the latest articles and information from news sites and various databases. It retrieves data from specified URLs and APIs, and stores the collected data in temporary storage.

[0207] Step 2: Filtering

[0208] The server applies natural language processing algorithms to the temporarily stored data. The algorithms detect personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images), and delete or anonymize the detected information. The filtered, secure data is then sent to the next step.

[0209] Step 3: Generating Data Characteristics

[0210] The server generates characteristics from the filtered data. In this step, features such as article genre (politics, economics, sports, etc.), age group, author, and keywords are extracted. The characteristic data is compiled as statistical information, and the distribution is displayed, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%." The generated characteristic data is stored in a database.

[0211] Step 4: Selecting Sample Data

[0212] The server selects representative sample data based on the generated characteristic data. This selection includes random sampling and sampling based on specific criteria. The sample data is formatted and stored along with the characteristic information.

[0213] Step 5: Generating sample data

[0214] The server converts sample data selected based on characteristic information into an appropriate format (e.g., JSON, XML). This prepares the sample dataset for public release.

[0215] Step 6: Disclosure of Samples and Characteristics

[0216] The server configures a web page or API endpoint to expose characteristic data and sample data. This allows users to access this information using their devices and verify the data's transparency.

[0217] Step 7: Collecting user ratings and feedback

[0218] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0219] Step 8: Analysis using the emotion engine

[0220] The server applies a sentiment engine to the collected ratings and feedback. The sentiment engine uses text analysis and machine learning algorithms to analyze the user's emotions (positive, negative, neutral, etc.) and generate sentiment data.

[0221] Step 9: Feedback Analysis and System Improvement

[0222] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, it adjusts the next data collection and filtering process to improve the overall system performance and reliability. For example, if a large number of negative sentiments are reported, it identifies the underlying data or algorithmic problems and implements improvements.

[0223] (Example 2)

[0224] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0225] Modern large-scale language models face numerous concerns regarding the transparency and quality of training data. These include issues such as the leakage of personal information, copyright infringement, and even the generation of fake news. Furthermore, analyzing user feedback and incorporating it into system improvements presents a significant challenge. New approaches are needed to effectively address these problems.

[0226] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0227] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, and means for performing sentiment analysis on the collected feedback.

[0228] This will allow us to protect personal information and copyrights while improving data transparency and quality, and to efficiently analyze user feedback to improve the system.

[0229] "Data collection" is the process of automatically obtaining articles and information from news sites, databases, and other sources on the internet.

[0230] "Data filtering" is the process of detecting and deleting or anonymizing personal information and copyrighted material from collected data.

[0231] "Natural language processing" is a general term for the technologies and methods that enable computers to understand and analyze human language.

[0232] "Characteristic generation" is the process of extracting features such as article genre, year, author, and keywords from filtered data and compiling them into statistical information.

[0233] "Sample data generation" is the process of selecting representative data based on the generated characteristics and formatting it for public release.

[0234] "Data publication" refers to the process of providing generated characteristic data and sample data to users through web pages or APIs.

[0235] "Collecting user ratings and feedback" is the process of collecting information from users who provide ratings and comments on publicly available characteristic data and sample data.

[0236] "Sentiment analysis" is the process of using an emotion engine to analyze the emotions behind collected user ratings and feedback.

[0237] "System improvement" is the process of improving datasets and filtering algorithms based on collected feedback and sentiment analysis results.

[0238] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0239] Overview of program processing

[0240] The server collects data from news sites and databases on the internet, and filters the collected data to remove or anonymize personal information and copyrighted material. This filtering uses web scraping libraries such as BeautifulSoup and Scrapy in Python, as well as the entity recognition function of spaCy. After filtering, characteristics are generated from the data using techniques such as topic modeling (LDA) and keyword extraction (TF-IDF), and sample data is selected and generated based on these characteristics.

[0241] The generated sample and characteristic data are published via web pages and APIs using web frameworks such as Django and Flask. Users can access this information using their devices and verify the data's transparency. Evaluations and feedback are provided via online forms and APIs.

[0242] The server collects user feedback and analyzes user emotions using an emotion engine (such as a BERT model or fastText). Based on this analysis, the dataset and filtering algorithm are improved. For example, the filtering algorithm is adjusted based on feedback that contains a lot of negative emotions, and this is reflected in the next data collection.

[0243] Specific example

[0244] As a concrete example, input the following prompt into the generation AI model.

[0245] Example of a prompt:

[0246] "Collect the latest news articles on politics and economics, and remove any personal information or copyrighted material. Then, extract the article genres and keywords as characteristics, and generate three representative sample data based on these characteristics. Finally, publish the generated data on a webpage and collect and analyze sentiment feedback from users."

[0247] Based on this prompt, the system executes specific steps such as data collection, filtering, characteristic generation, sample data generation, data publication, evaluation collection, and sentiment analysis. This allows for system improvement by effectively utilizing user feedback while maintaining data transparency and quality.

[0248] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0249] The program's processing is explained step by step.

[0250] Step 1: Data Collection

[0251] explanation

[0252] The server collects the latest articles and information from news sites and databases on the internet. This task is performed periodically using specific scripts (e.g., BeautifulSoup or Scrapy in Python).

[0253] Specific actions

[0254] The server accesses a specified list of URLs or API endpoints, parses the HTML structure of the page, and retrieves the necessary data. For example, it can extract the titles and content of the latest articles from a news website's URL.

[0255] Input and output

[0256] Input: A specified list of URLs or API endpoints

[0257] Output: Collected raw HTML or JSON data

[0258] Step 2: Data Filtering

[0259] explanation

[0260] The server detects and removes or anonymizes personal information and copyrighted material from the collected data. Natural language processing (NLP) algorithms are used for this process.

[0261] Specific actions

[0262] For example, spaCy's entity recognition feature is used to identify personal names and addresses. If a name or address is detected in the data, it is replaced with "[ANONYMIZED]".

[0263] Input and output

[0264] Input: Collected raw data (HTML or JSON)

[0265] Output: Text data with personal information and copyright protection information removed or anonymized.

[0266] Step 3: Generating Data Characteristics

[0267] explanation

[0268] The server generates characteristics such as article genre, year, author, and keywords from the filtered data.

[0269] Specific actions

[0270] For example, to classify the genre of articles, techniques such as topic modeling (LDA) and keyword extraction (TF-IDF) are used. Characteristic data is generated, such as the fact that political articles account for 40% of the total.

[0271] Input and output

[0272] Input: Filtered text data

[0273] Output: Characteristic data such as genre, era, author, and keywords.

[0274] Step 4: Generation of Sample Data

[0275] Description

[0276] The server selects and formats representative sample data based on characteristics.

[0277] Specific Operations

[0278] For example, sample data is selected using techniques such as random sampling based on the ratio of specific characteristics. This sample data is ultimately formatted for publication.

[0279] Input and Output

[0280] Input: Characteristic Data

[0281] Output: Sample Data Set

[0282] ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0290] Step 6: Collecting user ratings and feedback

[0291] explanation

[0292] Users use their devices to view publicly available characteristic data and sample data, and provide evaluations and feedback on them. This information is sent to the server.

[0293] Specific actions

[0294] For example, you could place a rating form on a webpage, allowing users to enter scores and comments.

[0295] Input and output

[0296] Input: User-submitted ratings and feedback

[0297] Output: Evaluation and feedback data stored on the server

[0298] Step 7: Analysis using the emotion engine

[0299] explanation

[0300] The server uses an emotion engine to analyze collected ratings and feedback and classify the user's emotions.

[0301] Specific actions

[0302] For example, sentiment engines such as the BERT model or fastText are used to classify each feedback comment as either "positive" or "negative."

[0303] Input and output

[0304] Input: Rating and feedback data collected from users

[0305] Output: Sentiment analysis result (e.g., "Positive: 80%, Negative: 20%")

[0306] Step 8: Analysis of feedback and system improvement

[0307] Explanation

[0308] The server improves the dataset and filtering algorithm based on the feedback results including sentiment analysis.

[0309] Specific operations

[0310] For example, if articles with which many users have negative feelings fail in filtering, a new algorithm to solve the problem is implemented and applied from the next data collection.

[0311] Input and output

[0312] Input: Sentiment analysis result and feedback data

[0313] Output: Improved filtering algorithm and dataset

[0314] By repeating these processing steps, the performance and reliability of the entire system are improved, and data transparency and user experience optimization are achieved.

[0315] (Application Example 2)

[0316] Next, Application Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0317] In modern large-scale language model training data, data transparency and reliability are not sufficiently ensured, making it difficult for users to evaluate data quality. Furthermore, in addition to copyright and personal data protection issues, there is a growing risk of fake news generation and data corruption. Moreover, the lack of collection and analysis of feedback that considers user sentiment leads to insufficient improvement of datasets and systems. A new system is needed to address these challenges.

[0318] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0319] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for analyzing user sentiment based on the evaluations and feedback, means for improving the dataset and system based on the sentiment analysis results, means for evaluating reliability, and means for improving the dataset using the evaluation results. This increases the transparency of training data for large-scale language models, allows users to evaluate the quality and reliability of the data, reduces the risk of fake news and data collapse, and improves the overall system.

[0320] "Means of data collection" refers to the function of obtaining data from various information sources on the internet.

[0321] "Means for filtering data and removing personal and copyrighted information" refers to functions that detect and remove or anonymize personal and copyrighted information contained in collected data.

[0322] "Methods for generating characteristics from filtered data" refers to functions that extract characteristics such as article genre, author name, and keywords from filtered data, and analyze the characteristics of the data.

[0323] "Means for selecting and generating sample data based on characteristics" refers to a function for selecting representative data based on the generated characteristics, formatting that data, and providing it as a sample dataset.

[0324] "Means for publishing generated characteristic data and sample data" refers to functions for making characteristic data and sample data publicly available through web pages or APIs.

[0325] "Means for collecting user ratings and feedback" refers to functions for collecting user ratings and feedback on publicly available data via online forms or APIs.

[0326] "Means for analyzing user emotions based on evaluations and feedback" refers to a function that performs sentiment analysis on collected feedback and evaluations to classify users' positive and negative emotions.

[0327] "Means for improving datasets and systems based on sentiment analysis results" refers to functions that improve datasets and system algorithms based on information obtained from sentiment analysis results, thereby improving the overall performance of the system.

[0328] A "means for evaluating reliability" refers to a function that analyzes user evaluations of publicly available data and its characteristics, and evaluates the reliability of that data.

[0329] "Means for improving datasets using evaluation results" refers to functions that improve the quality of datasets by reviewing the data filtering process and sample data selection method based on the analysis results of user evaluations.

[0330] This invention specifically illustrates a form of quality evaluation application for e-commerce websites. This system has functions for data collection, filtering, characteristic generation, sample data generation, publication, evaluation, and feedback collection and analysis.

[0331] Data acquisition methods

[0332] The server collects data from news sites and various databases on the internet. This collection process is performed periodically, and data is retrieved from specified URLs and APIs using specific scripts. Libraries such as BeautifulSoup and requests can be used for this process.

[0333] Filtering means

[0334] To remove personal information and copyrighted material from collected data, the server uses natural language processing algorithms. Collected data is stored in temporary storage, where natural language processing algorithms are used to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted material. This ensures data integrity and privacy.

[0335] Characteristic generation means

[0336] From the filtered data, the server extracts characteristics such as article genre, author name, and keywords. This characteristic generation process uses text analysis algorithms and machine learning models. For example, characteristics distributed as "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%" are displayed as statistical information.

[0337] Method for generating sample data

[0338] Based on characteristic data, the server selects representative sample data. This selection includes random sampling and selection based on specific criteria. The selected sample data is verified and formatted along with characteristic information. This dataset is then prepared for public release.

[0339] means of disclosure

[0340] The server configures a webpage or API to publish characteristic and sample data. Users can access this information using their devices and verify the data's transparency. For example, users can view product descriptions and reviews and evaluate their quality.

[0341] User evaluation and feedback collection methods

[0342] Users evaluate publicly available characteristic data and sample data and provide feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs. Evaluation feedback forms or survey systems can be used for this data collection process.

[0343] Emotion analysis means

[0344] The server applies a sentiment engine to the collected ratings and feedback to analyze the user's emotions. Using text analysis and machine learning algorithms, it identifies the emotions behind the feedback and classifies them, such as "Positive: 80%, Negative: 20%". This sentiment analysis utilizes the pipeline function from the Hugging Face Transformers library.

[0345] Dataset and system improvement means

[0346] The server identifies areas for improvement in the dataset and system based on feedback analysis results, including sentiment analysis results. Based on the analysis results, the next data collection and filtering processes are adjusted, thereby improving the overall system performance and reliability.

[0347] Reliability evaluation method

[0348] To assess reliability, the server analyzes user reviews of the published data and its characteristics. These review results are used to determine the quality and reliability of the data.

[0349] Dataset improvement methods using evaluation results

[0350] Based on the evaluation results, the server will review its data filtering process and sample data selection methods to improve the dataset. This will enhance the quality of the dataset.

[0351] Specific example:

[0352] When analyzing the sentiment of feedback using the prompt sentence, "This product is very good. The delivery was fast, and the quality was as expected," the sentiment analysis results are classified as "Positive: 80%, Negative: 20%." Based on these results, the dataset and system are improved.

[0353] Hardware and software to be used:

[0354] Server: Performs data collection, filtering, characteristic generation, sample data generation and publication, sentiment analysis, and feedback analysis.

[0355] Terminal: Users view characteristic data and sample data, and provide evaluations and feedback.

[0356] BeautifulSoup, requests (data collection)

[0357] Natural language processing algorithms (filtering)

[0358] Text analysis algorithms, machine learning models (characteristic generation)

[0359] Hugging Face Transformers Library (Emotion Analysis)

[0360] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0361] Step 1:

[0362] The server collects data from news sites and databases on the internet. Input is a specified URL or API, and the requests library is used to retrieve the data. Specifically, it periodically runs a script to retrieve the latest articles and information, parses the HTML content using the BeautifulSoup library, and extracts it as text data. The output is the collected raw data.

[0363] Step 2:

[0364] The server filters the collected data, removing personal information and copyrighted material. The input is the raw data collected in step 1, and natural language processing algorithms are used to detect personal information (e.g., name, address, phone number) and copyrighted material. Specifically, regular expressions and machine learning models are used to extract the relevant information and then remove or anonymize it. The output is the clean data after filtering.

[0365] Step 3:

[0366] The server generates characteristics from the filtered data. The input is the data filtered in step 2, and based on this, it extracts characteristics such as article genre, author name, and keywords. Specifically, it uses text analysis algorithms and machine learning models to analyze the data's features and display them as statistical information. The output is characteristic data.

[0367] Step 4:

[0368] The server selects and generates sample data based on characteristics. The input is the characteristic data generated in step 3, and representative sample data is selected based on this characteristic data. Specifically, it uses random sampling or selection methods based on specific criteria, and then formats the selected data. The output is a sample dataset.

[0369] Step 5:

[0370] The server publishes the generated characteristic data and sample data. The input is the characteristic data and sample dataset generated in step 4, which are then published via a web page or API. Specifically, the server formats the data and publishes it on the web page or API in a format that is easily accessible to users. The output is the published characteristic data and sample data.

[0371] Step 6:

[0372] Users view publicly available characteristic and sample data and provide evaluations and feedback. Input consists of data published on web pages or via APIs, and users evaluate it using scores or comments. Specifically, they input feedback using evaluation feedback forms or survey systems and send it online to the server. Output consists of user evaluations and feedback.

[0373] Step 7:

[0374] The server analyzes emotions based on user ratings and feedback. The input is the feedback data collected in step 6, and the emotion engine is used to classify positive and negative emotions. Specifically, it uses the pipeline function of the Hugging Face Transformers library to perform text analysis and classify emotions. The output is the emotion analysis result.

[0375] Step 8:

[0376] The server improves the dataset and system based on the sentiment analysis results. The input is the sentiment analysis results generated in step 7, and the server adjusts the next data collection and filtering process based on the analysis results. Specifically, it improves the overall system performance by analyzing the results, extracting feedback patterns, and introducing new algorithms. The output is the improved dataset and system settings.

[0377] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0378] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0379] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0380] [Second Embodiment]

[0381] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0382] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0383] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0384] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0385] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0386] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0387] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0388] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0389] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0390] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0391] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0392] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0393] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption caused by generative AI, while taking into consideration copyright and personal data protection issues.

[0394] Overview of program processing

[0395] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and uses it to improve the dataset and system.

[0396] Program description in natural language

[0397] Data collection and filtering

[0398] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly. The collected data is temporarily stored and then filtered. Filtering uses natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images). This process helps to avoid legal issues.

[0399] Data property generation

[0400] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0401] Generating sample data

[0402] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0403] Disclosure of samples and characteristics

[0404] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0405] Collecting user ratings and feedback

[0406] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments. The server collects these evaluations and feedback and performs analysis. The analysis results are used to improve the dataset and the system based on the collected evaluation data. User feedback is reflected in subsequent data collection and filtering processes, continuously improving the reliability and quality of the model.

[0407] The following describes the processing flow.

[0408] Step 1: Data Collection

[0409] The server collects the latest articles and information from news sites and various databases on the internet. A script for data collection is executed periodically, retrieving data from specified URLs and APIs. The collected data is stored in temporary storage.

[0410] Step 2: Filtering

[0411] The server applies natural language processing algorithms to the temporarily stored data to detect personal information (e.g., name, address, phone number) and copyrighted information. Detected personal information and copyrighted information are deleted or anonymized. The filtered data is stored for use in the next step.

[0412] Step 3: Generating Data Characteristics

[0413] The server generates characteristics from the filtered data. Specifically, it extracts features such as article genre (politics, economics, sports, etc.), age, author, and keywords. The generated characteristic data is compiled as statistical information and stored in a database.

[0414] Step 4: Selecting Sample Data

[0415] The server selects representative sample data based on the generated characteristic data. Random sampling is performed according to specific criteria, and appropriate samples are selected from each genre and characteristic. The selected sample data is stored.

[0416] Step 5: Generating sample data

[0417] The server formats the selected sample data and prepares it as a publicly available dataset. The sample data, along with its characteristic information, is converted into an appropriate format such as JSON or XML.

[0418] Step 6: Disclosure of Samples and Characteristics

[0419] The server publishes characteristic data and sample data through a configured web page or API endpoint. This data is made publicly available on the internet so that users can access it using their devices.

[0420] Step 7: Collecting user ratings and feedback

[0421] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores and comments and are sent to the server via online forms or APIs.

[0422] Step 8: Feedback analysis and system improvement

[0423] The server analyzes the collected feedback to identify areas for improvement in the dataset and filtering algorithms. Based on the analysis results, the next data collection and filtering processes are adjusted to improve overall system performance and reliability.

[0424] (Example 1)

[0425] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0426] Traditional large-scale language models have suffered from a lack of transparency, making it difficult for users to verify the quality and reliability of training data. Furthermore, there are risks of copyright infringement and personal data leaks from collected data, increasing the risk of fake news spread and data corruption through generative AI. Additionally, the lack of mechanisms to effectively collect user feedback and incorporate it into system improvements presents challenges in enhancing the reliability and quality of the models.

[0427] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0428] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for periodically processing the collected data, and means for storing the collected evaluations and feedback in a database. This improves data transparency and reliability, and enables continuous improvement of the system by utilizing user feedback while taking into consideration copyright and personal information protection.

[0429] "Means of data collection" refers to methods for regularly obtaining data from information sources on the internet.

[0430] "Means for removing personal information and copyrighted information" refers to means for detecting personal information such as names and addresses, as well as copyrighted information, from collected data, and for removing or anonymizing them.

[0431] "Means for generating characteristics" refers to methods for automatically extracting characteristic information such as genre, era, and keywords from filtered data.

[0432] "Means for selecting and generating sample data" refers to means for selecting representative sample data based on the generated characteristics and formatting it.

[0433] "Means for publishing characteristic data and sample data" refers to means for publishing the generated characteristic data and sample data through web pages or APIs.

[0434] "Means for collecting user ratings and feedback" refers to means for collecting user ratings and feedback on publicly available data.

[0435] "Means for improving datasets and systems" refers to means for improving datasets and systems based on collected feedback.

[0436] "Means for periodically processing collected data" refers to means of regularly performing data collection and processing to ensure that the latest information is always available.

[0437] "Means for storing collected ratings and feedback in a database" refers to means for storing ratings and feedback collected from users in a database and making them available for subsequent processing.

[0438] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption generated by AI, while also considering copyright and personal data protection. This system is implemented with the following configuration and procedure.

[0439] 1. Data Collection

[0440] The server periodically collects data from news sites and various databases. For this purpose, it uses scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, when collecting the latest political news from a specific news site, the server uses "BeautifulSoup" to analyze the news site's HTML and extract article titles and body text.

[0441] 2. Data filtering

[0442] The collected data is temporarily stored, and the server uses natural language processing libraries such as "Apache OpenNLP" to filter out personal and copyrighted information. For example, it detects and deletes or anonymizes personal information such as names and addresses. It also detects and deletes or anonymizes copyrighted text and images.

[0443] 3. Generating Data Characteristics

[0444] Once the filtering is complete, the server generates characteristics from the data. This is done using tools like "Scikit-learn" and "NLTK". For example, it automatically classifies the genre of articles (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. The characteristics are displayed as statistical information, distributed as follows: "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%".

[0445] 4. Generating sample data

[0446] Based on its characteristics, the server selects representative sample data and formats it using tools like "Pandas" or "Numpy." For example, when creating training data for a generative AI model, it includes a balanced mix of articles from diverse genres based on specific criteria.

[0447] 5. Data Disclosure

[0448] The server uses web frameworks such as "Flask" or "Django" to expose the generated characteristic data and sample data as web pages or APIs. For example, the server uses "Flask" to build a web page to display the characteristic data and sample data, and sets up an API endpoint so that the data can be accessed from programs as well.

[0449] 6. Collecting user ratings and feedback

[0450] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection.

[0451] As a concrete example, to create training data for a generative AI model and to verify the transparency and reliability of the data, one might consider inputting prompt statements like the following into the generative AI model.

[0452] Example of a prompt:

[0453] "Collect the latest political news and remove personal information such as names and addresses. Then, extract characteristics such as the genre, year, and author of the articles and display them as statistical information."

[0454] This makes it possible to quickly build transparent and reliable datasets. This process ensures that the training data for generative AI models is of high quality and reliable, effectively reducing the risk of fake news and data collapse.

[0455] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0456] Step 1: Data Collection

[0457] The server collects the latest articles and information from news sites and various databases. The collected data includes text data, image data, etc. This step utilizes scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, the server uses "BeautifulSoup" to parse the HTML of news sites and extract article titles and body text. A list of URLs is provided as input, and the collected raw data (HTML) is obtained as output.

[0458] Step 2: Data Filtering

[0459] The server temporarily stores the collected data and then performs a filtering process. This step uses natural language processing libraries such as "Apache OpenNLP". The server detects personal information and copyrighted information and removes or anonymizes them. Specifically, the server uses "Apache OpenNLP" to detect and remove personal information such as names and addresses. It also detects copyrighted text and images and removes or anonymizes them as well. Raw data collected is provided as input, and filtered, clean data is obtained as output.

[0460] Step 3: Generating Data Characteristics

[0461] The server generates characteristics from the filtered data. This step uses libraries such as "Scikit-learn" and "NLTK". For example, the server automatically classifies article genres (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. Specifically, the server uses "Scikit-learn" to classify article genres and "NLTK" to extract keywords, dates, and author information. Filtered data is provided as input, and the generated characteristic data is obtained as output.

[0462] Step 4: Generating sample data

[0463] The server selects sample data based on the generated characteristics. This step uses data manipulation libraries such as "Pandas" and "Numpy." For example, the server uses "Pandas" to apply characteristic-based filters and select representative sample data. Then, it uses "Numpy" to format the data and build a dataset for publication. Characteristic data is provided as input, and the generated sample data is obtained as output.

[0464] Step 5: Data Release

[0465] The server uses web frameworks such as Flask or Django to expose the generated characteristic data and sample data as web pages or APIs. For example, the server might use Flask to build a web page to display the characteristic data and sample data, and configure an API endpoint to allow programmatic access to the data. Sample data is provided as input, and the exposed web pages or API endpoints are obtained as output.

[0466] Step 6: Collecting user ratings and feedback

[0467] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection. User evaluations and feedback are provided as input, and the collected evaluation data and analysis results are obtained as output.

[0468] (Application Example 1)

[0469] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0470] In today's information society, fake news and data reliability are major problems. In particular, the quality and transparency of the source data are crucial when generating information using large-scale language models. Traditional methods lacked systems to enhance data transparency and enable users to evaluate reliability, making it difficult to handle data while considering personal information protection and copyright protection. Furthermore, there was a lack of means to effectively utilize user feedback to improve datasets and systems. In addition, there were no established methods for evaluating the quality of news data, anonymization, or publishing the generated data. This led to problems such as a decline in the accuracy of information and user trust.

[0471] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0472] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for evaluating the quality of news data based on the feedback, means for using a large-scale language model to generate article summaries, and means for storing and publishing the anonymized news data on the server. This makes it possible to enhance data transparency, provide a system that allows users to evaluate reliability, and improve the quality of news data while considering the protection of personal information and copyrights.

[0473] "Means of data collection" refers to the function of obtaining information from various information sources on the internet.

[0474] "Filtering" refers to the process of removing or anonymizing personal information or copyrighted material from collected data.

[0475] "Means for generating characteristics" refers to a function that extracts features such as genres and keywords from filtered data and forms characteristic information about the data.

[0476] "Means for selecting and generating sample data" refers to a function that selects and generates representative sample data based on its characteristics.

[0477] "Means for publishing generated characteristic data and sample data" refers to a function that makes characteristic data and sample data publicly available through a web page or API so that users can view them.

[0478] "Means for collecting user ratings and feedback" refers to functions for collecting ratings and opinions provided by users.

[0479] "Means for improving datasets and systems" refers to functions that improve datasets and systems based on collected feedback.

[0480] "Means for evaluating the quality of news data" refers to a function for evaluating the reliability and quality of news data based on feedback.

[0481] "Methods using large-scale language models" refer to algorithms that use natural language processing techniques to summarize and classify news articles.

[0482] "Means for storing and publishing news data generated through anonymization processing on a server" refers to a function that deletes or anonymizes personal information or identifiable information, then stores the news data on a server and makes it available for user access.

[0483] A specific embodiment of this invention will now be described. This system is implemented as a smartphone application called "News Reader" that allows users to view the latest news and evaluate its reliability and quality.

[0484] System Configuration

[0485] 1. Data acquisition methods

[0486] The server automatically collects information from various sources on the internet. This collection process utilizes web scraping techniques and the Requests library. Data sources include news sites and public databases.

[0487] 2. Filtering means

[0488] The server filters the collected data, removing or anonymizing personal information and copyrighted material. This filtering uses natural language processing algorithms (e.g., BeautifulSoup and the re library). The filtered data is temporarily stored in a database.

[0489] 3. Characteristic generation means

[0490] The server generates characteristics from the filtered data. These characteristics include article genres (e.g., politics, economics, sports), keywords, author names, and dates. A machine learning model (e.g., HuggingFace's Transformers library) is used for this characteristic generation.

[0491] 4. Sample data selection and generation methods

[0492] Representative sample data is selected and generated based on its characteristics. Random sampling techniques or conditional selection algorithms are used to select samples based on specific criteria. The selected data is formatted and prepared for publication.

[0493] 5. Means for publishing characteristic data and sample data

[0494] The server publishes characteristic and sample data via a webpage or API. Users can access this information using the smartphone app "News Reader." The data is published in JSON format and visualized for easy user understanding.

[0495] 6. Means of collecting user evaluations and feedback

[0496] Users can view published data through the "News Reader" and provide ratings and feedback. Ratings can be given in the form of stars or comments, and the collected feedback is stored in a database.

[0497] 7. Dataset and System Improvement Means

[0498] The server analyzes the collected feedback and uses it to improve the dataset and the system. This analysis employs statistical methods and machine learning algorithms.

[0499] 8. Methods for evaluating the quality of news data

[0500] The quality of news data is evaluated based on user feedback. Evaluation criteria include article length, keyword score, and user confidence. Data analysis tools are used to assess data quality.

[0501] 9. Methods using large-scale language models

[0502] The server uses a large-scale language model (e.g., HuggingFace's Transformers) to summarize and classify news articles. This model generates article summaries, allowing users to quickly grasp the content.

[0503] 10. Anonymization and Data Disclosure Methods

[0504] The data is anonymized, stored on a server, and then made public. The anonymization process uses filtering algorithms to remove personal and identifying information.

[0505] Examples of specific cases and prompt statements

[0506] For example, if an article contains the sentence, "John Smith has moved to a new address, 123 Main St," this text will be filtered to "John Smith has moved to a new address." Furthermore, summarizing this sentence would result in "John is moving to a new residence."

[0507] Examples of prompts for a generative AI model:

[0508] Please summarize the following news article:

[0509] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0510] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0511] Step 1:

[0512] Data collection

[0513] The server automatically collects data from news sites and public databases on the internet. Input is web article URLs or RSS feeds, and output is the text data of the web pages. Specifically, it uses the Requests library to retrieve the HTML content of web pages and BeautifulSoup to extract the text portion of the articles.

[0514] Step 2:

[0515] filtering

[0516] The server filters out personal and copyright-protected information from the collected text data. The input is the text data of the web page obtained in step 1, and the output is the filtered text data. Specifically, it uses Natural Language Processing (NLP) algorithms and regular expressions (re library) to detect and remove or anonymize personal information such as names and addresses.

[0517] Step 3:

[0518] Characteristic generation

[0519] The server generates characteristics from the filtered data. The input is the text data filtered in step 2, and the output is characteristic information (e.g., genre, keywords, publication year, etc.). Specifically, a machine learning model (HuggingFace's Transformers library) is used to extract keywords and classify genres.

[0520] Step 4:

[0521] Selection and generation of sample data

[0522] The server selects and generates representative sample data based on the characteristics. The input is the characteristic information and filtered text data generated in step 3, and the output is the selected and generated sample data. Specifically, random sampling techniques and condition-based selection algorithms are used to select data that matches the characteristics.

[0523] Step 5:

[0524] Disclosure of sample data and characteristic information

[0525] The server publishes sample data and characteristic information. The input is the sample data and characteristic information generated in step 4, and the output is a data file (in JSON format) for publication. Specifically, you configure the server or API to allow users to access this data.

[0526] Step 6:

[0527] Collecting user ratings and feedback

[0528] Users view the published sample data and characteristic information and provide ratings and feedback. The input is the data published in step 5, and the output is user ratings and feedback information. Specifically, star ratings and text comments are collected using the rating form and comment function within the app.

[0529] Step 7:

[0530] Dataset and system improvements

[0531] The server analyzes the collected feedback and uses it to improve the dataset and system. The input is the user ratings and feedback information collected in step 6, and the output is the improved dataset and system settings. Specifically, statistical analysis tools and machine learning algorithms are used to analyze the feedback and reflect it in the next data collection and filtering process.

[0532] Step 8:

[0533] Quality assessment of news data

[0534] The server evaluates the quality of the news data based on feedback. The input is the user ratings and feedback information collected in step 6, and the output is the evaluation result. Specifically, it calculates factors such as article length, keyword score, and user confidence to generate an overall quality score.

[0535] Step 9:

[0536] Summarization and classification using large-scale language models

[0537] The server uses a generative AI model (HuggingFace's Transformers) to generate summaries of news articles. The input is the collected and filtered news article text, and the output is the summarized text. Specifically, a specified prompt sentence is input to the generative AI model to obtain a concise summary.

[0538] Example of a prompt:

[0539] Please summarize the following news article:

[0540] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0541] Step 10:

[0542] Anonymization and data disclosure

[0543] The server stores and publishes anonymized news data. The input is filtered news article text, and the output is anonymized text and a data file for publication. Specifically, a filtering algorithm is used to remove or mask personal information and identifying information, and a data file (in JSON format) for publication is generated and stored on the server.

[0544] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0545] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0546] Overview of program processing

[0547] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and analyzes user emotions using a sentiment engine. Based on these analysis results, improvements are made to the dataset and system.

[0548] Program description in natural language

[0549] Data collection and filtering

[0550] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly, using specific scripts to retrieve data from designated URLs and APIs. The collected data is stored in temporary storage. Next, a filtering process is performed, using natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images).

[0551] Data property generation

[0552] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0553] Generating sample data

[0554] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0555] Disclosure of samples and characteristics

[0556] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0557] Collecting user ratings and feedback

[0558] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0559] Analysis using an emotion engine

[0560] The server applies an emotion engine to the collected ratings and feedback to analyze the user's emotions. This emotion analysis identifies the emotions behind the feedback and classifies them, for example, as "Positive: 80%, Negative: 20%." The emotion engine uses text analysis and machine learning algorithms to accurately analyze emotions.

[0561] Feedback analysis and system improvement

[0562] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, subsequent data collection and filtering processes are adjusted to improve overall system performance and reliability. For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm is introduced to resolve the issue.

[0563] The following describes the processing flow.

[0564] Step 1: Data Collection

[0565] The server periodically runs scripts to collect the latest articles and information from news sites and various databases. It retrieves data from specified URLs and APIs, and stores the collected data in temporary storage.

[0566] Step 2: Filtering

[0567] The server applies natural language processing algorithms to the temporarily stored data. The algorithms detect personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images), and delete or anonymize the detected information. The filtered, secure data is then sent to the next step.

[0568] Step 3: Generating Data Characteristics

[0569] The server generates characteristics from the filtered data. In this step, features such as article genre (politics, economics, sports, etc.), age group, author, and keywords are extracted. The characteristic data is compiled as statistical information, and the distribution is displayed, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%." The generated characteristic data is stored in a database.

[0570] Step 4: Selecting Sample Data

[0571] The server selects representative sample data based on the generated characteristic data. This selection includes random sampling and sampling based on specific criteria. The sample data is formatted and stored along with the characteristic information.

[0572] Step 5: Generating sample data

[0573] The server converts sample data selected based on characteristic information into an appropriate format (e.g., JSON, XML). This prepares the sample dataset for public release.

[0574] Step 6: Disclosure of Samples and Characteristics

[0575] The server configures a web page or API endpoint to expose characteristic data and sample data. This allows users to access this information using their devices and verify the data's transparency.

[0576] Step 7: Collecting user ratings and feedback

[0577] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0578] Step 8: Analysis using the emotion engine

[0579] The server applies a sentiment engine to the collected ratings and feedback. The sentiment engine uses text analysis and machine learning algorithms to analyze the user's emotions (positive, negative, neutral, etc.) and generate sentiment data.

[0580] Step 9: Feedback Analysis and System Improvement

[0581] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, it adjusts the next data collection and filtering process to improve the overall system performance and reliability. For example, if a large number of negative sentiments are reported, it identifies the underlying data or algorithmic problems and implements improvements.

[0582] (Example 2)

[0583] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0584] Modern large-scale language models face numerous concerns regarding the transparency and quality of training data. These include issues such as the leakage of personal information, copyright infringement, and even the generation of fake news. Furthermore, analyzing user feedback and incorporating it into system improvements presents a significant challenge. New approaches are needed to effectively address these problems.

[0585] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0586] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, and means for performing sentiment analysis on the collected feedback.

[0587] This will allow us to protect personal information and copyrights while improving data transparency and quality, and to efficiently analyze user feedback to improve the system.

[0588] "Data collection" is the process of automatically obtaining articles and information from news sites, databases, and other sources on the internet.

[0589] "Data filtering" is the process of detecting and deleting or anonymizing personal information and copyrighted material from collected data.

[0590] "Natural language processing" is a general term for the technologies and methods that enable computers to understand and analyze human language.

[0591] "Characteristic generation" is the process of extracting features such as article genre, year, author, and keywords from filtered data and compiling them into statistical information.

[0592] "Sample data generation" is the process of selecting representative data based on the generated characteristics and formatting it for public release.

[0593] "Data publication" refers to the process of providing generated characteristic data and sample data to users through web pages or APIs.

[0594] "Collecting user ratings and feedback" is the process of collecting information from users who provide ratings and comments on publicly available characteristic data and sample data.

[0595] "Sentiment analysis" is the process of using an emotion engine to analyze the emotions behind collected user ratings and feedback.

[0596] "System improvement" is the process of improving datasets and filtering algorithms based on collected feedback and sentiment analysis results.

[0597] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0598] Overview of program processing

[0599] The server collects data from news sites and databases on the internet, and filters the collected data to remove or anonymize personal information and copyrighted material. This filtering uses web scraping libraries such as BeautifulSoup and Scrapy in Python, as well as the entity recognition function of spaCy. After filtering, characteristics are generated from the data using techniques such as topic modeling (LDA) and keyword extraction (TF-IDF), and sample data is selected and generated based on these characteristics.

[0600] The generated sample and characteristic data are published via web pages and APIs using web frameworks such as Django and Flask. Users can access this information using their devices and verify the data's transparency. Evaluations and feedback are provided via online forms and APIs.

[0601] The server collects user feedback and analyzes user emotions using an emotion engine (such as a BERT model or fastText). Based on this analysis, the dataset and filtering algorithm are improved. For example, the filtering algorithm is adjusted based on feedback that contains a lot of negative emotions, and this is reflected in the next data collection.

[0602] Specific example

[0603] As a concrete example, input the following prompt into the generation AI model.

[0604] Example of a prompt:

[0605] "Collect the latest news articles on politics and economics, and remove any personal information or copyrighted material. Then, extract the article genres and keywords as characteristics, and generate three representative sample data based on these characteristics. Finally, publish the generated data on a webpage and collect and analyze sentiment feedback from users."

[0606] Based on this prompt, the system executes specific steps such as data collection, filtering, characteristic generation, sample data generation, data publication, evaluation collection, and sentiment analysis. This allows for system improvement by effectively utilizing user feedback while maintaining data transparency and quality.

[0607] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0608] The program's processing is explained step by step.

[0609] Step 1: Data Collection

[0610] explanation

[0611] The server collects the latest articles and information from news sites and databases on the internet. This task is performed periodically using specific scripts (e.g., BeautifulSoup or Scrapy in Python).

[0612] Specific actions

[0613] The server accesses a specified list of URLs or API endpoints, parses the HTML structure of the page, and retrieves the necessary data. For example, it can extract the titles and content of the latest articles from a news website's URL.

[0614] Input and output

[0615] Input: A specified list of URLs or API endpoints

[0616] Output: Collected raw HTML or JSON data

[0617] Step 2: Data Filtering

[0618] explanation

[0619] The server detects and removes or anonymizes personal information and copyrighted material from the collected data. Natural language processing (NLP) algorithms are used for this process.

[0620] Specific actions

[0621] For example, spaCy's entity recognition feature is used to identify personal names and addresses. If a name or address is detected in the data, it is replaced with "[ANONYMIZED]".

[0622] Input and output

[0623] Input: Collected raw data (HTML or JSON)

[0624] Output: Text data with personal information and copyright protection information removed or anonymized.

[0625] Step 3: Generating Data Characteristics

[0626] explanation

[0627] The server generates characteristics such as article genre, year, author, and keywords from the filtered data.

[0628] Specific actions

[0629] For example, to classify the genre of articles, techniques such as topic modeling (LDA) and keyword extraction (TF-IDF) are used. Characteristic data is generated, such as the fact that political articles account for 40% of the total.

[0630] Input and output

[0631] Input: Filtered text data

[0632] Output: Characteristic data such as genre, era, author, and keywords.

[0633] Step 4: Generating sample data

[0634] explanation

[0635] The server selects representative sample data based on its characteristics and formats it.

[0636] Specific actions

[0637] For example, sample data is selected using methods such as random sampling based on the proportion of specific characteristics. This sample data is then formatted for public release.

[0638] Input and output

[0639] Input: Characteristic data

[0640] Output: Sample dataset

[0641] Step 5: Disclosure of Samples and Characteristics

[0642] explanation

[0643] The server publishes the generated characteristic data and sample data through a web page or API.

[0644] Specific actions

[0645] For example, you can create a web page using a web framework such as Django or Flask and display characteristic data and sample data.

[0646] Input and output

[0647] Input: Sample dataset and characteristic data

[0648] Output: A web page or API endpoint exposed to users.

[0649] Step 6: Collecting user ratings and feedback

[0650] explanation

[0651] Users use their devices to view publicly available characteristic data and sample data, and provide evaluations and feedback on them. This information is sent to the server.

[0652] Specific actions

[0653] For example, you could place a rating form on a webpage, allowing users to enter scores and comments.

[0654] Input and output

[0655] Input: User-submitted ratings and feedback

[0656] Output: Evaluation and feedback data stored on the server

[0657] Step 7: Analysis using the emotion engine

[0658] explanation

[0659] The server uses an emotion engine to analyze collected ratings and feedback and classify the user's emotions.

[0660] Specific actions

[0661] For example, sentiment engines such as the BERT model or fastText are used to classify each feedback comment as either "positive" or "negative."

[0662] Input and output

[0663] Input: Rating and feedback data collected from users

[0664] Output: Sentiment analysis results (Example: "Positive: 80%, Negative: 20%")

[0665] Step 8: Feedback analysis and system improvement

[0666] explanation

[0667] The server improves the dataset and filtering algorithms based on feedback results, including sentiment analysis.

[0668] Specific actions

[0669] For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm will be implemented to address this issue and applied to the next data collection.

[0670] Input and output

[0671] Input: Sentiment analysis results and feedback data

[0672] Output: Improved filtering algorithm and dataset

[0673] By repeating these processing steps, the overall system performance and reliability are improved, and data transparency and user experience are optimized.

[0674] (Application Example 2)

[0675] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0676] In modern large-scale language model training data, data transparency and reliability are not sufficiently ensured, making it difficult for users to evaluate data quality. Furthermore, in addition to copyright and personal data protection issues, there is a growing risk of fake news generation and data corruption. Moreover, the lack of collection and analysis of feedback that considers user sentiment leads to insufficient improvement of datasets and systems. A new system is needed to address these challenges.

[0677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0678] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for analyzing user sentiment based on the evaluations and feedback, means for improving the dataset and system based on the sentiment analysis results, means for evaluating reliability, and means for improving the dataset using the evaluation results. This increases the transparency of training data for large-scale language models, allows users to evaluate the quality and reliability of the data, reduces the risk of fake news and data collapse, and improves the overall system.

[0679] "Means of data collection" refers to the function of obtaining data from various information sources on the internet.

[0680] "Means for filtering data and removing personal and copyrighted information" refers to functions that detect and remove or anonymize personal and copyrighted information contained in collected data.

[0681] "Methods for generating characteristics from filtered data" refers to functions that extract characteristics such as article genre, author name, and keywords from filtered data, and analyze the characteristics of the data.

[0682] "Means for selecting and generating sample data based on characteristics" refers to a function for selecting representative data based on the generated characteristics, formatting that data, and providing it as a sample dataset.

[0683] "Means for publishing generated characteristic data and sample data" refers to functions for making characteristic data and sample data publicly available through web pages or APIs.

[0684] "Means for collecting user ratings and feedback" refers to functions for collecting user ratings and feedback on publicly available data via online forms or APIs.

[0685] "Means for analyzing user emotions based on evaluations and feedback" refers to a function that performs sentiment analysis on collected feedback and evaluations to classify users' positive and negative emotions.

[0686] "Means for improving datasets and systems based on sentiment analysis results" refers to functions that improve datasets and system algorithms based on information obtained from sentiment analysis results, thereby improving the overall performance of the system.

[0687] A "means for evaluating reliability" refers to a function that analyzes user evaluations of publicly available data and its characteristics, and evaluates the reliability of that data.

[0688] "Means for improving datasets using evaluation results" refers to functions that improve the quality of datasets by reviewing the data filtering process and sample data selection method based on the analysis results of user evaluations.

[0689] This invention specifically illustrates a form of quality evaluation application for e-commerce websites. This system has functions for data collection, filtering, characteristic generation, sample data generation, publication, evaluation, and feedback collection and analysis.

[0690] Data acquisition methods

[0691] The server collects data from news sites and various databases on the internet. This collection process is performed periodically, and data is retrieved from specified URLs and APIs using specific scripts. Libraries such as BeautifulSoup and requests can be used for this process.

[0692] Filtering means

[0693] To remove personal information and copyrighted material from collected data, the server uses natural language processing algorithms. Collected data is stored in temporary storage, where natural language processing algorithms are used to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted material. This ensures data integrity and privacy.

[0694] Characteristic generation means

[0695] From the filtered data, the server extracts characteristics such as article genre, author name, and keywords. This characteristic generation process uses text analysis algorithms and machine learning models. For example, characteristics distributed as "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%" are displayed as statistical information.

[0696] Method for generating sample data

[0697] Based on characteristic data, the server selects representative sample data. This selection includes random sampling and selection based on specific criteria. The selected sample data is verified and formatted along with characteristic information. This dataset is then prepared for public release.

[0698] means of disclosure

[0699] The server configures a webpage or API to publish characteristic and sample data. Users can access this information using their devices and verify the data's transparency. For example, users can view product descriptions and reviews and evaluate their quality.

[0700] User evaluation and feedback collection methods

[0701] Users evaluate publicly available characteristic data and sample data and provide feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs. Evaluation feedback forms or survey systems can be used for this data collection process.

[0702] Emotion analysis means

[0703] The server applies a sentiment engine to the collected ratings and feedback to analyze the user's emotions. Using text analysis and machine learning algorithms, it identifies the emotions behind the feedback and classifies them, such as "Positive: 80%, Negative: 20%". This sentiment analysis utilizes the pipeline function from the Hugging Face Transformers library.

[0704] Dataset and system improvement means

[0705] The server identifies areas for improvement in the dataset and system based on feedback analysis results, including sentiment analysis results. Based on the analysis results, the next data collection and filtering processes are adjusted, thereby improving the overall system performance and reliability.

[0706] Reliability evaluation method

[0707] To assess reliability, the server analyzes user reviews of the published data and its characteristics. These review results are used to determine the quality and reliability of the data.

[0708] Dataset improvement methods using evaluation results

[0709] Based on the evaluation results, the server will review its data filtering process and sample data selection methods to improve the dataset. This will enhance the quality of the dataset.

[0710] Specific example:

[0711] When analyzing the sentiment of feedback using the prompt sentence, "This product is very good. The delivery was fast, and the quality was as expected," the sentiment analysis results are classified as "Positive: 80%, Negative: 20%." Based on these results, the dataset and system are improved.

[0712] Hardware and software to be used:

[0713] Server: Performs data collection, filtering, characteristic generation, sample data generation and publication, sentiment analysis, and feedback analysis.

[0714] Terminal: Users view characteristic data and sample data, and provide evaluations and feedback.

[0715] BeautifulSoup, requests (data collection)

[0716] Natural language processing algorithms (filtering)

[0717] Text analysis algorithms, machine learning models (characteristic generation)

[0718] Hugging Face Transformers Library (Emotion Analysis)

[0719] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0720] Step 1:

[0721] The server collects data from news sites and databases on the internet. Input is a specified URL or API, and the requests library is used to retrieve the data. Specifically, it periodically runs a script to retrieve the latest articles and information, parses the HTML content using the BeautifulSoup library, and extracts it as text data. The output is the collected raw data.

[0722] Step 2:

[0723] The server filters the collected data, removing personal information and copyrighted material. The input is the raw data collected in step 1, and natural language processing algorithms are used to detect personal information (e.g., name, address, phone number) and copyrighted material. Specifically, regular expressions and machine learning models are used to extract the relevant information and then remove or anonymize it. The output is the clean data after filtering.

[0724] Step 3:

[0725] The server generates characteristics from the filtered data. The input is the data filtered in step 2, and based on this, it extracts characteristics such as article genre, author name, and keywords. Specifically, it uses text analysis algorithms and machine learning models to analyze the data's features and display them as statistical information. The output is characteristic data.

[0726] Step 4:

[0727] The server selects and generates sample data based on characteristics. The input is the characteristic data generated in step 3, and representative sample data is selected based on this characteristic data. Specifically, it uses random sampling or selection methods based on specific criteria, and then formats the selected data. The output is a sample dataset.

[0728] Step 5:

[0729] The server publishes the generated characteristic data and sample data. The input is the characteristic data and sample dataset generated in step 4, which are then published via a web page or API. Specifically, the server formats the data and publishes it on the web page or API in a format that is easily accessible to users. The output is the published characteristic data and sample data.

[0730] Step 6:

[0731] Users view publicly available characteristic and sample data and provide evaluations and feedback. Input consists of data published on web pages or via APIs, and users evaluate it using scores or comments. Specifically, they input feedback using evaluation feedback forms or survey systems and send it online to the server. Output consists of user evaluations and feedback.

[0732] Step 7:

[0733] The server analyzes emotions based on user ratings and feedback. The input is the feedback data collected in step 6, and the emotion engine is used to classify positive and negative emotions. Specifically, it uses the pipeline function of the Hugging Face Transformers library to perform text analysis and classify emotions. The output is the emotion analysis result.

[0734] Step 8:

[0735] The server improves the dataset and system based on the sentiment analysis results. The input is the sentiment analysis results generated in step 7, and the server adjusts the next data collection and filtering process based on the analysis results. Specifically, it improves the overall system performance by analyzing the results, extracting feedback patterns, and introducing new algorithms. The output is the improved dataset and system settings.

[0736] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0737] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0738] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0739] [Third Embodiment]

[0740] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0741] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0742] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0743] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0744] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0745] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0746] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0747] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0748] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0749] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0750] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0751] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0752] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption caused by generative AI, while taking into consideration copyright and personal data protection issues.

[0753] Overview of program processing

[0754] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and uses it to improve the dataset and system.

[0755] Program description in natural language

[0756] Data collection and filtering

[0757] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly. The collected data is temporarily stored and then filtered. Filtering uses natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images). This process helps to avoid legal issues.

[0758] Data property generation

[0759] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0760] Generating sample data

[0761] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0762] Disclosure of samples and characteristics

[0763] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0764] Collecting user ratings and feedback

[0765] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments. The server collects these evaluations and feedback and performs analysis. The analysis results are used to improve the dataset and the system based on the collected evaluation data. User feedback is reflected in subsequent data collection and filtering processes, continuously improving the reliability and quality of the model.

[0766] The following describes the processing flow.

[0767] Step 1: Data Collection

[0768] The server collects the latest articles and information from news sites and various databases on the internet. A script for data collection is executed periodically, retrieving data from specified URLs and APIs. The collected data is stored in temporary storage.

[0769] Step 2: Filtering

[0770] The server applies natural language processing algorithms to the temporarily stored data to detect personal information (e.g., name, address, phone number) and copyrighted information. Detected personal information and copyrighted information are deleted or anonymized. The filtered data is stored for use in the next step.

[0771] Step 3: Generating Data Characteristics

[0772] The server generates characteristics from the filtered data. Specifically, it extracts features such as article genre (politics, economics, sports, etc.), age, author, and keywords. The generated characteristic data is compiled as statistical information and stored in a database.

[0773] Step 4: Selecting Sample Data

[0774] The server selects representative sample data based on the generated characteristic data. Random sampling is performed according to specific criteria, and appropriate samples are selected from each genre and characteristic. The selected sample data is stored.

[0775] Step 5: Generating sample data

[0776] The server formats the selected sample data and prepares it as a publicly available dataset. The sample data, along with its characteristic information, is converted into an appropriate format such as JSON or XML.

[0777] Step 6: Disclosure of Samples and Characteristics

[0778] The server publishes characteristic data and sample data through a configured web page or API endpoint. This data is made publicly available on the internet so that users can access it using their devices.

[0779] Step 7: Collecting user ratings and feedback

[0780] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores and comments and are sent to the server via online forms or APIs.

[0781] Step 8: Feedback analysis and system improvement

[0782] The server analyzes the collected feedback to identify areas for improvement in the dataset and filtering algorithms. Based on the analysis results, the next data collection and filtering processes are adjusted to improve overall system performance and reliability.

[0783] (Example 1)

[0784] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0785] Traditional large-scale language models have suffered from a lack of transparency, making it difficult for users to verify the quality and reliability of training data. Furthermore, there are risks of copyright infringement and personal data leaks from collected data, increasing the risk of fake news spread and data corruption through generative AI. Additionally, the lack of mechanisms to effectively collect user feedback and incorporate it into system improvements presents challenges in enhancing the reliability and quality of the models.

[0786] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0787] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for periodically processing the collected data, and means for storing the collected evaluations and feedback in a database. This improves data transparency and reliability, and enables continuous improvement of the system by utilizing user feedback while taking into consideration copyright and personal information protection.

[0788] "Means of data collection" refers to methods for regularly obtaining data from information sources on the internet.

[0789] "Means for removing personal information and copyrighted information" refers to means for detecting personal information such as names and addresses, as well as copyrighted information, from collected data, and for removing or anonymizing them.

[0790] "Means for generating characteristics" refers to methods for automatically extracting characteristic information such as genre, era, and keywords from filtered data.

[0791] "Means for selecting and generating sample data" refers to means for selecting representative sample data based on the generated characteristics and formatting it.

[0792] "Means for publishing characteristic data and sample data" refers to means for publishing the generated characteristic data and sample data through web pages or APIs.

[0793] "Means for collecting user ratings and feedback" refers to means for collecting user ratings and feedback on publicly available data.

[0794] "Means for improving datasets and systems" refers to means for improving datasets and systems based on collected feedback.

[0795] "Means for periodically processing collected data" refers to means of regularly performing data collection and processing to ensure that the latest information is always available.

[0796] "Means for storing collected ratings and feedback in a database" refers to means for storing ratings and feedback collected from users in a database and making them available for subsequent processing.

[0797] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption generated by AI, while also considering copyright and personal data protection. This system is implemented with the following configuration and procedure.

[0798] 1. Data Collection

[0799] The server periodically collects data from news sites and various databases. For this purpose, it uses scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, when collecting the latest political news from a specific news site, the server uses "BeautifulSoup" to analyze the news site's HTML and extract article titles and body text.

[0800] 2. Data filtering

[0801] The collected data is temporarily stored, and the server uses natural language processing libraries such as "Apache OpenNLP" to filter out personal and copyrighted information. For example, it detects and deletes or anonymizes personal information such as names and addresses. It also detects and deletes or anonymizes copyrighted text and images.

[0802] 3. Generating Data Characteristics

[0803] Once the filtering is complete, the server generates characteristics from the data. This is done using tools like "Scikit-learn" and "NLTK". For example, it automatically classifies the genre of articles (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. The characteristics are displayed as statistical information, distributed as follows: "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%".

[0804] 4. Generating sample data

[0805] Based on its characteristics, the server selects representative sample data and formats it using tools like "Pandas" or "Numpy." For example, when creating training data for a generative AI model, it includes a balanced mix of articles from diverse genres based on specific criteria.

[0806] 5. Data Disclosure

[0807] The server uses web frameworks such as "Flask" or "Django" to expose the generated characteristic data and sample data as web pages or APIs. For example, the server uses "Flask" to build a web page to display the characteristic data and sample data, and sets up an API endpoint so that the data can be accessed from programs as well.

[0808] 6. Collecting user ratings and feedback

[0809] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection.

[0810] As a concrete example, to create training data for a generative AI model and to verify the transparency and reliability of the data, one might consider inputting prompt statements like the following into the generative AI model.

[0811] Example of a prompt:

[0812] "Collect the latest political news and remove personal information such as names and addresses. Then, extract characteristics such as the genre, year, and author of the articles and display them as statistical information."

[0813] This makes it possible to quickly build transparent and reliable datasets. This process ensures that the training data for generative AI models is of high quality and reliable, effectively reducing the risk of fake news and data collapse.

[0814] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0815] Step 1: Data Collection

[0816] The server collects the latest articles and information from news sites and various databases. The collected data includes text data, image data, etc. This step utilizes scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, the server uses "BeautifulSoup" to parse the HTML of news sites and extract article titles and body text. A list of URLs is provided as input, and the collected raw data (HTML) is obtained as output.

[0817] Step 2: Data Filtering

[0818] The server temporarily stores the collected data and then performs a filtering process. This step uses natural language processing libraries such as "Apache OpenNLP". The server detects personal information and copyrighted information and removes or anonymizes them. Specifically, the server uses "Apache OpenNLP" to detect and remove personal information such as names and addresses. It also detects copyrighted text and images and removes or anonymizes them as well. Raw data collected is provided as input, and filtered, clean data is obtained as output.

[0819] Step 3: Generating Data Characteristics

[0820] The server generates characteristics from the filtered data. This step uses libraries such as "Scikit-learn" and "NLTK". For example, the server automatically classifies article genres (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. Specifically, the server uses "Scikit-learn" to classify article genres and "NLTK" to extract keywords, dates, and author information. Filtered data is provided as input, and the generated characteristic data is obtained as output.

[0821] Step 4: Generating sample data

[0822] The server selects sample data based on the generated characteristics. This step uses data manipulation libraries such as "Pandas" and "Numpy." For example, the server uses "Pandas" to apply characteristic-based filters and select representative sample data. Then, it uses "Numpy" to format the data and build a dataset for publication. Characteristic data is provided as input, and the generated sample data is obtained as output.

[0823] Step 5: Data Release

[0824] The server uses web frameworks such as Flask or Django to expose the generated characteristic data and sample data as web pages or APIs. For example, the server might use Flask to build a web page to display the characteristic data and sample data, and configure an API endpoint to allow programmatic access to the data. Sample data is provided as input, and the exposed web pages or API endpoints are obtained as output.

[0825] Step 6: Collecting user ratings and feedback

[0826] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection. User evaluations and feedback are provided as input, and the collected evaluation data and analysis results are obtained as output.

[0827] (Application Example 1)

[0828] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0829] In today's information society, fake news and data reliability are major problems. In particular, the quality and transparency of the source data are crucial when generating information using large-scale language models. Traditional methods lacked systems to enhance data transparency and enable users to evaluate reliability, making it difficult to handle data while considering personal information protection and copyright protection. Furthermore, there was a lack of means to effectively utilize user feedback to improve datasets and systems. In addition, there were no established methods for evaluating the quality of news data, anonymization, or publishing the generated data. This led to problems such as a decline in the accuracy of information and user trust.

[0830] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0831] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for evaluating the quality of news data based on the feedback, means for using a large-scale language model to generate article summaries, and means for storing and publishing the anonymized news data on the server. This makes it possible to enhance data transparency, provide a system that allows users to evaluate reliability, and improve the quality of news data while considering the protection of personal information and copyrights.

[0832] "Means of data collection" refers to the function of obtaining information from various information sources on the internet.

[0833] "Filtering" refers to the process of removing or anonymizing personal information or copyrighted material from collected data.

[0834] "Means for generating characteristics" refers to a function that extracts features such as genres and keywords from filtered data and forms characteristic information about the data.

[0835] "Means for selecting and generating sample data" refers to a function that selects and generates representative sample data based on its characteristics.

[0836] "Means for publishing generated characteristic data and sample data" refers to a function that makes characteristic data and sample data publicly available through a web page or API so that users can view them.

[0837] "Means for collecting user ratings and feedback" refers to functions for collecting ratings and opinions provided by users.

[0838] "Means for improving datasets and systems" refers to functions that improve datasets and systems based on collected feedback.

[0839] "Means for evaluating the quality of news data" refers to a function for evaluating the reliability and quality of news data based on feedback.

[0840] "Methods using large-scale language models" refer to algorithms that use natural language processing techniques to summarize and classify news articles.

[0841] "Means for storing and publishing news data generated through anonymization processing on a server" refers to a function that deletes or anonymizes personal information or identifiable information, then stores the news data on a server and makes it available for user access.

[0842] A specific embodiment of this invention will now be described. This system is implemented as a smartphone application called "News Reader" that allows users to view the latest news and evaluate its reliability and quality.

[0843] System Configuration

[0844] 1. Data acquisition methods

[0845] The server automatically collects information from various sources on the internet. This collection process utilizes web scraping techniques and the Requests library. Data sources include news sites and public databases.

[0846] 2. Filtering means

[0847] The server filters the collected data, removing or anonymizing personal information and copyrighted material. This filtering uses natural language processing algorithms (e.g., BeautifulSoup and the re library). The filtered data is temporarily stored in a database.

[0848] 3. Characteristic generation means

[0849] The server generates characteristics from the filtered data. These characteristics include article genres (e.g., politics, economics, sports), keywords, author names, and dates. A machine learning model (e.g., HuggingFace's Transformers library) is used for this characteristic generation.

[0850] 4. Sample data selection and generation methods

[0851] Representative sample data is selected and generated based on its characteristics. Random sampling techniques or conditional selection algorithms are used to select samples based on specific criteria. The selected data is formatted and prepared for publication.

[0852] 5. Means for publishing characteristic data and sample data

[0853] The server publishes characteristic and sample data via a webpage or API. Users can access this information using the smartphone app "News Reader." The data is published in JSON format and visualized for easy user understanding.

[0854] 6. Means of collecting user evaluations and feedback

[0855] Users can view published data through the "News Reader" and provide ratings and feedback. Ratings can be given in the form of stars or comments, and the collected feedback is stored in a database.

[0856] 7. Dataset and System Improvement Means

[0857] The server analyzes the collected feedback and uses it to improve the dataset and the system. This analysis employs statistical methods and machine learning algorithms.

[0858] 8. Methods for evaluating the quality of news data

[0859] The quality of news data is evaluated based on user feedback. Evaluation criteria include article length, keyword score, and user confidence. Data analysis tools are used to assess data quality.

[0860] 9. Methods using large-scale language models

[0861] The server uses a large-scale language model (e.g., HuggingFace's Transformers) to summarize and classify news articles. This model generates article summaries, allowing users to quickly grasp the content.

[0862] 10. Anonymization and Data Disclosure Methods

[0863] The data is anonymized, stored on a server, and then made public. The anonymization process uses filtering algorithms to remove personal and identifying information.

[0864] Examples of specific cases and prompt statements

[0865] For example, if an article contains the sentence, "John Smith has moved to a new address, 123 Main St," this text will be filtered to "John Smith has moved to a new address." Furthermore, summarizing this sentence would result in "John is moving to a new residence."

[0866] Examples of prompts for a generative AI model:

[0867] Please summarize the following news article:

[0868] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0869] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0870] Step 1:

[0871] Data collection

[0872] The server automatically collects data from news sites and public databases on the internet. Input is web article URLs or RSS feeds, and output is the text data of the web pages. Specifically, it uses the Requests library to retrieve the HTML content of web pages and BeautifulSoup to extract the text portion of the articles.

[0873] Step 2:

[0874] filtering

[0875] The server filters out personal and copyright-protected information from the collected text data. The input is the text data of the web page obtained in step 1, and the output is the filtered text data. Specifically, it uses Natural Language Processing (NLP) algorithms and regular expressions (re library) to detect and remove or anonymize personal information such as names and addresses.

[0876] Step 3:

[0877] Characteristic generation

[0878] The server generates characteristics from the filtered data. The input is the text data filtered in step 2, and the output is characteristic information (e.g., genre, keywords, publication year, etc.). Specifically, a machine learning model (HuggingFace's Transformers library) is used to extract keywords and classify genres.

[0879] Step 4:

[0880] Selection and generation of sample data

[0881] The server selects and generates representative sample data based on the characteristics. The input is the characteristic information and filtered text data generated in step 3, and the output is the selected and generated sample data. Specifically, random sampling techniques and condition-based selection algorithms are used to select data that matches the characteristics.

[0882] Step 5:

[0883] Disclosure of sample data and characteristic information

[0884] The server publishes sample data and characteristic information. The input is the sample data and characteristic information generated in step 4, and the output is a data file (in JSON format) for publication. Specifically, you configure the server or API to allow users to access this data.

[0885] Step 6:

[0886] Collecting user ratings and feedback

[0887] Users view the published sample data and characteristic information and provide ratings and feedback. The input is the data published in step 5, and the output is user ratings and feedback information. Specifically, star ratings and text comments are collected using the rating form and comment function within the app.

[0888] Step 7:

[0889] Dataset and system improvements

[0890] The server analyzes the collected feedback and uses it to improve the dataset and system. The input is the user ratings and feedback information collected in step 6, and the output is the improved dataset and system settings. Specifically, statistical analysis tools and machine learning algorithms are used to analyze the feedback and reflect it in the next data collection and filtering process.

[0891] Step 8:

[0892] Quality assessment of news data

[0893] The server evaluates the quality of the news data based on feedback. The input is the user ratings and feedback information collected in step 6, and the output is the evaluation result. Specifically, it calculates factors such as article length, keyword score, and user confidence to generate an overall quality score.

[0894] Step 9:

[0895] Summarization and classification using large-scale language models

[0896] The server uses a generative AI model (HuggingFace's Transformers) to generate summaries of news articles. The input is the collected and filtered news article text, and the output is the summarized text. Specifically, a specified prompt sentence is input to the generative AI model to obtain a concise summary.

[0897] Example of a prompt:

[0898] Please summarize the following news article:

[0899] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[0900] Step 10:

[0901] Anonymization and data disclosure

[0902] The server stores and publishes anonymized news data. The input is filtered news article text, and the output is anonymized text and a data file for publication. Specifically, a filtering algorithm is used to remove or mask personal information and identifying information, and a data file (in JSON format) for publication is generated and stored on the server.

[0903] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0904] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0905] Overview of program processing

[0906] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and analyzes user emotions using a sentiment engine. Based on these analysis results, improvements are made to the dataset and system.

[0907] Program description in natural language

[0908] Data collection and filtering

[0909] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly, using specific scripts to retrieve data from designated URLs and APIs. The collected data is stored in temporary storage. Next, a filtering process is performed, using natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images).

[0910] Data property generation

[0911] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[0912] Generating sample data

[0913] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[0914] Disclosure of samples and characteristics

[0915] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[0916] Collecting user ratings and feedback

[0917] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0918] Analysis using an emotion engine

[0919] The server applies an emotion engine to the collected ratings and feedback to analyze the user's emotions. This emotion analysis identifies the emotions behind the feedback and classifies them, for example, as "Positive: 80%, Negative: 20%." The emotion engine uses text analysis and machine learning algorithms to accurately analyze emotions.

[0920] Feedback analysis and system improvement

[0921] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, subsequent data collection and filtering processes are adjusted to improve overall system performance and reliability. For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm is introduced to resolve the issue.

[0922] The following describes the processing flow.

[0923] Step 1: Data Collection

[0924] The server periodically runs scripts to collect the latest articles and information from news sites and various databases. It retrieves data from specified URLs and APIs, and stores the collected data in temporary storage.

[0925] Step 2: Filtering

[0926] The server applies natural language processing algorithms to the temporarily stored data. The algorithms detect personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images), and delete or anonymize the detected information. The filtered, secure data is then sent to the next step.

[0927] Step 3: Generating Data Characteristics

[0928] The server generates characteristics from the filtered data. In this step, features such as article genre (politics, economics, sports, etc.), age group, author, and keywords are extracted. The characteristic data is compiled as statistical information, and the distribution is displayed, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%." The generated characteristic data is stored in a database.

[0929] Step 4: Selecting Sample Data

[0930] The server selects representative sample data based on the generated characteristic data. This selection includes random sampling and sampling based on specific criteria. The sample data is formatted and stored along with the characteristic information.

[0931] Step 5: Generating sample data

[0932] The server converts sample data selected based on characteristic information into an appropriate format (e.g., JSON, XML). This prepares the sample dataset for public release.

[0933] Step 6: Disclosure of Samples and Characteristics

[0934] The server configures a web page or API endpoint to expose characteristic data and sample data. This allows users to access this information using their devices and verify the data's transparency.

[0935] Step 7: Collecting user ratings and feedback

[0936] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[0937] Step 8: Analysis using the emotion engine

[0938] The server applies a sentiment engine to the collected ratings and feedback. The sentiment engine uses text analysis and machine learning algorithms to analyze the user's emotions (positive, negative, neutral, etc.) and generate sentiment data.

[0939] Step 9: Feedback Analysis and System Improvement

[0940] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, it adjusts the next data collection and filtering process to improve the overall system performance and reliability. For example, if a large number of negative sentiments are reported, it identifies the underlying data or algorithmic problems and implements improvements.

[0941] (Example 2)

[0942] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0943] Modern large-scale language models face numerous concerns regarding the transparency and quality of training data. These include issues such as the leakage of personal information, copyright infringement, and even the generation of fake news. Furthermore, analyzing user feedback and incorporating it into system improvements presents a significant challenge. New approaches are needed to effectively address these problems.

[0944] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0945] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, and means for performing sentiment analysis on the collected feedback.

[0946] This will allow us to protect personal information and copyrights while improving data transparency and quality, and to efficiently analyze user feedback to improve the system.

[0947] "Data collection" is the process of automatically obtaining articles and information from news sites, databases, and other sources on the internet.

[0948] "Data filtering" is the process of detecting and deleting or anonymizing personal information and copyrighted material from collected data.

[0949] "Natural language processing" is a general term for the technologies and methods that enable computers to understand and analyze human language.

[0950] "Characteristic generation" is the process of extracting features such as article genre, year, author, and keywords from filtered data and compiling them into statistical information.

[0951] "Sample data generation" is the process of selecting representative data based on the generated characteristics and formatting it for public release.

[0952] "Data publication" refers to the process of providing generated characteristic data and sample data to users through web pages or APIs.

[0953] "Collecting user ratings and feedback" is the process of collecting information from users who provide ratings and comments on publicly available characteristic data and sample data.

[0954] "Sentiment analysis" is the process of using an emotion engine to analyze the emotions behind collected user ratings and feedback.

[0955] "System improvement" is the process of improving datasets and filtering algorithms based on collected feedback and sentiment analysis results.

[0956] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[0957] Overview of program processing

[0958] The server collects data from news sites and databases on the internet, and filters the collected data to remove or anonymize personal information and copyrighted material. This filtering uses web scraping libraries such as BeautifulSoup and Scrapy in Python, as well as the entity recognition function of spaCy. After filtering, characteristics are generated from the data using techniques such as topic modeling (LDA) and keyword extraction (TF-IDF), and sample data is selected and generated based on these characteristics.

[0959] The generated sample and characteristic data are published via web pages and APIs using web frameworks such as Django and Flask. Users can access this information using their devices and verify the data's transparency. Evaluations and feedback are provided via online forms and APIs.

[0960] The server collects user feedback and analyzes user emotions using an emotion engine (such as a BERT model or fastText). Based on this analysis, the dataset and filtering algorithm are improved. For example, the filtering algorithm is adjusted based on feedback that contains a lot of negative emotions, and this is reflected in the next data collection.

[0961] Specific example

[0962] As a concrete example, input the following prompt into the generation AI model.

[0963] Example of a prompt:

[0964] "Collect the latest news articles on politics and economics, and remove any personal information or copyrighted material. Then, extract the article genres and keywords as characteristics, and generate three representative sample data based on these characteristics. Finally, publish the generated data on a webpage and collect and analyze sentiment feedback from users."

[0965] Based on this prompt, the system executes specific steps such as data collection, filtering, characteristic generation, sample data generation, data publication, evaluation collection, and sentiment analysis. This allows for system improvement by effectively utilizing user feedback while maintaining data transparency and quality.

[0966] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0967] The program's processing is explained step by step.

[0968] Step 1: Data Collection

[0969] explanation

[0970] The server collects the latest articles and information from news sites and databases on the internet. This task is performed periodically using specific scripts (e.g., BeautifulSoup or Scrapy in Python).

[0971] Specific actions

[0972] The server accesses a specified list of URLs or API endpoints, parses the HTML structure of the page, and retrieves the necessary data. For example, it can extract the titles and content of the latest articles from a news website's URL.

[0973] Input and output

[0974] Input: A specified list of URLs or API endpoints

[0975] Output: Collected raw HTML or JSON data

[0976] Step 2: Data Filtering

[0977] explanation

[0978] The server detects and removes or anonymizes personal information and copyrighted material from the collected data. Natural language processing (NLP) algorithms are used for this process.

[0979] Specific actions

[0980] For example, spaCy's entity recognition feature is used to identify personal names and addresses. If a name or address is detected in the data, it is replaced with "[ANONYMIZED]".

[0981] Input and output

[0982] Input: Collected raw data (HTML or JSON)

[0983] Output: Text data with personal information and copyright protection information removed or anonymized.

[0984] Step 3: Generating Data Characteristics

[0985] explanation

[0986] The server generates characteristics such as article genre, year, author, and keywords from the filtered data.

[0987] Specific actions

[0988] For example, to classify the genre of articles, techniques such as topic modeling (LDA) and keyword extraction (TF-IDF) are used. Characteristic data is generated, such as the fact that political articles account for 40% of the total.

[0989] Input and output

[0990] Input: Filtered text data

[0991] Output: Characteristic data such as genre, era, author, and keywords.

[0992] Step 4: Generating sample data

[0993] explanation

[0994] The server selects representative sample data based on its characteristics and formats it.

[0995] Specific actions

[0996] For example, sample data is selected using methods such as random sampling based on the proportion of specific characteristics. This sample data is then formatted for public release.

[0997] Input and output

[0998] Input: Characteristic data

[0999] Output: Sample dataset

[1000] Step 5: Disclosure of Samples and Characteristics

[1001] explanation

[1002] The server publishes the generated characteristic data and sample data through a web page or API.

[1003] Specific actions

[1004] For example, you can create a web page using a web framework such as Django or Flask and display characteristic data and sample data.

[1005] Input and output

[1006] Input: Sample dataset and characteristic data

[1007] Output: A web page or API endpoint exposed to users.

[1008] Step 6: Collecting user ratings and feedback

[1009] explanation

[1010] Users use their devices to view publicly available characteristic data and sample data, and provide evaluations and feedback on them. This information is sent to the server.

[1011] Specific actions

[1012] For example, you could place a rating form on a webpage, allowing users to enter scores and comments.

[1013] Input and output

[1014] Input: User-submitted ratings and feedback

[1015] Output: Evaluation and feedback data stored on the server

[1016] Step 7: Analysis using the emotion engine

[1017] explanation

[1018] The server uses an emotion engine to analyze collected ratings and feedback and classify the user's emotions.

[1019] Specific actions

[1020] For example, sentiment engines such as the BERT model or fastText are used to classify each feedback comment as either "positive" or "negative."

[1021] Input and output

[1022] Input: Rating and feedback data collected from users

[1023] Output: Sentiment analysis results (Example: "Positive: 80%, Negative: 20%")

[1024] Step 8: Feedback analysis and system improvement

[1025] explanation

[1026] The server improves the dataset and filtering algorithms based on feedback results, including sentiment analysis.

[1027] Specific actions

[1028] For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm will be implemented to address this issue and applied to the next data collection.

[1029] Input and output

[1030] Input: Sentiment analysis results and feedback data

[1031] Output: Improved filtering algorithm and dataset

[1032] By repeating these processing steps, the overall system performance and reliability are improved, and data transparency and user experience are optimized.

[1033] (Application Example 2)

[1034] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1035] In modern large-scale language model training data, data transparency and reliability are not sufficiently ensured, making it difficult for users to evaluate data quality. Furthermore, in addition to copyright and personal data protection issues, there is a growing risk of fake news generation and data corruption. Moreover, the lack of collection and analysis of feedback that considers user sentiment leads to insufficient improvement of datasets and systems. A new system is needed to address these challenges.

[1036] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1037] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for analyzing user sentiment based on the evaluations and feedback, means for improving the dataset and system based on the sentiment analysis results, means for evaluating reliability, and means for improving the dataset using the evaluation results. This increases the transparency of training data for large-scale language models, allows users to evaluate the quality and reliability of the data, reduces the risk of fake news and data collapse, and improves the overall system.

[1038] "Means of data collection" refers to the function of obtaining data from various information sources on the internet.

[1039] "Means for filtering data and removing personal and copyrighted information" refers to functions that detect and remove or anonymize personal and copyrighted information contained in collected data.

[1040] "Methods for generating characteristics from filtered data" refers to functions that extract characteristics such as article genre, author name, and keywords from filtered data, and analyze the characteristics of the data.

[1041] "Means for selecting and generating sample data based on characteristics" refers to a function for selecting representative data based on the generated characteristics, formatting that data, and providing it as a sample dataset.

[1042] "Means for publishing generated characteristic data and sample data" refers to functions for making characteristic data and sample data publicly available through web pages or APIs.

[1043] "Means for collecting user ratings and feedback" refers to functions for collecting user ratings and feedback on publicly available data via online forms or APIs.

[1044] "Means for analyzing user emotions based on evaluations and feedback" refers to a function that performs sentiment analysis on collected feedback and evaluations to classify users' positive and negative emotions.

[1045] "Means for improving datasets and systems based on sentiment analysis results" refers to functions that improve datasets and system algorithms based on information obtained from sentiment analysis results, thereby improving the overall performance of the system.

[1046] A "means for evaluating reliability" refers to a function that analyzes user evaluations of publicly available data and its characteristics, and evaluates the reliability of that data.

[1047] "Means for improving datasets using evaluation results" refers to functions that improve the quality of datasets by reviewing the data filtering process and sample data selection method based on the analysis results of user evaluations.

[1048] This invention specifically illustrates a form of quality evaluation application for e-commerce websites. This system has functions for data collection, filtering, characteristic generation, sample data generation, publication, evaluation, and feedback collection and analysis.

[1049] Data acquisition methods

[1050] The server collects data from news sites and various databases on the internet. This collection process is performed periodically, and data is retrieved from specified URLs and APIs using specific scripts. Libraries such as BeautifulSoup and requests can be used for this process.

[1051] Filtering means

[1052] To remove personal information and copyrighted material from collected data, the server uses natural language processing algorithms. Collected data is stored in temporary storage, where natural language processing algorithms are used to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted material. This ensures data integrity and privacy.

[1053] Characteristic generation means

[1054] From the filtered data, the server extracts characteristics such as article genre, author name, and keywords. This characteristic generation process uses text analysis algorithms and machine learning models. For example, characteristics distributed as "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%" are displayed as statistical information.

[1055] Method for generating sample data

[1056] Based on characteristic data, the server selects representative sample data. This selection includes random sampling and selection based on specific criteria. The selected sample data is verified and formatted along with characteristic information. This dataset is then prepared for public release.

[1057] means of disclosure

[1058] The server configures a webpage or API to publish characteristic and sample data. Users can access this information using their devices and verify the data's transparency. For example, users can view product descriptions and reviews and evaluate their quality.

[1059] User evaluation and feedback collection methods

[1060] Users evaluate publicly available characteristic data and sample data and provide feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs. Evaluation feedback forms or survey systems can be used for this data collection process.

[1061] Emotion analysis means

[1062] The server applies a sentiment engine to the collected ratings and feedback to analyze the user's emotions. Using text analysis and machine learning algorithms, it identifies the emotions behind the feedback and classifies them, such as "Positive: 80%, Negative: 20%". This sentiment analysis utilizes the pipeline function from the Hugging Face Transformers library.

[1063] Dataset and system improvement means

[1064] The server identifies areas for improvement in the dataset and system based on feedback analysis results, including sentiment analysis results. Based on the analysis results, the next data collection and filtering processes are adjusted, thereby improving the overall system performance and reliability.

[1065] Reliability evaluation method

[1066] To assess reliability, the server analyzes user reviews of the published data and its characteristics. These review results are used to determine the quality and reliability of the data.

[1067] Dataset improvement methods using evaluation results

[1068] Based on the evaluation results, the server will review its data filtering process and sample data selection methods to improve the dataset. This will enhance the quality of the dataset.

[1069] Specific example:

[1070] When analyzing the sentiment of feedback using the prompt sentence, "This product is very good. The delivery was fast, and the quality was as expected," the sentiment analysis results are classified as "Positive: 80%, Negative: 20%." Based on these results, the dataset and system are improved.

[1071] Hardware and software to be used:

[1072] Server: Performs data collection, filtering, characteristic generation, sample data generation and publication, sentiment analysis, and feedback analysis.

[1073] Terminal: Users view characteristic data and sample data, and provide evaluations and feedback.

[1074] BeautifulSoup, requests (data collection)

[1075] Natural language processing algorithms (filtering)

[1076] Text analysis algorithms, machine learning models (characteristic generation)

[1077] Hugging Face Transformers Library (Emotion Analysis)

[1078] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1079] Step 1:

[1080] The server collects data from news sites and databases on the internet. Input is a specified URL or API, and the requests library is used to retrieve the data. Specifically, it periodically runs a script to retrieve the latest articles and information, parses the HTML content using the BeautifulSoup library, and extracts it as text data. The output is the collected raw data.

[1081] Step 2:

[1082] The server filters the collected data, removing personal information and copyrighted material. The input is the raw data collected in step 1, and natural language processing algorithms are used to detect personal information (e.g., name, address, phone number) and copyrighted material. Specifically, regular expressions and machine learning models are used to extract the relevant information and then remove or anonymize it. The output is the clean data after filtering.

[1083] Step 3:

[1084] The server generates characteristics from the filtered data. The input is the data filtered in step 2, and based on this, it extracts characteristics such as article genre, author name, and keywords. Specifically, it uses text analysis algorithms and machine learning models to analyze the data's features and display them as statistical information. The output is characteristic data.

[1085] Step 4:

[1086] The server selects and generates sample data based on characteristics. The input is the characteristic data generated in step 3, and representative sample data is selected based on this characteristic data. Specifically, it uses random sampling or selection methods based on specific criteria, and then formats the selected data. The output is a sample dataset.

[1087] Step 5:

[1088] The server publishes the generated characteristic data and sample data. The input is the characteristic data and sample dataset generated in step 4, which are then published via a web page or API. Specifically, the server formats the data and publishes it on the web page or API in a format that is easily accessible to users. The output is the published characteristic data and sample data.

[1089] Step 6:

[1090] Users view publicly available characteristic and sample data and provide evaluations and feedback. Input consists of data published on web pages or via APIs, and users evaluate it using scores or comments. Specifically, they input feedback using evaluation feedback forms or survey systems and send it online to the server. Output consists of user evaluations and feedback.

[1091] Step 7:

[1092] The server analyzes emotions based on user ratings and feedback. The input is the feedback data collected in step 6, and the emotion engine is used to classify positive and negative emotions. Specifically, it uses the pipeline function of the Hugging Face Transformers library to perform text analysis and classify emotions. The output is the emotion analysis result.

[1093] Step 8:

[1094] The server improves the dataset and system based on the sentiment analysis results. The input is the sentiment analysis results generated in step 7, and the server adjusts the next data collection and filtering process based on the analysis results. Specifically, it improves the overall system performance by analyzing the results, extracting feedback patterns, and introducing new algorithms. The output is the improved dataset and system settings.

[1095] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1096] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1097] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1098] [Fourth Embodiment]

[1099] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1100] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1101] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1102] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1103] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1104] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1105] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1106] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1107] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1108] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1109] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1110] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1111] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1112] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption caused by generative AI, while taking into consideration copyright and personal data protection issues.

[1113] Overview of program processing

[1114] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and uses it to improve the dataset and system.

[1115] Program description in natural language

[1116] Data collection and filtering

[1117] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly. The collected data is temporarily stored and then filtered. Filtering uses natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images). This process helps to avoid legal issues.

[1118] Data property generation

[1119] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[1120] Generating sample data

[1121] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[1122] Disclosure of samples and characteristics

[1123] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[1124] Collecting user ratings and feedback

[1125] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments. The server collects these evaluations and feedback and performs analysis. The analysis results are used to improve the dataset and the system based on the collected evaluation data. User feedback is reflected in subsequent data collection and filtering processes, continuously improving the reliability and quality of the model.

[1126] The following describes the processing flow.

[1127] Step 1: Data Collection

[1128] The server collects the latest articles and information from news sites and various databases on the internet. A script for data collection is executed periodically, retrieving data from specified URLs and APIs. The collected data is stored in temporary storage.

[1129] Step 2: Filtering

[1130] The server applies natural language processing algorithms to the temporarily stored data to detect personal information (e.g., name, address, phone number) and copyrighted information. Detected personal information and copyrighted information are deleted or anonymized. The filtered data is stored for use in the next step.

[1131] Step 3: Generating Data Characteristics

[1132] The server generates characteristics from the filtered data. Specifically, it extracts features such as article genre (politics, economics, sports, etc.), age, author, and keywords. The generated characteristic data is compiled as statistical information and stored in a database.

[1133] Step 4: Selecting Sample Data

[1134] The server selects representative sample data based on the generated characteristic data. Random sampling is performed according to specific criteria, and appropriate samples are selected from each genre and characteristic. The selected sample data is stored.

[1135] Step 5: Generating sample data

[1136] The server formats the selected sample data and prepares it as a publicly available dataset. The sample data, along with its characteristic information, is converted into an appropriate format such as JSON or XML.

[1137] Step 6: Disclosure of Samples and Characteristics

[1138] The server publishes characteristic data and sample data through a configured web page or API endpoint. This data is made publicly available on the internet so that users can access it using their devices.

[1139] Step 7: Collecting user ratings and feedback

[1140] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores and comments and are sent to the server via online forms or APIs.

[1141] Step 8: Feedback analysis and system improvement

[1142] The server analyzes the collected feedback to identify areas for improvement in the dataset and filtering algorithms. Based on the analysis results, the next data collection and filtering processes are adjusted to improve overall system performance and reliability.

[1143] (Example 1)

[1144] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1145] Traditional large-scale language models have suffered from a lack of transparency, making it difficult for users to verify the quality and reliability of training data. Furthermore, there are risks of copyright infringement and personal data leaks from collected data, increasing the risk of fake news spread and data corruption through generative AI. Additionally, the lack of mechanisms to effectively collect user feedback and incorporate it into system improvements presents challenges in enhancing the reliability and quality of the models.

[1146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1147] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for periodically processing the collected data, and means for storing the collected evaluations and feedback in a database. This improves data transparency and reliability, and enables continuous improvement of the system by utilizing user feedback while taking into consideration copyright and personal information protection.

[1148] "Means of data collection" refers to methods for regularly obtaining data from information sources on the internet.

[1149] "Means for removing personal information and copyrighted information" refers to means for detecting personal information such as names and addresses, as well as copyrighted information, from collected data, and for removing or anonymizing them.

[1150] "Means for generating characteristics" refers to methods for automatically extracting characteristic information such as genre, era, and keywords from filtered data.

[1151] "Means for selecting and generating sample data" refers to means for selecting representative sample data based on the generated characteristics and formatting it.

[1152] "Means for publishing characteristic data and sample data" refers to means for publishing the generated characteristic data and sample data through web pages or APIs.

[1153] "Means for collecting user ratings and feedback" refers to means for collecting user ratings and feedback on publicly available data.

[1154] "Means for improving datasets and systems" refers to means for improving datasets and systems based on collected feedback.

[1155] "Means for periodically processing collected data" refers to means of regularly performing data collection and processing to ensure that the latest information is always available.

[1156] "Means for storing collected ratings and feedback in a database" refers to means for storing ratings and feedback collected from users in a database and making them available for subsequent processing.

[1157] This invention is a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data. At the same time, it aims to reduce the risk of fake news and data corruption generated by AI, while also considering copyright and personal data protection. This system is implemented with the following configuration and procedure.

[1158] 1. Data Collection

[1159] The server periodically collects data from news sites and various databases. For this purpose, it uses scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, when collecting the latest political news from a specific news site, the server uses "BeautifulSoup" to analyze the news site's HTML and extract article titles and body text.

[1160] 2. Data filtering

[1161] The collected data is temporarily stored, and the server uses natural language processing libraries such as "Apache OpenNLP" to filter out personal and copyrighted information. For example, it detects and deletes or anonymizes personal information such as names and addresses. It also detects and deletes or anonymizes copyrighted text and images.

[1162] 3. Generating Data Characteristics

[1163] Once the filtering is complete, the server generates characteristics from the data. This is done using tools like "Scikit-learn" and "NLTK". For example, it automatically classifies the genre of articles (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. The characteristics are displayed as statistical information, distributed as follows: "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%".

[1164] 4. Generating sample data

[1165] Based on its characteristics, the server selects representative sample data and formats it using tools like "Pandas" or "Numpy." For example, when creating training data for a generative AI model, it includes a balanced mix of articles from diverse genres based on specific criteria.

[1166] 5. Data Disclosure

[1167] The server uses web frameworks such as "Flask" or "Django" to expose the generated characteristic data and sample data as web pages or APIs. For example, the server uses "Flask" to build a web page to display the characteristic data and sample data, and sets up an API endpoint so that the data can be accessed from programs as well.

[1168] 6. Collecting user ratings and feedback

[1169] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection.

[1170] As a concrete example, to create training data for a generative AI model and to verify the transparency and reliability of the data, one might consider inputting prompt statements like the following into the generative AI model.

[1171] Example of a prompt:

[1172] "Collect the latest political news and remove personal information such as names and addresses. Then, extract characteristics such as the genre, year, and author of the articles and display them as statistical information."

[1173] This makes it possible to quickly build transparent and reliable datasets. This process ensures that the training data for generative AI models is of high quality and reliable, effectively reducing the risk of fake news and data collapse.

[1174] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1175] Step 1: Data Collection

[1176] The server collects the latest articles and information from news sites and various databases. The collected data includes text data, image data, etc. This step utilizes scraping tools such as "BeautifulSoup" and "Scrapy," or APIs. For example, the server uses "BeautifulSoup" to parse the HTML of news sites and extract article titles and body text. A list of URLs is provided as input, and the collected raw data (HTML) is obtained as output.

[1177] Step 2: Data Filtering

[1178] The server temporarily stores the collected data and then performs a filtering process. This step uses natural language processing libraries such as "Apache OpenNLP". The server detects personal information and copyrighted information and removes or anonymizes them. Specifically, the server uses "Apache OpenNLP" to detect and remove personal information such as names and addresses. It also detects copyrighted text and images and removes or anonymizes them as well. Raw data collected is provided as input, and filtered, clean data is obtained as output.

[1179] Step 3: Generating Data Characteristics

[1180] The server generates characteristics from the filtered data. This step uses libraries such as "Scikit-learn" and "NLTK". For example, the server automatically classifies article genres (politics, economics, sports, etc.) and extracts metadata such as keywords, dates, and author information. Specifically, the server uses "Scikit-learn" to classify article genres and "NLTK" to extract keywords, dates, and author information. Filtered data is provided as input, and the generated characteristic data is obtained as output.

[1181] Step 4: Generating sample data

[1182] The server selects sample data based on the generated characteristics. This step uses data manipulation libraries such as "Pandas" and "Numpy." For example, the server uses "Pandas" to apply characteristic-based filters and select representative sample data. Then, it uses "Numpy" to format the data and build a dataset for publication. Characteristic data is provided as input, and the generated sample data is obtained as output.

[1183] Step 5: Data Release

[1184] The server uses web frameworks such as Flask or Django to expose the generated characteristic data and sample data as web pages or APIs. For example, the server might use Flask to build a web page to display the characteristic data and sample data, and configure an API endpoint to allow programmatic access to the data. Sample data is provided as input, and the exposed web pages or API endpoints are obtained as output.

[1185] Step 6: Collecting user ratings and feedback

[1186] Users can view publicly available data characteristics and sample data through their devices and provide evaluations and feedback. Evaluations are given in the form of scores or comments, which the server collects and analyzes. For example, if a user provides feedback that "there are few economic-related articles," the number of economic-related articles can be increased in the next data collection. User evaluations and feedback are provided as input, and the collected evaluation data and analysis results are obtained as output.

[1187] (Application Example 1)

[1188] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1189] In today's information society, fake news and data reliability are major problems. In particular, the quality and transparency of the source data are crucial when generating information using large-scale language models. Traditional methods lacked systems to enhance data transparency and enable users to evaluate reliability, making it difficult to handle data while considering personal information protection and copyright protection. Furthermore, there was a lack of means to effectively utilize user feedback to improve datasets and systems. In addition, there were no established methods for evaluating the quality of news data, anonymization, or publishing the generated data. This led to problems such as a decline in the accuracy of information and user trust.

[1190] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1191] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, means for evaluating the quality of news data based on the feedback, means for using a large-scale language model to generate article summaries, and means for storing and publishing the anonymized news data on the server. This makes it possible to enhance data transparency, provide a system that allows users to evaluate reliability, and improve the quality of news data while considering the protection of personal information and copyrights.

[1192] "Means of data collection" refers to the function of obtaining information from various information sources on the internet.

[1193] "Filtering" refers to the process of removing or anonymizing personal information or copyrighted material from collected data.

[1194] "Means for generating characteristics" refers to a function that extracts features such as genres and keywords from filtered data and forms characteristic information about the data.

[1195] "Means for selecting and generating sample data" refers to a function that selects and generates representative sample data based on its characteristics.

[1196] "Means for publishing generated characteristic data and sample data" refers to a function that makes characteristic data and sample data publicly available through a web page or API so that users can view them.

[1197] "Means for collecting user ratings and feedback" refers to functions for collecting ratings and opinions provided by users.

[1198] "Means for improving datasets and systems" refers to functions that improve datasets and systems based on collected feedback.

[1199] "Means for evaluating the quality of news data" refers to a function for evaluating the reliability and quality of news data based on feedback.

[1200] "Methods using large-scale language models" refer to algorithms that use natural language processing techniques to summarize and classify news articles.

[1201] "Means for storing and publishing news data generated through anonymization processing on a server" refers to a function that deletes or anonymizes personal information or identifiable information, then stores the news data on a server and makes it available for user access.

[1202] A specific embodiment of this invention will now be described. This system is implemented as a smartphone application called "News Reader" that allows users to view the latest news and evaluate its reliability and quality.

[1203] System Configuration

[1204] 1. Data acquisition methods

[1205] The server automatically collects information from various sources on the internet. This collection process utilizes web scraping techniques and the Requests library. Data sources include news sites and public databases.

[1206] 2. Filtering means

[1207] The server filters the collected data, removing or anonymizing personal information and copyrighted material. This filtering uses natural language processing algorithms (e.g., BeautifulSoup and the re library). The filtered data is temporarily stored in a database.

[1208] 3. Characteristic generation means

[1209] The server generates characteristics from the filtered data. These characteristics include article genres (e.g., politics, economics, sports), keywords, author names, and dates. A machine learning model (e.g., HuggingFace's Transformers library) is used for this characteristic generation.

[1210] 4. Sample data selection and generation methods

[1211] Representative sample data is selected and generated based on its characteristics. Random sampling techniques or conditional selection algorithms are used to select samples based on specific criteria. The selected data is formatted and prepared for publication.

[1212] 5. Means for publishing characteristic data and sample data

[1213] The server publishes characteristic and sample data via a webpage or API. Users can access this information using the smartphone app "News Reader." The data is published in JSON format and visualized for easy user understanding.

[1214] 6. Means of collecting user evaluations and feedback

[1215] Users can view published data through the "News Reader" and provide ratings and feedback. Ratings can be given in the form of stars or comments, and the collected feedback is stored in a database.

[1216] 7. Dataset and System Improvement Means

[1217] The server analyzes the collected feedback and uses it to improve the dataset and the system. This analysis employs statistical methods and machine learning algorithms.

[1218] 8. Methods for evaluating the quality of news data

[1219] The quality of news data is evaluated based on user feedback. Evaluation criteria include article length, keyword score, and user confidence. Data analysis tools are used to assess data quality.

[1220] 9. Methods using large-scale language models

[1221] The server uses a large-scale language model (e.g., HuggingFace's Transformers) to summarize and classify news articles. This model generates article summaries, allowing users to quickly grasp the content.

[1222] 10. Anonymization and Data Disclosure Methods

[1223] The data is anonymized, stored on a server, and then made public. The anonymization process uses filtering algorithms to remove personal and identifying information.

[1224] Examples of specific cases and prompt statements

[1225] For example, if an article contains the sentence, "John Smith has moved to a new address, 123 Main St," this text will be filtered to "John Smith has moved to a new address." Furthermore, summarizing this sentence would result in "John is moving to a new residence."

[1226] Examples of prompts for a generative AI model:

[1227] Please summarize the following news article:

[1228] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[1229] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1230] Step 1:

[1231] Data collection

[1232] The server automatically collects data from news sites and public databases on the internet. Input is web article URLs or RSS feeds, and output is the text data of the web pages. Specifically, it uses the Requests library to retrieve the HTML content of web pages and BeautifulSoup to extract the text portion of the articles.

[1233] Step 2:

[1234] filtering

[1235] The server filters out personal and copyright-protected information from the collected text data. The input is the text data of the web page obtained in step 1, and the output is the filtered text data. Specifically, it uses Natural Language Processing (NLP) algorithms and regular expressions (re library) to detect and remove or anonymize personal information such as names and addresses.

[1236] Step 3:

[1237] Characteristic generation

[1238] The server generates characteristics from the filtered data. The input is the text data filtered in step 2, and the output is characteristic information (e.g., genre, keywords, publication year, etc.). Specifically, a machine learning model (HuggingFace's Transformers library) is used to extract keywords and classify genres.

[1239] Step 4:

[1240] Selection and generation of sample data

[1241] The server selects and generates representative sample data based on the characteristics. The input is the characteristic information and filtered text data generated in step 3, and the output is the selected and generated sample data. Specifically, random sampling techniques and condition-based selection algorithms are used to select data that matches the characteristics.

[1242] Step 5:

[1243] Disclosure of sample data and characteristic information

[1244] The server publishes sample data and characteristic information. The input is the sample data and characteristic information generated in step 4, and the output is a data file (in JSON format) for publication. Specifically, you configure the server or API to allow users to access this data.

[1245] Step 6:

[1246] Collecting user ratings and feedback

[1247] Users view the published sample data and characteristic information and provide ratings and feedback. The input is the data published in step 5, and the output is user ratings and feedback information. Specifically, star ratings and text comments are collected using the rating form and comment function within the app.

[1248] Step 7:

[1249] Dataset and system improvements

[1250] The server analyzes the collected feedback and uses it to improve the dataset and system. The input is the user ratings and feedback information collected in step 6, and the output is the improved dataset and system settings. Specifically, statistical analysis tools and machine learning algorithms are used to analyze the feedback and reflect it in the next data collection and filtering process.

[1251] Step 8:

[1252] Quality assessment of news data

[1253] The server evaluates the quality of the news data based on feedback. The input is the user ratings and feedback information collected in step 6, and the output is the evaluation result. Specifically, it calculates factors such as article length, keyword score, and user confidence to generate an overall quality score.

[1254] Step 9:

[1255] Summarization and classification using large-scale language models

[1256] The server uses a generative AI model (HuggingFace's Transformers) to generate summaries of news articles. The input is the collected and filtered news article text, and the output is the summarized text. Specifically, a specified prompt sentence is input to the generative AI model to obtain a concise summary.

[1257] Example of a prompt:

[1258] Please summarize the following news article:

[1259] John Smith has moved to his new address, 123 Main St. He is enjoying his new living environment.

[1260] Step 10:

[1261] Anonymization and data disclosure

[1262] The server stores and publishes anonymized news data. The input is filtered news article text, and the output is anonymized text and a data file for publication. Specifically, a filtering algorithm is used to remove or mask personal information and identifying information, and a data file (in JSON format) for publication is generated and stored on the server.

[1263] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1264] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[1265] Overview of program processing

[1266] The server collects data from various internet sources (news sites, databases, etc.) and filters the collected data to remove or anonymize personal information and copyrighted material. It then generates characteristics from the filtered data and selects and generates sample data based on these characteristics. The generated sample data and characteristic data are made publicly available via web pages and APIs, allowing users to view them and provide evaluations and feedback. The server collects user feedback and analyzes user emotions using a sentiment engine. Based on these analysis results, improvements are made to the dataset and system.

[1267] Program description in natural language

[1268] Data collection and filtering

[1269] The server collects the latest articles and information from news sites and various databases. This collection process is performed regularly, using specific scripts to retrieve data from designated URLs and APIs. The collected data is stored in temporary storage. Next, a filtering process is performed, using natural language processing algorithms to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images).

[1270] Data property generation

[1271] After filtering the data, the server generates characteristics. This characteristic generation process includes steps to extract features such as article genre (politics, economics, sports, etc.), age group, author, and keywords. These characteristics are displayed as statistical information, distributed as, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%."

[1272] Generating sample data

[1273] Based on the characteristics, the server selects representative sample data. This includes random sampling and selection based on specific criteria. The selected sample data is further verified and formatted as a sample dataset along with characteristic information. This dataset is then prepared for public release.

[1274] Disclosure of samples and characteristics

[1275] The server configures a webpage or API to publish characteristic data and sample data. Users can access this information using their devices and verify the data's transparency. The published information is presented in a format that users can easily view and evaluate.

[1276] Collecting user ratings and feedback

[1277] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[1278] Analysis using an emotion engine

[1279] The server applies an emotion engine to the collected ratings and feedback to analyze the user's emotions. This emotion analysis identifies the emotions behind the feedback and classifies them, for example, as "Positive: 80%, Negative: 20%." The emotion engine uses text analysis and machine learning algorithms to accurately analyze emotions.

[1280] Feedback analysis and system improvement

[1281] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, subsequent data collection and filtering processes are adjusted to improve overall system performance and reliability. For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm is introduced to resolve the issue.

[1282] The following describes the processing flow.

[1283] Step 1: Data Collection

[1284] The server periodically runs scripts to collect the latest articles and information from news sites and various databases. It retrieves data from specified URLs and APIs, and stores the collected data in temporary storage.

[1285] Step 2: Filtering

[1286] The server applies natural language processing algorithms to the temporarily stored data. The algorithms detect personal information (e.g., names, addresses, phone numbers) and copyrighted information (e.g., copyrighted text, images), and delete or anonymize the detected information. The filtered, secure data is then sent to the next step.

[1287] Step 3: Generating Data Characteristics

[1288] The server generates characteristics from the filtered data. In this step, features such as article genre (politics, economics, sports, etc.), age group, author, and keywords are extracted. The characteristic data is compiled as statistical information, and the distribution is displayed, for example, "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%." The generated characteristic data is stored in a database.

[1289] Step 4: Selecting Sample Data

[1290] The server selects representative sample data based on the generated characteristic data. This selection includes random sampling and sampling based on specific criteria. The sample data is formatted and stored along with the characteristic information.

[1291] Step 5: Generating sample data

[1292] The server converts sample data selected based on characteristic information into an appropriate format (e.g., JSON, XML). This prepares the sample dataset for public release.

[1293] Step 6: Disclosure of Samples and Characteristics

[1294] The server configures a web page or API endpoint to expose characteristic data and sample data. This allows users to access this information using their devices and verify the data's transparency.

[1295] Step 7: Collecting user ratings and feedback

[1296] Users view publicly available characteristic data and sample data through their devices and provide evaluations and feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs.

[1297] Step 8: Analysis using the emotion engine

[1298] The server applies a sentiment engine to the collected ratings and feedback. The sentiment engine uses text analysis and machine learning algorithms to analyze the user's emotions (positive, negative, neutral, etc.) and generate sentiment data.

[1299] Step 9: Feedback Analysis and System Improvement

[1300] The server identifies areas for improvement in the dataset and filtering algorithms based on feedback analysis results, including sentiment analysis. Based on the analysis results, it adjusts the next data collection and filtering process to improve the overall system performance and reliability. For example, if a large number of negative sentiments are reported, it identifies the underlying data or algorithmic problems and implements improvements.

[1301] (Example 2)

[1302] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1303] Modern large-scale language models face numerous concerns regarding the transparency and quality of training data. These include issues such as the leakage of personal information, copyright infringement, and even the generation of fake news. Furthermore, analyzing user feedback and incorporating it into system improvements presents a significant challenge. New approaches are needed to effectively address these problems.

[1304] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1305] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for improving the dataset and system based on the collected feedback, and means for performing sentiment analysis on the collected feedback.

[1306] This will allow us to protect personal information and copyrights while improving data transparency and quality, and to efficiently analyze user feedback to improve the system.

[1307] "Data collection" is the process of automatically obtaining articles and information from news sites, databases, and other sources on the internet.

[1308] "Data filtering" is the process of detecting and deleting or anonymizing personal information and copyrighted material from collected data.

[1309] "Natural language processing" is a general term for the technologies and methods that enable computers to understand and analyze human language.

[1310] "Characteristic generation" is the process of extracting features such as article genre, year, author, and keywords from filtered data and compiling them into statistical information.

[1311] "Sample data generation" is the process of selecting representative data based on the generated characteristics and formatting it for public release.

[1312] "Data publication" refers to the process of providing generated characteristic data and sample data to users through web pages or APIs.

[1313] "Collecting user ratings and feedback" is the process of collecting information from users who provide ratings and comments on publicly available characteristic data and sample data.

[1314] "Sentiment analysis" is the process of using an emotion engine to analyze the emotions behind collected user ratings and feedback.

[1315] "System improvement" is the process of improving datasets and filtering algorithms based on collected feedback and sentiment analysis results.

[1316] This invention aims to provide a system that enhances the transparency of training data for large-scale language models, enabling users to evaluate the quality and reliability of the data, while also addressing copyright and personal data protection issues and reducing the risk of fake news and data corruption generated by AI. Furthermore, by combining it with an emotion engine that recognizes user emotions and improves the quality of feedback, it enables user sentiment analysis.

[1317] Overview of program processing

[1318] The server collects data from news sites and databases on the internet, and filters the collected data to remove or anonymize personal information and copyrighted material. This filtering uses web scraping libraries such as BeautifulSoup and Scrapy in Python, as well as the entity recognition function of spaCy. After filtering, characteristics are generated from the data using techniques such as topic modeling (LDA) and keyword extraction (TF-IDF), and sample data is selected and generated based on these characteristics.

[1319] The generated sample and characteristic data are published via web pages and APIs using web frameworks such as Django and Flask. Users can access this information using their devices and verify the data's transparency. Evaluations and feedback are provided via online forms and APIs.

[1320] The server collects user feedback and analyzes user emotions using an emotion engine (such as a BERT model or fastText). Based on this analysis, the dataset and filtering algorithm are improved. For example, the filtering algorithm is adjusted based on feedback that contains a lot of negative emotions, and this is reflected in the next data collection.

[1321] Specific example

[1322] As a concrete example, input the following prompt into the generation AI model.

[1323] Example of a prompt:

[1324] "Collect the latest news articles on politics and economics, and remove any personal information or copyrighted material. Then, extract the article genres and keywords as characteristics, and generate three representative sample data based on these characteristics. Finally, publish the generated data on a webpage and collect and analyze sentiment feedback from users."

[1325] Based on this prompt, the system executes specific steps such as data collection, filtering, characteristic generation, sample data generation, data publication, evaluation collection, and sentiment analysis. This allows for system improvement by effectively utilizing user feedback while maintaining data transparency and quality.

[1326] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1327] The program's processing is explained step by step.

[1328] Step 1: Data Collection

[1329] explanation

[1330] The server collects the latest articles and information from news sites and databases on the internet. This task is performed periodically using specific scripts (e.g., BeautifulSoup or Scrapy in Python).

[1331] Specific actions

[1332] The server accesses a specified list of URLs or API endpoints, parses the HTML structure of the page, and retrieves the necessary data. For example, it can extract the titles and content of the latest articles from a news website's URL.

[1333] Input and output

[1334] Input: A specified list of URLs or API endpoints

[1335] Output: Collected raw HTML or JSON data

[1336] Step 2: Data Filtering

[1337] explanation

[1338] The server detects and removes or anonymizes personal information and copyrighted material from the collected data. Natural language processing (NLP) algorithms are used for this process.

[1339] Specific actions

[1340] For example, spaCy's entity recognition feature is used to identify personal names and addresses. If a name or address is detected in the data, it is replaced with "[ANONYMIZED]".

[1341] Input and output

[1342] Input: Collected raw data (HTML or JSON)

[1343] Output: Text data with personal information and copyright protection information removed or anonymized.

[1344] Step 3: Generating Data Characteristics

[1345] explanation

[1346] The server generates characteristics such as article genre, year, author, and keywords from the filtered data.

[1347] Specific actions

[1348] For example, to classify the genre of articles, techniques such as topic modeling (LDA) and keyword extraction (TF-IDF) are used. Characteristic data is generated, such as the fact that political articles account for 40% of the total.

[1349] Input and output

[1350] Input: Filtered text data

[1351] Output: Characteristic data such as genre, era, author, and keywords.

[1352] Step 4: Generating sample data

[1353] explanation

[1354] The server selects representative sample data based on its characteristics and formats it.

[1355] Specific actions

[1356] For example, sample data is selected using methods such as random sampling based on the proportion of specific characteristics. This sample data is then formatted for public release.

[1357] Input and output

[1358] Input: Characteristic data

[1359] Output: Sample dataset

[1360] Step 5: Disclosure of Samples and Characteristics

[1361] explanation

[1362] The server publishes the generated characteristic data and sample data through a web page or API.

[1363] Specific actions

[1364] For example, you can create a web page using a web framework such as Django or Flask and display characteristic data and sample data.

[1365] Input and output

[1366] Input: Sample dataset and characteristic data

[1367] Output: A web page or API endpoint exposed to users.

[1368] Step 6: Collecting user ratings and feedback

[1369] explanation

[1370] Users use their devices to view publicly available characteristic data and sample data, and provide evaluations and feedback on them. This information is sent to the server.

[1371] Specific actions

[1372] For example, you could place a rating form on a webpage, allowing users to enter scores and comments.

[1373] Input and output

[1374] Input: User-submitted ratings and feedback

[1375] Output: Evaluation and feedback data stored on the server

[1376] Step 7: Analysis using the emotion engine

[1377] explanation

[1378] The server uses an emotion engine to analyze collected ratings and feedback and classify the user's emotions.

[1379] Specific actions

[1380] For example, sentiment engines such as the BERT model or fastText are used to classify each feedback comment as either "positive" or "negative."

[1381] Input and output

[1382] Input: Rating and feedback data collected from users

[1383] Output: Sentiment analysis results (Example: "Positive: 80%, Negative: 20%")

[1384] Step 8: Feedback analysis and system improvement

[1385] explanation

[1386] The server improves the dataset and filtering algorithms based on feedback results, including sentiment analysis.

[1387] Specific actions

[1388] For example, if an article that evokes negative emotions in many users fails to filter, a new algorithm will be implemented to address this issue and applied to the next data collection.

[1389] Input and output

[1390] Input: Sentiment analysis results and feedback data

[1391] Output: Improved filtering algorithm and dataset

[1392] By repeating these processing steps, the overall system performance and reliability are improved, and data transparency and user experience are optimized.

[1393] (Application Example 2)

[1394] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1395] In modern large-scale language model training data, data transparency and reliability are not sufficiently ensured, making it difficult for users to evaluate data quality. Furthermore, in addition to copyright and personal data protection issues, there is a growing risk of fake news generation and data corruption. Moreover, the lack of collection and analysis of feedback that considers user sentiment leads to insufficient improvement of datasets and systems. A new system is needed to address these challenges.

[1396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1397] In this invention, the server includes means for collecting data, means for filtering the collected data and removing personal information and copyrighted information, means for generating characteristics from the filtered data, means for selecting and generating sample data based on the characteristics, means for publishing the generated characteristic data and sample data, means for collecting evaluations and feedback from users, means for analyzing user sentiment based on the evaluations and feedback, means for improving the dataset and system based on the sentiment analysis results, means for evaluating reliability, and means for improving the dataset using the evaluation results. This increases the transparency of training data for large-scale language models, allows users to evaluate the quality and reliability of the data, reduces the risk of fake news and data collapse, and improves the overall system.

[1398] "Means of data collection" refers to the function of obtaining data from various information sources on the internet.

[1399] "Means for filtering data and removing personal and copyrighted information" refers to functions that detect and remove or anonymize personal and copyrighted information contained in collected data.

[1400] "Methods for generating characteristics from filtered data" refers to functions that extract characteristics such as article genre, author name, and keywords from filtered data, and analyze the characteristics of the data.

[1401] "Means for selecting and generating sample data based on characteristics" refers to a function for selecting representative data based on the generated characteristics, formatting that data, and providing it as a sample dataset.

[1402] "Means for publishing generated characteristic data and sample data" refers to functions for making characteristic data and sample data publicly available through web pages or APIs.

[1403] "Means for collecting user ratings and feedback" refers to functions for collecting user ratings and feedback on publicly available data via online forms or APIs.

[1404] "Means for analyzing user emotions based on evaluations and feedback" refers to a function that performs sentiment analysis on collected feedback and evaluations to classify users' positive and negative emotions.

[1405] "Means for improving datasets and systems based on sentiment analysis results" refers to functions that improve datasets and system algorithms based on information obtained from sentiment analysis results, thereby improving the overall performance of the system.

[1406] A "means for evaluating reliability" refers to a function that analyzes user evaluations of publicly available data and its characteristics, and evaluates the reliability of that data.

[1407] "Means for improving datasets using evaluation results" refers to functions that improve the quality of datasets by reviewing the data filtering process and sample data selection method based on the analysis results of user evaluations.

[1408] This invention specifically illustrates a form of quality evaluation application for e-commerce websites. This system has functions for data collection, filtering, characteristic generation, sample data generation, publication, evaluation, and feedback collection and analysis.

[1409] Data acquisition methods

[1410] The server collects data from news sites and various databases on the internet. This collection process is performed periodically, and data is retrieved from specified URLs and APIs using specific scripts. Libraries such as BeautifulSoup and requests can be used for this process.

[1411] Filtering means

[1412] To remove personal information and copyrighted material from collected data, the server uses natural language processing algorithms. Collected data is stored in temporary storage, where natural language processing algorithms are used to detect and remove or anonymize personal information (e.g., names, addresses, phone numbers) and copyrighted material. This ensures data integrity and privacy.

[1413] Characteristic generation means

[1414] From the filtered data, the server extracts characteristics such as article genre, author name, and keywords. This characteristic generation process uses text analysis algorithms and machine learning models. For example, characteristics distributed as "Politics: 40%, Economics: 30%, Sports: 20%, Other: 10%" are displayed as statistical information.

[1415] Method for generating sample data

[1416] Based on characteristic data, the server selects representative sample data. This selection includes random sampling and selection based on specific criteria. The selected sample data is verified and formatted along with characteristic information. This dataset is then prepared for public release.

[1417] means of disclosure

[1418] The server configures a webpage or API to publish characteristic and sample data. Users can access this information using their devices and verify the data's transparency. For example, users can view product descriptions and reviews and evaluate their quality.

[1419] User evaluation and feedback collection methods

[1420] Users evaluate publicly available characteristic data and sample data and provide feedback. Evaluations are conducted in the form of scores or comments and are sent to the server via online forms or APIs. Evaluation feedback forms or survey systems can be used for this data collection process.

[1421] Emotion analysis means

[1422] The server applies a sentiment engine to the collected ratings and feedback to analyze the user's emotions. Using text analysis and machine learning algorithms, it identifies the emotions behind the feedback and classifies them, such as "Positive: 80%, Negative: 20%". This sentiment analysis utilizes the pipeline function from the Hugging Face Transformers library.

[1423] Dataset and system improvement means

[1424] The server identifies areas for improvement in the dataset and system based on feedback analysis results, including sentiment analysis results. Based on the analysis results, the next data collection and filtering processes are adjusted, thereby improving the overall system performance and reliability.

[1425] Reliability evaluation method

[1426] To assess reliability, the server analyzes user reviews of the published data and its characteristics. These review results are used to determine the quality and reliability of the data.

[1427] Dataset improvement methods using evaluation results

[1428] Based on the evaluation results, the server will review its data filtering process and sample data selection methods to improve the dataset. This will enhance the quality of the dataset.

[1429] Specific example:

[1430] When analyzing the sentiment of feedback using the prompt sentence, "This product is very good. The delivery was fast, and the quality was as expected," the sentiment analysis results are classified as "Positive: 80%, Negative: 20%." Based on these results, the dataset and system are improved.

[1431] Hardware and software to be used:

[1432] Server: Performs data collection, filtering, characteristic generation, sample data generation and publication, sentiment analysis, and feedback analysis.

[1433] Terminal: Users view characteristic data and sample data, and provide evaluations and feedback.

[1434] BeautifulSoup, requests (data collection)

[1435] Natural language processing algorithms (filtering)

[1436] Text analysis algorithms, machine learning models (characteristic generation)

[1437] Hugging Face Transformers Library (Emotion Analysis)

[1438] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1439] Step 1:

[1440] The server collects data from news sites and databases on the internet. Input is a specified URL or API, and the requests library is used to retrieve the data. Specifically, it periodically runs a script to retrieve the latest articles and information, parses the HTML content using the BeautifulSoup library, and extracts it as text data. The output is the collected raw data.

[1441] Step 2:

[1442] The server filters the collected data, removing personal information and copyrighted material. The input is the raw data collected in step 1, and natural language processing algorithms are used to detect personal information (e.g., name, address, phone number) and copyrighted material. Specifically, regular expressions and machine learning models are used to extract the relevant information and then remove or anonymize it. The output is the clean data after filtering.

[1443] Step 3:

[1444] The server generates characteristics from the filtered data. The input is the data filtered in step 2, and based on this, it extracts characteristics such as article genre, author name, and keywords. Specifically, it uses text analysis algorithms and machine learning models to analyze the data's features and display them as statistical information. The output is characteristic data.

[1445] Step 4:

[1446] The server selects and generates sample data based on characteristics. The input is the characteristic data generated in step 3, and representative sample data is selected based on this characteristic data. Specifically, it uses random sampling or selection methods based on specific criteria, and then formats the selected data. The output is a sample dataset.

[1447] Step 5:

[1448] The server publishes the generated characteristic data and sample data. The input is the characteristic data and sample dataset generated in step 4, which are then published via a web page or API. Specifically, the server formats the data and publishes it on the web page or API in a format that is easily accessible to users. The output is the published characteristic data and sample data.

[1449] Step 6:

[1450] Users view publicly available characteristic and sample data and provide evaluations and feedback. Input consists of data published on web pages or via APIs, and users evaluate it using scores or comments. Specifically, they input feedback using evaluation feedback forms or survey systems and send it online to the server. Output consists of user evaluations and feedback.

[1451] Step 7:

[1452] The server analyzes emotions based on user ratings and feedback. The input is the feedback data collected in step 6, and the emotion engine is used to classify positive and negative emotions. Specifically, it uses the pipeline function of the Hugging Face Transformers library to perform text analysis and classify emotions. The output is the emotion analysis result.

[1453] Step 8:

[1454] The server improves the dataset and system based on the sentiment analysis results. The input is the sentiment analysis results generated in step 7, and the server adjusts the next data collection and filtering process based on the analysis results. Specifically, it improves the overall system performance by analyzing the results, extracting feedback patterns, and introducing new algorithms. The output is the improved dataset and system settings.

[1455] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1456] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1457] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1458] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1459] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1460] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1461] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1462] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1463] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1464] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1465] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1466] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1467] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1468] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1469] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1470] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1471] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1472] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1473] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1474] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1475] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1476] The following is further disclosed regarding the embodiments described above.

[1477] (Claim 1)

[1478] Means of collecting data,

[1479] A means of filtering the collected data and removing personal information and copyrighted information,

[1480] A means of generating characteristics from filtered data,

[1481] A means for selecting and generating sample data based on characteristics,

[1482] A means of publishing the generated characteristic data and sample data,

[1483] A means of collecting user ratings and feedback,

[1484] A system that includes means for improving the dataset and the system based on collected feedback.

[1485] (Claim 2)

[1486] The system according to claim 1, comprising means of using natural language processing to filter data.

[1487] (Claim 3)

[1488] The system according to claim 1, comprising means for analyzing the characteristics of the published data and user evaluations of the sample data as statistical information.

[1489] "Example 1"

[1490] (Claim 1)

[1491] Means of collecting data,

[1492] A means of filtering the collected data and removing personal information and copyrighted information,

[1493] A means of generating characteristics from filtered data,

[1494] A means for selecting and generating sample data based on characteristics,

[1495] A means of publishing the generated characteristic data and sample data,

[1496] A means of collecting user ratings and feedback,

[1497] Means for improving the dataset and system based on collected feedback,

[1498] A means of periodically processing the collected data,

[1499] A system that includes means for storing collected evaluations and feedback in a database.

[1500] (Claim 2)

[1501] The system according to claim 1, which uses natural language processing to filter data.

[1502] (Claim 3)

[1503] The system according to claim 1, which analyzes the characteristics of the published data and the user's evaluation of the sample data as statistical information.

[1504] "Application Example 1"

[1505] (Claim 1)

[1506] Means of collecting data,

[1507] A means of filtering the collected data and removing personal information and copyrighted information,

[1508] A means of generating characteristics from filtered data,

[1509] A means for selecting and generating sample data based on characteristics,

[1510] A means of publishing the generated characteristic data and sample data,

[1511] A means of collecting user ratings and feedback,

[1512] Means for improving the dataset and system based on collected feedback,

[1513] A means of evaluating the quality of news data based on feedback,

[1514] A method using a large-scale language model to generate article summaries,

[1515] A means of storing and publishing news data generated after anonymization on a server,

[1516] A system that includes this.

[1517] (Claim 2)

[1518] The system according to claim 1, comprising means of using natural language processing to filter data.

[1519] (Claim 3)

[1520] The system according to claim 1, comprising means for analyzing the characteristics of the published data and user evaluations of the sample data as statistical information.

[1521] "Example 2 of combining an emotion engine"

[1522] (Claim 1)

[1523] Means of collecting data,

[1524] A means of filtering the collected data and removing personal information and copyrighted information,

[1525] A means of generating characteristics from filtered data,

[1526] A means for selecting and generating sample data based on characteristics,

[1527] A means of publishing the generated characteristic data and sample data,

[1528] A means of collecting user ratings and feedback,

[1529] Means for improving the dataset and system based on collected feedback,

[1530] A system that includes means for performing sentiment analysis on collected feedback.

[1531] (Claim 2)

[1532] The system according to claim 1, comprising means of using natural language processing to filter data and means of using online forms or APIs to collect evaluations and feedback.

[1533] (Claim 3)

[1534] The system according to claim 1, comprising means for analyzing collected evaluations and feedback using an emotion engine and classifying the user's emotions.

[1535] "Application example 2 when combining with an emotional engine"

[1536] (Claim 1)

[1537] Means of collecting data,

[1538] A means of filtering the collected data and removing personal information and copyrighted information,

[1539] A means of generating characteristics from filtered data,

[1540] A means for selecting and generating sample data based on characteristics,

[1541] A means of publishing the generated characteristic data and sample data,

[1542] A means of collecting user ratings and feedback,

[1543] A means of analyzing user emotions based on evaluations and feedback,

[1544] A means for improving the dataset and system based on the results of sentiment analysis,

[1545] Means for evaluating reliability,

[1546] A means of improving the dataset using the evaluation results,

[1547] A system that includes this.

[1548] (Claim 2)

[1549] The system according to claim 1, comprising means of using natural language processing to filter data.

[1550] (Claim 3)

[1551] The system according to claim 1, comprising means for analyzing the characteristics of the published data and user evaluations of the sample data as statistical information. [Explanation of Symbols]

[1552] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting data, A means of filtering the collected data and removing personal information and copyrighted information, A means of generating characteristics from filtered data, A means for selecting and generating sample data based on characteristics, A means of publishing the generated characteristic data and sample data, A means of collecting user ratings and feedback, A system that includes means for improving the dataset and the system based on collected feedback.

2. The system according to claim 1, comprising means for using natural language processing to filter data.

3. The system according to claim 1, comprising means for analyzing the characteristics of the published data and user evaluations of the sample data as statistical information.

Images